PASS
coverage: 2.2% of statements in github.com/pingcap/tiflow/...
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.batch_update_to_no_batch.cli.59955.out cli changefeed resume -c test
PASS
coverage: 2.2% of statements in github.com/pingcap/tiflow/...
+ set +x
table batch_update_to_no_batch.v1 exists
table batch_update_to_no_batch.recover_and_insert not exists for 1-th check, retry later
table batch_update_to_no_batch.recover_and_insert not exists for 2-th check, retry later
table batch_update_to_no_batch.recover_and_insert not exists for 3-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
table batch_update_to_no_batch.recover_and_insert exists
table batch_update_to_no_batch.finish_mark not exists for 1-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
***************** properties *****************
"mysql.port"="4000"
"requestdistribution"="uniform"
"recordcount"="10"
"scanproportion"="0"
"insertproportion"="0"
"readallfields"="true"
"mysql.host"="127.0.0.1"
"updateproportion"="0"
"readproportion"="0"
"mysql.user"="root"
"mysql.db"="testSync"
"operationcount"="0"
"dotransactions"="false"
"workload"="core"
"threadcount"="2"
**********************************************
Run finished, takes 4.355536ms
INSERT - Takes(s): 0.0, Count: 10, OPS: 3753.2, Avg(us): 803, Min(us): 520, Max(us): 1669, 95th(us): 2000, 99th(us): 2000
table batch_update_to_no_batch.finish_mark not exists for 2-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
table batch_update_to_no_batch.finish_mark not exists for 3-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
table ddl_sequence.finish_mark not exists for 1-th check, retry later
table ddl_sequence.finish_mark not exists for 2-th check, retry later
table ddl_sequence.finish_mark not exists for 3-th check, retry later
table ddl_sequence.finish_mark not exists for 4-th check, retry later
table ddl_sequence.finish_mark not exists for 5-th check, retry later
table batch_update_to_no_batch.finish_mark not exists for 4-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
table batch_update_to_no_batch.finish_mark exists
check diff successfully
table ddl_sequence.finish_mark not exists for 6-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 15:55:50 CST 2024] <<<<<< run test case batch_update_to_no_batch success! >>>>>>
table ddl_sequence.finish_mark exists
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 15:55:51 CST 2024] <<<<<< run test case ddl_sequence success! >>>>>>
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
\033[0;36m<<< Run all test success >>>\033[0m
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-6.5-pull_cdc_integration_test-167/tiflow-cdc already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
Failed to start upstream TiDB
start tidb cluster failed
The 2 times to try to start tidb cluster...
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/bdr_mode/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
start tidb cluster in /tmp/tidb_cdc_test/bdr_mode
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
start tidb cluster in /tmp/tidb_cdc_test/changefeed_fast_fail
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d1e00002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60985, start at 2024-04-28 15:56:19.961712879 +0800 CST m=+6.221334422 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.975 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.960 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.960 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/bdr_mode/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/bdr_mode/tiflash/log/error.log
Starting CDC state checker...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ce00001b Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:55964, start at 2024-04-28 15:54:08.688883634 +0800 CST m=+11.339258367 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.689 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.689 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ce00001b Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:55964, start at 2024-04-28 15:54:08.688883634 +0800 CST m=+11.339258367 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.689 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.689 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ca2c0012 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:56028, start at 2024-04-28 15:54:08.410682214 +0800 CST m=+11.004434022 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.450 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.395 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.395 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/changefeed_fast_fail/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/changefeed_fast_fail/tiflash/log/error.log
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 3 times to try to start tidb cluster...
***************** properties *****************
"updateproportion"="0"
"scanproportion"="0"
"workload"="core"
"readproportion"="0"
"requestdistribution"="uniform"
"recordcount"="10"
"mysql.port"="4000"
"threadcount"="2"
"mysql.db"="testSync"
"insertproportion"="0"
"operationcount"="0"
"mysql.user"="root"
"readallfields"="true"
"mysql.host"="127.0.0.1"
"dotransactions"="false"
**********************************************
Run finished, takes 4.303848ms
INSERT - Takes(s): 0.0, Count: 10, OPS: 3758.3, Avg(us): 788, Min(us): 507, Max(us): 1651, 95th(us): 2000, 99th(us): 2000
start tidb cluster in /tmp/tidb_cdc_test/bdr_mode
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
start tidb cluster in /tmp/tidb_cdc_test/changefeed_fast_fail
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d1e00002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60985, start at 2024-04-28 15:56:19.961712879 +0800 CST m=+6.221334422 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.975 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.960 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.960 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ce00001b Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:55964, start at 2024-04-28 15:54:08.688883634 +0800 CST m=+11.339258367 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.689 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.689 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ce00001b Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:55964, start at 2024-04-28 15:54:08.688883634 +0800 CST m=+11.339258367 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.689 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.689 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ec9ca2c0012 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--d43s1, pid:56028, start at 2024-04-28 15:54:08.410682214 +0800 CST m=+11.004434022 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:57:08.450 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:54:08.395 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:44:08.395 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.changefeed_fast_fail.cli.60295.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449391104042991617
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449391104042991617 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[2024/04/28 15:57:03.484 +08:00] [WARN] [retry_interceptor.go:62] ["retrying of unary invoker failed"] [target=etcd-endpoints://0xc000d5ca80/127.0.0.1:2379] [attempt=0] [error="rpc error: code = DeadlineExceeded desc = context deadline exceeded"]
start tidb cluster in /tmp/tidb_cdc_test/bdr_mode
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d0e40014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60914, start at 2024-04-28 15:56:19.940603484 +0800 CST m=+6.273700751 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.955 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.946 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.946 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8ed1d1e00002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--2v4jl, pid:60985, start at 2024-04-28 15:56:19.961712879 +0800 CST m=+6.221334422 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-15:58:19.975 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:56:19.960 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:46:19.960 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
[Sun Apr 28 15:57:15 CST 2024] <<<<<< START cdc server in bdr_mode case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ GO_FAILPOINTS=
+ [[ no != \n\o ]]
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.bdr_mode.6464064642.out server --log-file /tmp/tidb_cdc_test/bdr_mode/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/bdr_mode/cdc_data --cluster-id default
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 28 Apr 2024 07:57:19 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/cffa5427-47eb-4b86-a697-2674bff45264
{"id":"cffa5427-47eb-4b86-a697-2674bff45264","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b5254990
cffa5427-47eb-4b86-a697-2674bff45264
/tidb/cdc/default/default/upstream/7362823653182852390
{"id":7362823653182852390,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/cffa5427-47eb-4b86-a697-2674bff45264
{"id":"cffa5427-47eb-4b86-a697-2674bff45264","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b5254990
cffa5427-47eb-4b86-a697-2674bff45264
/tidb/cdc/default/default/upstream/7362823653182852390
{"id":7362823653182852390,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/cffa5427-47eb-4b86-a697-2674bff45264
{"id":"cffa5427-47eb-4b86-a697-2674bff45264","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b5254990
cffa5427-47eb-4b86-a697-2674bff45264
/tidb/cdc/default/default/upstream/7362823653182852390
{"id":7362823653182852390,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
[Sun Apr 28 15:57:19 CST 2024] <<<<<< START cdc server in bdr_mode case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ GO_FAILPOINTS=
+ '[' -z '' ']'
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.bdr_mode.6470164703.out server --log-file /tmp/tidb_cdc_test/bdr_mode/cdcdown.log --log-level debug --data-dir /tmp/tidb_cdc_test/bdr_mode/cdc_datadown --cluster-id default --addr 127.0.0.1:8400 --pd http://127.0.0.1:2479
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8400/debug/info'
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8400/debug/info
* About to connect() to 127.0.0.1 port 8400 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8400; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8400/debug/info
* About to connect() to 127.0.0.1 port 8400 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8400 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8400
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 28 Apr 2024 07:57:22 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/a7f01f21-8417-43c7-b5e4-75b09be5af7f
{"id":"a7f01f21-8417-43c7-b5e4-75b09be5af7f","address":"127.0.0.1:8400","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/35118f23b5258b5d
a7f01f21-8417-43c7-b5e4-75b09be5af7f
/tidb/cdc/default/default/upstream/7362823650803485598
{"id":7362823650803485598,"pd-endpoints":"http://127.0.0.1:2479,http://127.0.0.1:2479","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/a7f01f21-8417-43c7-b5e4-75b09be5af7f
{"id":"a7f01f21-8417-43c7-b5e4-75b09be5af7f","address":"127.0.0.1:8400","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/35118f23b5258b5d
a7f01f21-8417-43c7-b5e4-75b09be5af7f
/tidb/cdc/default/default/upstream/7362823650803485598
{"id":7362823650803485598,"pd-endpoints":"http://127.0.0.1:2479,http://127.0.0.1:2479","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/a7f01f21-8417-43c7-b5e4-75b09be5af7f
{"id":"a7f01f21-8417-43c7-b5e4-75b09be5af7f","address":"127.0.0.1:8400","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/35118f23b5258b5d
a7f01f21-8417-43c7-b5e4-75b09be5af7f
/tidb/cdc/default/default/upstream/7362823650803485598
{"id":7362823650803485598,"pd-endpoints":"http://127.0.0.1:2479,http://127.0.0.1:2479","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
table testSync.usertable exists
table testSync.simple1 exists
table testSync.simple2 exists
[Sun Apr 28 15:57:53 CST 2024] <<<<<< START cdc server in changefeed_fast_fail case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ GO_FAILPOINTS='github.com/pingcap/tiflow/cdc/owner/InjectChangefeedFastFailError=return(true)'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.changefeed_fast_fail.6040260404.out server --log-file /tmp/tidb_cdc_test/changefeed_fast_fail/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/changefeed_fast_fail/cdc_data --cluster-id default
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 28 Apr 2024 07:57:56 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c266be0e-f12d-4888-8bb1-5a4baa35d971
{"id":"c266be0e-f12d-4888-8bb1-5a4baa35d971","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b4d56bfa
c266be0e-f12d-4888-8bb1-5a4baa35d971
/tidb/cdc/default/default/upstream/7362823043724076637
{"id":7362823043724076637,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c266be0e-f12d-4888-8bb1-5a4baa35d971
{"id":"c266be0e-f12d-4888-8bb1-5a4baa35d971","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b4d56bfa
c266be0e-f12d-4888-8bb1-5a4baa35d971
/tidb/cdc/default/default/upstream/7362823043724076637
{"id":7362823043724076637,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c266be0e-f12d-4888-8bb1-5a4baa35d971
{"id":"c266be0e-f12d-4888-8bb1-5a4baa35d971","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b4d56bfa
c266be0e-f12d-4888-8bb1-5a4baa35d971
/tidb/cdc/default/default/upstream/7362823043724076637
{"id":7362823043724076637,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.changefeed_fast_fail.cli.60459.out cli changefeed create --start-ts=449391104042991617 '--sink-uri=mysql://normal:123456@127.0.0.1:3306/?max-txn-row=1' -c changefeed-fast-fail
Create changefeed successfully!
ID: changefeed-fast-fail
Info: {"upstream_id":7362823043724076637,"namespace":"default","id":"changefeed-fast-fail","sink_uri":"mysql://normal:xxxxx@127.0.0.1:3306/?max-txn-row=1","create_time":"2024-04-28T15:57:57.09324776+08:00","start_ts":449391104042991617,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":false,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-2-ge6698aa29","resolved_ts":449391104042991617,"checkpoint_ts":449391104042991617,"checkpoint_time":"2024-04-28 15:56:55.789"}
PASS
coverage: 2.4% of statements in github.com/pingcap/tiflow/...
+ set +x
check_changefeed_state http://127.0.0.1:2379 changefeed-fast-fail failed ErrStartTsBeforeGC
+ endpoints=http://127.0.0.1:2379
+ changefeed_id=changefeed-fast-fail
+ expected_state=failed
+ error_msg=ErrStartTsBeforeGC
+ tls_dir=ErrStartTsBeforeGC
+ [[ http://127.0.0.1:2379 =~ https ]]
++ cdc cli changefeed query --pd=http://127.0.0.1:2379 -c changefeed-fast-fail -s
+ info='{
"upstream_id": 7362823043724076637,
"namespace": "default",
"id": "changefeed-fast-fail",
"state": "failed",
"checkpoint_tso": 0,
"checkpoint_time": "1970-01-01 08:00:00.000",
"error": {
"time": "2024-04-28T15:57:57.098749091+08:00",
"addr": "127.0.0.1:8300",
"code": "CDC:ErrStartTsBeforeGC",
"message": "[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError"
}
}'
+ echo '{
"upstream_id": 7362823043724076637,
"namespace": "default",
"id": "changefeed-fast-fail",
"state": "failed",
"checkpoint_tso": 0,
"checkpoint_time": "1970-01-01 08:00:00.000",
"error": {
"time": "2024-04-28T15:57:57.098749091+08:00",
"addr": "127.0.0.1:8300",
"code": "CDC:ErrStartTsBeforeGC",
"message": "[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError"
}
}'
{
"upstream_id": 7362823043724076637,
"namespace": "default",
"id": "changefeed-fast-fail",
"state": "failed",
"checkpoint_tso": 0,
"checkpoint_time": "1970-01-01 08:00:00.000",
"error": {
"time": "2024-04-28T15:57:57.098749091+08:00",
"addr": "127.0.0.1:8300",
"code": "CDC:ErrStartTsBeforeGC",
"message": "[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError"
}
}
++ jq -r .state
++ echo '{' '"upstream_id":' 7362823043724076637, '"namespace":' '"default",' '"id":' '"changefeed-fast-fail",' '"state":' '"failed",' '"checkpoint_tso":' 0, '"checkpoint_time":' '"1970-01-01' '08:00:00.000",' '"error":' '{' '"time":' '"2024-04-28T15:57:57.098749091+08:00",' '"addr":' '"127.0.0.1:8300",' '"code":' '"CDC:ErrStartTsBeforeGC",' '"message":' '"[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError"' '}' '}'
+ state=failed
+ [[ ! failed == \f\a\i\l\e\d ]]
++ echo '{' '"upstream_id":' 7362823043724076637, '"namespace":' '"default",' '"id":' '"changefeed-fast-fail",' '"state":' '"failed",' '"checkpoint_tso":' 0, '"checkpoint_time":' '"1970-01-01' '08:00:00.000",' '"error":' '{' '"time":' '"2024-04-28T15:57:57.098749091+08:00",' '"addr":' '"127.0.0.1:8300",' '"code":' '"CDC:ErrStartTsBeforeGC",' '"message":' '"[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError"' '}' '}'
++ jq -r .error.message
+ message='[CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError'
+ [[ ! [CDC:ErrStartTsBeforeGC]InjectChangefeedFastFailError =~ ErrStartTsBeforeGC ]]
run task successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 15:57:58 CST 2024] <<<<<< run test case changefeed_fast_fail success! >>>>>>
\033[0;36m<<< Run all test success >>>\033[0m
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-6.5-pull_cdc_integration_test-167/tiflow-cdc already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
table bdr_mode.t1 exists
table bdr_mode.t1 exists
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.bdr_mode.cli.64852.out cli changefeed create --sink-uri=mysql://root@127.0.0.1:3306 -c test-1 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/bdr_mode/conf/up.toml
Create changefeed successfully!
ID: test-1
Info: {"upstream_id":7362823653182852390,"namespace":"default","id":"test-1","sink_uri":"mysql://root@127.0.0.1:3306","create_time":"2024-04-28T15:58:13.743620108+08:00","start_ts":449391124438843397,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":true,"bdr_mode":true,"sync_point_interval":30000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-2-ge6698aa29","resolved_ts":449391124438843397,"checkpoint_ts":449391124438843397,"checkpoint_time":"2024-04-28 15:58:13.593"}
PASS
coverage: 2.5% of statements in github.com/pingcap/tiflow/...
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.bdr_mode.cli.64899.out cli changefeed create --sink-uri=mysql://root@127.0.0.1:4000 -c test-2 --server http://127.0.0.1:8400 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/bdr_mode/conf/down.toml
Create changefeed successfully!
ID: test-2
Info: {"upstream_id":7362823650803485598,"namespace":"default","id":"test-2","sink_uri":"mysql://root@127.0.0.1:4000","create_time":"2024-04-28T15:58:14.318881032+08:00","start_ts":449391124600848388,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":true,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-2-ge6698aa29","resolved_ts":449391124600848388,"checkpoint_ts":449391124600848388,"checkpoint_time":"2024-04-28 15:58:14.211"}
PASS
coverage: 2.5% of statements in github.com/pingcap/tiflow/...
+ set +x
table tidb_cdc.syncpoint_v1 exists
table tidb_cdc.syncpoint_v1 does not exists
table bdr_mode.finish_mark exists
table bdr_mode.finish_mark exists
check diff failed 1-th time, retry later
check diff failed 2-th time, retry later
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 15:58:22 CST 2024] <<<<<< run test case bdr_mode success! >>>>>>
\033[0;36m<<< Run all test success >>>\033[0m
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-6.5-pull_cdc_integration_test-167/tiflow-cdc already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
____________________________________
*************************** 1. row ***************************
primary_ts: 449391068269248513
secondary_ts: 449391070630117390
*************************** 2. row ***************************
primary_ts: 449391076133568512
secondary_ts: 449391076410392577
*************************** 3. row ***************************
primary_ts: 449391083997888512
secondary_ts: 449391084300926977
*************************** 4. row ***************************
primary_ts: 449391091862208512
secondary_ts: 449391092165246977
*************************** 5. row ***************************
primary_ts: 449391099726528512
secondary_ts: 449391100029304833
*************************** 6. row ***************************
primary_ts: 449391107590848512
secondary_ts: 449391107893886977
*************************** 7. row ***************************
primary_ts: 449391115455168512
secondary_ts: 449391115758206977
*************************** 8. row ***************************
primary_ts: 449391123319488512
secondary_ts: 449391123386597377
skip invalid syncpoint primary_ts: 449391068269248513, first_ddl_ts: 449391070956486673
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 15:58:37 CST 2024] <<<<<< run test case syncpoint success! >>>>>>
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/tidb_mysql_test/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc086c000e Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75143, start at 2024-04-28 15:59:07.302575868 +0800 CST m=+4.291402144 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.309 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.291 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.291 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/tidb_mysql_test/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/tidb_mysql_test/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc086c000e Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75143, start at 2024-04-28 15:59:07.302575868 +0800 CST m=+4.291402144 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.309 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.291 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.291 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc06f00014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75044, start at 2024-04-28 15:59:07.223157806 +0800 CST m=+4.294847548 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.230 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.196 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.196 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8edc086c000e Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:75143, start at 2024-04-28 15:59:07.302575868 +0800 CST m=+4.291402144 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:01:07.309 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-15:59:07.291 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:49:07.291 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tidb_mysql_test.cli.79161.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449391151629729796
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449391151629729796 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[Sun Apr 28 15:59:57 CST 2024] <<<<<< START cdc server in tidb_mysql_test case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ GO_FAILPOINTS=
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tidb_mysql_test.7920179203.out server --log-file /tmp/tidb_cdc_test/tidb_mysql_test/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/tidb_mysql_test/cdc_data --cluster-id default
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 28 Apr 2024 08:00:00 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c0dfea5d-9159-4c80-9524-80decbab9c73
{"id":"c0dfea5d-9159-4c80-9524-80decbab9c73","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b79ab38e
c0dfea5d-9159-4c80-9524-80decbab9c73
/tidb/cdc/default/default/upstream/7362824378797235954
{"id":7362824378797235954,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c0dfea5d-9159-4c80-9524-80decbab9c73
{"id":"c0dfea5d-9159-4c80-9524-80decbab9c73","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b79ab38e
c0dfea5d-9159-4c80-9524-80decbab9c73
/tidb/cdc/default/default/upstream/7362824378797235954
{"id":7362824378797235954,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c0dfea5d-9159-4c80-9524-80decbab9c73
{"id":"c0dfea5d-9159-4c80-9524-80decbab9c73","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23b79ab38e
c0dfea5d-9159-4c80-9524-80decbab9c73
/tidb/cdc/default/default/upstream/7362824378797235954
{"id":7362824378797235954,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 142 0 0 100 142 0 1654 --:--:-- --:--:-- --:--:-- 1670
mysql_test start
go: downloading github.com/maxshuang/mysql-tester v0.0.2
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading github.com/pingcap/errors v0.11.5-0.20211009033009-93128226aaa3
go: downloading github.com/pingcap/tidb/parser v0.0.0-20211122065349-4d16aabf900b
go: downloading go.uber.org/atomic v1.9.0
go: downloading golang.org/x/sys v0.0.0-20211117180635-dee7805ff2e1
go: downloading go.uber.org/zap v1.19.1
go: downloading github.com/pingcap/log v0.0.0-20210906054005-afc726e70354
go: downloading golang.org/x/text v0.3.7
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
go: downloading go.uber.org/multierr v1.7.0
build mysql_test successfully
run all mysql test cases
time="2024-04-28T16:00:18+08:00" level=info msg="running tests: [alter_table bigint composite_index concurrent_ddl date_formats drop mysql_replace]"
time="2024-04-28T16:00:18+08:00" level=warning msg="Create new db&{0 0xc000126000 0 {0 0} [0xc00012c090] map[] 0 1 0xc00012a000 false map[0xc00012c090:map[0xc00012c090:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/alter_table.test: ok! 70 test cases passed, take time 20.959265128 s
time="2024-04-28T16:01:16+08:00" level=info msg="run test [alter_table] ok"
time="2024-04-28T16:01:16+08:00" level=warning msg="Create new db&{0 0xc000126350 0 {0 0} [0xc000384000] map[] 0 1 0xc00012aa20 false map[0xc000384000:map[0xc000384000:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/bigint.test: ok! 138 test cases passed, take time 12.891988256 s
time="2024-04-28T16:01:29+08:00" level=info msg="run test [bigint] ok"
time="2024-04-28T16:01:29+08:00" level=warning msg="Create new db&{0 0xc000014240 0 {0 0} [0xc000384bd0] map[] 0 1 0xc000038a80 false map[0xc000384bd0:map[0xc000384bd0:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/composite_index.test: ok! 12 test cases passed, take time 1.5642882390000001 s
time="2024-04-28T16:01:31+08:00" level=info msg="run test [composite_index] ok"
time="2024-04-28T16:01:31+08:00" level=warning msg="Create new db&{0 0xc0001262e8 0 {0 0} [0xc000384cf0] map[] 0 1 0xc0003d0de0 false map[0xc000384cf0:map[0xc000384cf0:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/concurrent_ddl.test: ok! 8 test cases passed, take time 1.026785552 s
time="2024-04-28T16:01:33+08:00" level=info msg="run test [concurrent_ddl] ok"
time="2024-04-28T16:01:33+08:00" level=warning msg="Create new db&{0 0xc000406038 0 {0 0} [0xc000352000] map[] 0 1 0xc000446120 false map[0xc000352000:map[0xc000352000:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/date_formats.test: ok! 99 test cases passed, take time 8.844123046 s
time="2024-04-28T16:01:42+08:00" level=info msg="run test [date_formats] ok"
time="2024-04-28T16:01:42+08:00" level=warning msg="Create new db&{0 0xc00038c510 0 {0 0} [0xc000211d40] map[] 0 1 0xc00012b860 false map[0xc000211d40:map[0xc000211d40:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/drop.test: ok! 48 test cases passed, take time 10.384107172 s
time="2024-04-28T16:01:54+08:00" level=info msg="run test [drop] ok"
time="2024-04-28T16:01:54+08:00" level=warning msg="Create new db&{0 0xc000014128 0 {0 0} [0xc0002101b0] map[] 0 1 0xc000300840 false map[0xc0002101b0:map[0xc0002101b0:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/mysql_replace.test: ok! 16 test cases passed, take time 1.093828934 s
time="2024-04-28T16:01:56+08:00" level=info msg="run test [mysql_replace] ok"
Great, All tests passed
mysqltest end
table test.finish_mark not exists for 1-th check, retry later
table test.finish_mark not exists for 2-th check, retry later
table test.finish_mark not exists for 3-th check, retry later
table test.finish_mark not exists for 4-th check, retry later
table test.finish_mark not exists for 5-th check, retry later
table test.finish_mark not exists for 6-th check, retry later
table test.finish_mark not exists for 7-th check, retry later
table test.finish_mark not exists for 8-th check, retry later
table test.finish_mark not exists for 9-th check, retry later
table test.finish_mark not exists for 10-th check, retry later
table test.finish_mark not exists for 11-th check, retry later
table test.finish_mark not exists for 12-th check, retry later
table test.finish_mark not exists for 13-th check, retry later
table test.finish_mark not exists for 14-th check, retry later
table test.finish_mark not exists for 15-th check, retry later
table test.finish_mark not exists for 16-th check, retry later
table test.finish_mark not exists for 17-th check, retry later
table test.finish_mark not exists for 18-th check, retry later
table test.finish_mark not exists for 19-th check, retry later
table test.finish_mark not exists for 20-th check, retry later
table test.finish_mark not exists for 21-th check, retry later
table test.finish_mark not exists for 22-th check, retry later
table test.finish_mark not exists for 23-th check, retry later
table test.finish_mark not exists for 24-th check, retry later
table test.finish_mark not exists for 25-th check, retry later
table test.finish_mark not exists for 26-th check, retry later
table test.finish_mark not exists for 27-th check, retry later
table test.finish_mark not exists for 28-th check, retry later
table test.finish_mark not exists for 29-th check, retry later
table test.finish_mark not exists for 30-th check, retry later
table test.finish_mark not exists for 31-th check, retry later
table test.finish_mark not exists for 32-th check, retry later
table test.finish_mark not exists for 33-th check, retry later
table test.finish_mark not exists for 34-th check, retry later
table test.finish_mark not exists for 35-th check, retry later
table test.finish_mark not exists for 36-th check, retry later
table test.finish_mark not exists for 37-th check, retry later
table test.finish_mark exists
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 16:03:12 CST 2024] <<<<<< run test case tidb_mysql_test success! >>>>>>
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/tiflash/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec52e40015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:82006, start at 2024-04-28 16:03:34.223554211 +0800 CST m=+4.400436506 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:34.229 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:34.201 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:34.201 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/tiflash/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/tiflash/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec52e40015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:82006, start at 2024-04-28 16:03:34.223554211 +0800 CST m=+4.400436506 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:34.229 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:34.201 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:34.201 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-4-gddc65433fc
Edition: Community
Git Commit Hash: ddc65433fcf393abb894262a21de6ff45994ea4e
Git Branch: release-6.5
UTC Build Time: 2024-04-28 05:18:15
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec4f140015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:81919, start at 2024-04-28 16:03:33.986874465 +0800 CST m=+4.247177306 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:33.994 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:33.957 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:33.957 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c8eec52e40015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-167--l76b5, pid:82006, start at 2024-04-28 16:03:34.223554211 +0800 CST m=+4.400436506 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240428-16:05:34.229 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240428-16:03:34.201 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240428-15:53:34.201 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.cli.86115.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449391220242251777
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449391220242251777 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[Sun Apr 28 16:04:19 CST 2024] <<<<<< START cdc server in tiflash case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ GO_FAILPOINTS=
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.8615386155.out server --log-file /tmp/tidb_cdc_test/tiflash/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/tiflash/cdc_data --cluster-id default
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Sun, 28 Apr 2024 08:04:22 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
{"id":"7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23bb985a8e
7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
/tidb/cdc/default/default/upstream/7362825526335224241
{"id":7362825526335224241,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
{"id":"7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23bb985a8e
7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
/tidb/cdc/default/default/upstream/7362825526335224241
{"id":7362825526335224241,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
{"id":"7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc","address":"127.0.0.1:8300","version":"v6.5.9-2-ge6698aa29"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f23bb985a8e
7dd82bcd-8b50-4c1d-8fc6-4caa24ca49fc
/tidb/cdc/default/default/upstream/7362825526335224241
{"id":7362825526335224241,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.cli.86220.out cli changefeed create --start-ts=449391220242251777 --sink-uri=mysql://normal:123456@127.0.0.1:3306/
Create changefeed successfully!
ID: 026c8385-ccd9-45e7-a6e8-bec582ce2a9a
Info: {"upstream_id":7362825526335224241,"namespace":"default","id":"026c8385-ccd9-45e7-a6e8-bec582ce2a9a","sink_uri":"mysql://normal:xxxxx@127.0.0.1:3306/","create_time":"2024-04-28T16:04:22.706202902+08:00","start_ts":449391220242251777,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":false,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-2-ge6698aa29","resolved_ts":449391220242251777,"checkpoint_ts":449391220242251777,"checkpoint_time":"2024-04-28 16:04:19.054"}
PASS
coverage: 2.4% of statements in github.com/pingcap/tiflow/...
+ set +x
table cdc_tiflash_test.multi_data_type exists
check diff failed 1-th time, retry later
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Sun Apr 28 16:05:20 CST 2024] <<<<<< run test case tiflash success! >>>>>>
\033[0;36m<<< Run all test success >>>\033[0m
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-6.5-pull_cdc_integration_test-167/tiflow-cdc already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
![[Jenkins]](/jenkins/static/62b7ca6c/images/svgs/logo.svg){kind=logo}
