tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:24:20.500 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:14:20.500 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
start tidb cluster in /tmp/tidb_cdc_test/synced_status
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c6898ae840024 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:53246, start at 2024-04-26 19:24:20.302862069 +0800 CST m=+4.186997936 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:26:20.310 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:24:20.307 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:14:20.307 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c6898ae840024 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:53246, start at 2024-04-26 19:24:20.302862069 +0800 CST m=+4.186997936 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:26:20.310 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:24:20.307 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:14:20.307 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c6898b1880019 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:53340, start at 2024-04-26 19:24:20.49826846 +0800 CST m=+4.328216044 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:26:20.504 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:24:20.500 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:14:20.500 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ cd /tmp/tidb_cdc_test/synced_status
++ run_cdc_cli_tso_query 127.0.0.1 2379
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.cli.57274.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449349078555820033
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449349078555820033 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
+ start_ts=449349078555820033
+ run_cdc_server --workdir /tmp/tidb_cdc_test/synced_status --binary cdc.test
[Fri Apr 26 19:25:01 CST 2024] <<<<<< START cdc server in synced_status case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ GO_FAILPOINTS=
+ (( i <= 50 ))
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.5731557317.out server --log-file /tmp/tidb_cdc_test/synced_status/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/synced_status/cdc_data --cluster-id default
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 26 Apr 2024 11:25:04 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/2d75041a-1ebc-4883-a454-2055862ead26
{"id":"2d75041a-1ebc-4883-a454-2055862ead26","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a26a6ea90
2d75041a-1ebc-4883-a454-2055862ead26
/tidb/cdc/default/default/upstream/7362135087186245698
{"id":7362135087186245698,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/2d75041a-1ebc-4883-a454-2055862ead26
{"id":"2d75041a-1ebc-4883-a454-2055862ead26","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a26a6ea90
2d75041a-1ebc-4883-a454-2055862ead26
/tidb/cdc/default/default/upstream/7362135087186245698
{"id":7362135087186245698,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/2d75041a-1ebc-4883-a454-2055862ead26
{"id":"2d75041a-1ebc-4883-a454-2055862ead26","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a26a6ea90
2d75041a-1ebc-4883-a454-2055862ead26
/tidb/cdc/default/default/upstream/7362135087186245698
{"id":7362135087186245698,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ config_path=conf/changefeed-redo.toml
+ SINK_URI='mysql://root@127.0.0.1:3306/?max-txn-row=1'
+ run_cdc_cli changefeed create --start-ts=449349078555820033 '--sink-uri=mysql://root@127.0.0.1:3306/?max-txn-row=1' --changefeed-id=test-1 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/synced_status/conf/changefeed-redo.toml
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.cli.57387.out cli changefeed create --start-ts=449349078555820033 '--sink-uri=mysql://root@127.0.0.1:3306/?max-txn-row=1' --changefeed-id=test-1 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/synced_status/conf/changefeed-redo.toml
Create changefeed successfully!
ID: test-1
Info: {"upstream_id":7362135087186245698,"namespace":"default","id":"test-1","sink_uri":"mysql://root@127.0.0.1:3306/?max-txn-row=1","create_time":"2024-04-26T19:25:04.998772928+08:00","start_ts":449349078555820033,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":false,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"eventual","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"storage":"file:///tmp/tidb_cdc_test/synced_status/redo","use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":120,"checkpoint_interval":20}},"state":"normal","creator_version":"v6.5.9-1-gc3f88235b","resolved_ts":449349078555820033,"checkpoint_ts":449349078555820033,"checkpoint_time":"2024-04-26 19:25:01.278"}
PASS
coverage: 2.5% of statements in github.com/pingcap/tiflow/...
+ set +x
+ run_sql 'USE TEST;Create table t1(a int primary key, b int);insert into t1 values(1,2);insert into t1 values(2,3);'
+ check_table_exists test.t1 127.0.0.1 3306
table test.t1 not exists for 1-th check, retry later
table test.t1 exists
+ sleep 5
+ kill_tidb
++ ps aux
++ grep tidb-server
++ grep /tmp/tidb_cdc_test/synced_status
+ info='jenkins 57128 1.0 0.0 1967100 137296 ? Sl 19:24 0:00 tidb-server -P 4000 -config /tmp/tidb_cdc_test/synced_status/tidb-config-1714130698939851429.toml --store tikv --path 127.0.0.1:2379 --status=10080 --log-file /tmp/tidb_cdc_test/synced_status/tidb.log
jenkins 57132 0.7 0.0 2140176 120720 ? Sl 19:24 0:00 tidb-server -P 4001 -config /tmp/tidb_cdc_test/synced_status/tidb-config-1714130698942789381.toml --store tikv --path 127.0.0.1:2379 --status=10081 --log-file /tmp/tidb_cdc_test/synced_status/tidb_other.log
jenkins 57215 0.9 0.0 2148372 141028 ? Sl 19:24 0:00 tidb-server -P 3306 -config /tmp/tidb_cdc_test/synced_status/tidb-config-1714130698992532265.toml --store tikv --path 127.0.0.1:2479 --status=20080 --log-file /tmp/tidb_cdc_test/synced_status/tidb_down.log'
++ ps aux
++ grep tidb-server
++ grep /tmp/tidb_cdc_test/synced_status
++ awk '{print $2}'
++ xargs kill -9
++ curl -X GET http://127.0.0.1:8300/api/v2/changefeeds/test-1/synced
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 243 100 243 0 0 31990 0 --:--:-- --:--:-- --:--:-- 34714
+ synced_status='{"synced":false,"sink_checkpoint_ts":"2024-04-26 19:26:02.178","puller_resolved_ts":"1970-01-01 08:00:00.000","last_synced_ts":"2024-04-26 19:25:58.178","now_ts":"2024-04-26 19:26:05.000","info":"The data syncing is not finished, please wait"}'
++ echo '{"synced":false,"sink_checkpoint_ts":"2024-04-26' '19:26:02.178","puller_resolved_ts":"1970-01-01' '08:00:00.000","last_synced_ts":"2024-04-26' '19:25:58.178","now_ts":"2024-04-26' '19:26:05.000","info":"The' data syncing is not finished, please 'wait"}'
++ jq .synced
+ status=false
+ '[' false '!=' false ']'
++ echo '{"synced":false,"sink_checkpoint_ts":"2024-04-26' '19:26:02.178","puller_resolved_ts":"1970-01-01' '08:00:00.000","last_synced_ts":"2024-04-26' '19:25:58.178","now_ts":"2024-04-26' '19:26:05.000","info":"The' data syncing is not finished, please 'wait"}'
++ jq -r .info
+ info='The data syncing is not finished, please wait'
+ target_message='The data syncing is not finished, please wait'
+ '[' 'The data syncing is not finished, please wait' '!=' 'The data syncing is not finished, please wait' ']'
+ sleep 130
++ curl -X GET http://127.0.0.1:8300/api/v2/changefeeds/test-1/synced
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 221 100 221 0 0 5526 0 --:--:-- --:--:-- --:--:-- 5666
+ synced_status='{"synced":true,"sink_checkpoint_ts":"2024-04-26 19:28:12.328","puller_resolved_ts":"2024-04-26 19:28:05.329","last_synced_ts":"2024-04-26 19:25:58.178","now_ts":"2024-04-26 19:28:15.000","info":"Data syncing is finished"}'
++ echo '{"synced":true,"sink_checkpoint_ts":"2024-04-26' '19:28:12.328","puller_resolved_ts":"2024-04-26' '19:28:05.329","last_synced_ts":"2024-04-26' '19:25:58.178","now_ts":"2024-04-26' '19:28:15.000","info":"Data' syncing is 'finished"}'
++ jq .synced
+ status=true
+ '[' true '!=' true ']'
++ echo '{"synced":true,"sink_checkpoint_ts":"2024-04-26' '19:28:12.328","puller_resolved_ts":"2024-04-26' '19:28:05.329","last_synced_ts":"2024-04-26' '19:25:58.178","now_ts":"2024-04-26' '19:28:15.000","info":"Data' syncing is 'finished"}'
++ jq -r .info
+ info='Data syncing is finished'
+ target_message='Data syncing is finished'
+ '[' 'Data syncing is finished' '!=' 'Data syncing is finished' ']'
+ cleanup_process cdc.test
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
+ stop_tidb_cluster
+ run_case_with_failpoint conf/changefeed-redo.toml
+ rm -rf /tmp/tidb_cdc_test/synced_status
+ mkdir -p /tmp/tidb_cdc_test/synced_status
+ start_tidb_cluster --workdir /tmp/tidb_cdc_test/synced_status
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
The 1 times to try to start tidb cluster...
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
start tidb cluster in /tmp/tidb_cdc_test/synced_status
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a835980017 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58476, start at 2024-04-26 19:28:34.703337192 +0800 CST m=+4.312678362 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.712 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.712 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/synced_status/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/synced_status/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
start tidb cluster in /tmp/tidb_cdc_test/synced_status
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a835980017 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58476, start at 2024-04-26 19:28:34.703337192 +0800 CST m=+4.312678362 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.712 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.712 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
start tidb cluster in /tmp/tidb_cdc_test/synced_status
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a8341c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58377, start at 2024-04-26 19:28:34.59233094 +0800 CST m=+4.247080436 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.599 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.567 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.567 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68a835980017 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:58476, start at 2024-04-26 19:28:34.703337192 +0800 CST m=+4.312678362 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:30:34.711 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:28:34.712 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:18:34.712 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ cd /tmp/tidb_cdc_test/synced_status
+ export 'GO_FAILPOINTS=github.com/pingcap/tiflow/cdc/owner/ChangefeedOwnerNotUpdateCheckpoint=return(true)'
+ GO_FAILPOINTS='github.com/pingcap/tiflow/cdc/owner/ChangefeedOwnerNotUpdateCheckpoint=return(true)'
++ run_cdc_cli_tso_query 127.0.0.1 2379
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.cli.62511.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449349145983713281
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449349145983713281 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
+ start_ts=449349145983713281
+ run_cdc_server --workdir /tmp/tidb_cdc_test/synced_status --binary cdc.test
[Fri Apr 26 19:29:18 CST 2024] <<<<<< START cdc server in synced_status case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ GO_FAILPOINTS='github.com/pingcap/tiflow/cdc/owner/ChangefeedOwnerNotUpdateCheckpoint=return(true)'
+ [[ no != \n\o ]]
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.6254962551.out server --log-file /tmp/tidb_cdc_test/synced_status/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/synced_status/cdc_data --cluster-id default
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 26 Apr 2024 11:29:21 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/9af90d4a-d376-4828-8ac3-1b612ecf584d
{"id":"9af90d4a-d376-4828-8ac3-1b612ecf584d","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2a8f8e90
9af90d4a-d376-4828-8ac3-1b612ecf584d
/tidb/cdc/default/default/upstream/7362136190620545819
{"id":7362136190620545819,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/9af90d4a-d376-4828-8ac3-1b612ecf584d
{"id":"9af90d4a-d376-4828-8ac3-1b612ecf584d","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2a8f8e90
9af90d4a-d376-4828-8ac3-1b612ecf584d
/tidb/cdc/default/default/upstream/7362136190620545819
{"id":7362136190620545819,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/9af90d4a-d376-4828-8ac3-1b612ecf584d
{"id":"9af90d4a-d376-4828-8ac3-1b612ecf584d","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2a8f8e90
9af90d4a-d376-4828-8ac3-1b612ecf584d
/tidb/cdc/default/default/upstream/7362136190620545819
{"id":7362136190620545819,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ config_path=conf/changefeed-redo.toml
+ SINK_URI='mysql://root@127.0.0.1:3306/?max-txn-row=1'
+ run_cdc_cli changefeed create --start-ts=449349145983713281 '--sink-uri=mysql://root@127.0.0.1:3306/?max-txn-row=1' --changefeed-id=test-1 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/synced_status/conf/changefeed-redo.toml
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.synced_status.cli.62611.out cli changefeed create --start-ts=449349145983713281 '--sink-uri=mysql://root@127.0.0.1:3306/?max-txn-row=1' --changefeed-id=test-1 --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/synced_status/conf/changefeed-redo.toml
Create changefeed successfully!
ID: test-1
Info: {"upstream_id":7362136190620545819,"namespace":"default","id":"test-1","sink_uri":"mysql://root@127.0.0.1:3306/?max-txn-row=1","create_time":"2024-04-26T19:29:22.181901432+08:00","start_ts":449349145983713281,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":false,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"eventual","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"storage":"file:///tmp/tidb_cdc_test/synced_status/redo","use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":120,"checkpoint_interval":20}},"state":"normal","creator_version":"v6.5.9-1-gc3f88235b","resolved_ts":449349145983713281,"checkpoint_ts":449349145983713281,"checkpoint_time":"2024-04-26 19:29:18.495"}
PASS
coverage: 2.5% of statements in github.com/pingcap/tiflow/...
+ set +x
+ sleep 20
++ curl -X GET http://127.0.0.1:8300/api/v2/changefeeds/test-1/synced
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 723 100 723 0 0 19022 0 --:--:-- --:--:-- --:--:-- 19540
+ synced_status='{"synced":false,"sink_checkpoint_ts":"2024-04-26 19:29:18.495","puller_resolved_ts":"1970-01-01 08:00:00.000","last_synced_ts":"1970-01-01 08:00:00.000","now_ts":"2024-04-26 19:29:42.000","info":"Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view '\''TiKV-Details'\'' \u003e '\''Resolved-Ts'\'' \u003e '\''Max Leader Resolved TS gap'\'' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait"}'
++ echo '{"synced":false,"sink_checkpoint_ts":"2024-04-26' '19:29:18.495","puller_resolved_ts":"1970-01-01' '08:00:00.000","last_synced_ts":"1970-01-01' '08:00:00.000","now_ts":"2024-04-26' '19:29:42.000","info":"Please' check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view ''\''TiKV-Details'\''' '\u003e' ''\''Resolved-Ts'\''' '\u003e' ''\''Max' Leader Resolved TS 'gap'\''' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please 'wait"}'
++ jq .synced
+ status=false
+ '[' false '!=' false ']'
++ echo '{"synced":false,"sink_checkpoint_ts":"2024-04-26' '19:29:18.495","puller_resolved_ts":"1970-01-01' '08:00:00.000","last_synced_ts":"1970-01-01' '08:00:00.000","now_ts":"2024-04-26' '19:29:42.000","info":"Please' check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view ''\''TiKV-Details'\''' '\u003e' ''\''Resolved-Ts'\''' '\u003e' ''\''Max' Leader Resolved TS 'gap'\''' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please 'wait"}'
++ jq -r .info
+ info='Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view '\''TiKV-Details'\'' > '\''Resolved-Ts'\'' > '\''Max Leader Resolved TS gap'\'' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait'
+ target_message='Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view '\''TiKV-Details'\'' > '\''Resolved-Ts'\'' > '\''Max Leader Resolved TS gap'\'' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait'
+ '[' 'Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view '\''TiKV-Details'\'' > '\''Resolved-Ts'\'' > '\''Max Leader Resolved TS gap'\'' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait' '!=' 'Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view '\''TiKV-Details'\'' > '\''Resolved-Ts'\'' > '\''Max Leader Resolved TS gap'\'' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait' ']'
+ export GO_FAILPOINTS=
+ GO_FAILPOINTS=
+ cleanup_process cdc.test
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
+ stop_tidb_cluster
+ check_logs /tmp/tidb_cdc_test/synced_status
++ date
+ echo '[Fri Apr 26 19:29:52 CST 2024] <<<<<< run test case synced_status success! >>>>>>'
[Fri Apr 26 19:29:52 CST 2024] <<<<<< run test case synced_status success! >>>>>>
+ stop_tidb_cluster
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/syncpoint/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/syncpoint
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade5f8001a Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63691, start at 2024-04-26 19:30:07.916634062 +0800 CST m=+4.550858120 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.924 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.920 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.920 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/syncpoint/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/syncpoint/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/syncpoint
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade5f8001a Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63691, start at 2024-04-26 19:30:07.916634062 +0800 CST m=+4.550858120 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.924 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.920 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.920 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/syncpoint
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade6e00018 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63598, start at 2024-04-26 19:30:07.96309021 +0800 CST m=+4.664809567 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.969 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.928 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.928 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68ade5f8001a Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:63691, start at 2024-04-26 19:30:07.916634062 +0800 CST m=+4.550858120 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:32:07.924 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:30:07.920 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:20:07.920 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.syncpoint.cli.67744.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449349185826979841
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449349185826979841 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[Fri Apr 26 19:31:50 CST 2024] <<<<<< START cdc server in syncpoint case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ [[ no != \n\o ]]
+ GO_FAILPOINTS='github.com/pingcap/tiflow/cdc/owner/DDLSinkInitializeSlowly=return(true)'
+ (( i = 0 ))
+ (( i <= 50 ))
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.syncpoint.6779067792.out server --log-file /tmp/tidb_cdc_test/syncpoint/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/syncpoint/cdc_data --cluster-id default
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 26 Apr 2024 11:31:53 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c325ddc2-9121-4c1d-9734-bc93fd1c52d1
{"id":"c325ddc2-9121-4c1d-9734-bc93fd1c52d1","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2bfc5e10
c325ddc2-9121-4c1d-9734-bc93fd1c52d1
/tidb/cdc/default/default/upstream/7362136585372395897
{"id":7362136585372395897,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c325ddc2-9121-4c1d-9734-bc93fd1c52d1
{"id":"c325ddc2-9121-4c1d-9734-bc93fd1c52d1","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2bfc5e10
c325ddc2-9121-4c1d-9734-bc93fd1c52d1
/tidb/cdc/default/default/upstream/7362136585372395897
{"id":7362136585372395897,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/c325ddc2-9121-4c1d-9734-bc93fd1c52d1
{"id":"c325ddc2-9121-4c1d-9734-bc93fd1c52d1","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a2bfc5e10
c325ddc2-9121-4c1d-9734-bc93fd1c52d1
/tidb/cdc/default/default/upstream/7362136585372395897
{"id":7362136585372395897,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.syncpoint.cli.67850.out cli changefeed create --start-ts=449349185826979841 '--sink-uri=mysql://root@127.0.0.1:3306/?max-txn-row=1' --config=/home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/syncpoint/conf/changefeed.toml
Create changefeed successfully!
ID: 8eb0b4ad-6dcf-4d40-ba54-13b52a83c0b8
Info: {"upstream_id":7362136585372395897,"namespace":"default","id":"8eb0b4ad-6dcf-4d40-ba54-13b52a83c0b8","sink_uri":"mysql://root@127.0.0.1:3306/?max-txn-row=1","create_time":"2024-04-26T19:31:54.195673152+08:00","start_ts":449349185826979841,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":true,"bdr_mode":false,"sync_point_interval":30000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-1-gc3f88235b","resolved_ts":449349185826979841,"checkpoint_ts":449349185826979841,"checkpoint_time":"2024-04-26 19:31:50.485"}
PASS
coverage: 2.5% of statements in github.com/pingcap/tiflow/...
+ set +x
***************** properties *****************
"mysql.host"="127.0.0.1"
"threadcount"="2"
"updateproportion"="0"
"dotransactions"="false"
"operationcount"="0"
"mysql.user"="root"
"readallfields"="true"
"readproportion"="0"
"mysql.db"="testSync"
"requestdistribution"="uniform"
"mysql.port"="4000"
"scanproportion"="0"
"recordcount"="10"
"workload"="core"
"insertproportion"="0"
**********************************************
Run finished, takes 9.206659ms
INSERT - Takes(s): 0.0, Count: 10, OPS: 2188.1, Avg(us): 1760, Min(us): 874, Max(us): 4474, 95th(us): 5000, 99th(us): 5000
***************** properties *****************
"recordcount"="10"
"dotransactions"="false"
"mysql.port"="4000"
"threadcount"="2"
"workload"="core"
"mysql.user"="root"
"readallfields"="true"
"mysql.host"="127.0.0.1"
"scanproportion"="0"
"operationcount"="0"
"mysql.db"="testSync"
"insertproportion"="0"
"requestdistribution"="uniform"
"readproportion"="0"
"updateproportion"="0"
**********************************************
Run finished, takes 4.078798ms
INSERT - Takes(s): 0.0, Count: 10, OPS: 3516.8, Avg(us): 725, Min(us): 483, Max(us): 1436, 95th(us): 2000, 99th(us): 2000
***************** properties *****************
"scanproportion"="0"
"updateproportion"="0"
"workload"="core"
"mysql.host"="127.0.0.1"
"operationcount"="0"
"mysql.user"="root"
"readproportion"="0"
"recordcount"="10"
"requestdistribution"="uniform"
"mysql.db"="testSync"
"threadcount"="2"
"dotransactions"="false"
"readallfields"="true"
"insertproportion"="0"
"mysql.port"="4000"
**********************************************
Run finished, takes 3.848561ms
INSERT - Takes(s): 0.0, Count: 10, OPS: 3704.9, Avg(us): 657, Min(us): 430, Max(us): 1445, 95th(us): 2000, 99th(us): 2000
table testSync.usertable exists
table testSync.simple1 exists
table testSync.simple2 exists
____________________________________
*************************** 1. row ***************************
primary_ts: 449349185826979841
secondary_ts: 449349188171857933
*************************** 2. row ***************************
primary_ts: 449349193691299840
secondary_ts: 449349193807953921
*************************** 3. row ***************************
primary_ts: 449349201555619840
secondary_ts: 449349201698488321
*************************** 4. row ***************************
primary_ts: 449349209419939840
secondary_ts: 449349209562808321
*************************** 5. row ***************************
primary_ts: 449349217284259840
secondary_ts: 449349217427128321
*************************** 6. row ***************************
primary_ts: 449349225148579840
secondary_ts: 449349225317400577
*************************** 7. row ***************************
primary_ts: 449349233012899840
secondary_ts: 449349233181720577
*************************** 8. row ***************************
primary_ts: 449349240877219840
secondary_ts: 449349241046302721
*************************** 9. row ***************************
primary_ts: 449349248741539840
secondary_ts: 449349248936574977
skip invalid syncpoint primary_ts: 449349185826979841, first_ddl_ts: 449349188605706253
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Fri Apr 26 19:35:54 CST 2024] <<<<<< run test case syncpoint success! >>>>>>
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/tidb_mysql_test/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c475f8000f Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70155, start at 2024-04-26 19:36:17.547930987 +0800 CST m=+4.079904339 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.553 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.534 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.534 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/tidb_mysql_test/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/tidb_mysql_test/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c475f8000f Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70155, start at 2024-04-26 19:36:17.547930987 +0800 CST m=+4.079904339 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.553 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.534 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.534 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tidb_mysql_test
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c4754c0014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70067, start at 2024-04-26 19:36:17.518310574 +0800 CST m=+4.110198395 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.524 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.491 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.491 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68c475f8000f Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:70155, start at 2024-04-26 19:36:17.547930987 +0800 CST m=+4.079904339 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:38:17.553 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:36:17.534 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:26:17.534 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tidb_mysql_test.cli.74181.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449349267026870273
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449349267026870273 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[Fri Apr 26 19:37:00 CST 2024] <<<<<< START cdc server in tidb_mysql_test case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ [[ no != \n\o ]]
+ (( i = 0 ))
+ (( i <= 50 ))
+ GO_FAILPOINTS=
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tidb_mysql_test.7422574227.out server --log-file /tmp/tidb_cdc_test/tidb_mysql_test/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/tidb_mysql_test/cdc_data --cluster-id default
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 26 Apr 2024 11:37:03 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7f246157-d545-456d-a475-adc72c630710
{"id":"7f246157-d545-456d-a475-adc72c630710","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a319f598e
7f246157-d545-456d-a475-adc72c630710
/tidb/cdc/default/default/upstream/7362138174627663522
{"id":7362138174627663522,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7f246157-d545-456d-a475-adc72c630710
{"id":"7f246157-d545-456d-a475-adc72c630710","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a319f598e
7f246157-d545-456d-a475-adc72c630710
/tidb/cdc/default/default/upstream/7362138174627663522
{"id":7362138174627663522,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/7f246157-d545-456d-a475-adc72c630710
{"id":"7f246157-d545-456d-a475-adc72c630710","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a319f598e
7f246157-d545-456d-a475-adc72c630710
/tidb/cdc/default/default/upstream/7362138174627663522
{"id":7362138174627663522,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 142 0 0 100 142 0 1651 --:--:-- --:--:-- --:--:-- 1670
mysql_test start
go: downloading github.com/maxshuang/mysql-tester v0.0.2
go: downloading github.com/pingcap/tidb/parser v0.0.0-20211122065349-4d16aabf900b
go: downloading github.com/pingcap/errors v0.11.5-0.20211009033009-93128226aaa3
go: downloading github.com/sirupsen/logrus v1.8.1
go: downloading go.uber.org/atomic v1.9.0
go: downloading golang.org/x/sys v0.0.0-20211117180635-dee7805ff2e1
go: downloading go.uber.org/zap v1.19.1
go: downloading github.com/pingcap/log v0.0.0-20210906054005-afc726e70354
go: downloading golang.org/x/text v0.3.7
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0
go: downloading go.uber.org/multierr v1.7.0
build mysql_test successfully
run all mysql test cases
time="2024-04-26T19:37:23+08:00" level=info msg="running tests: [alter_table bigint composite_index concurrent_ddl date_formats drop mysql_replace]"
time="2024-04-26T19:37:23+08:00" level=warning msg="Create new db&{0 0xc000014098 0 {0 0} [0xc000116000] map[] 0 1 0xc000038240 false map[0xc000116000:map[0xc000116000:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/alter_table.test: ok! 70 test cases passed, take time 6.364359044 s
time="2024-04-26T19:38:04+08:00" level=info msg="run test [alter_table] ok"
time="2024-04-26T19:38:04+08:00" level=warning msg="Create new db&{0 0xc00030e160 0 {0 0} [0xc000210990] map[] 0 1 0xc0003187e0 false map[0xc000210990:map[0xc000210990:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/bigint.test: ok! 138 test cases passed, take time 4.9159583829999995 s
time="2024-04-26T19:38:09+08:00" level=info msg="run test [bigint] ok"
time="2024-04-26T19:38:09+08:00" level=warning msg="Create new db&{0 0xc00030e720 0 {0 0} [0xc00044e5a0] map[] 0 1 0xc0003199e0 false map[0xc00044e5a0:map[0xc00044e5a0:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/composite_index.test: ok! 12 test cases passed, take time 0.246441871 s
time="2024-04-26T19:38:09+08:00" level=info msg="run test [composite_index] ok"
time="2024-04-26T19:38:09+08:00" level=warning msg="Create new db&{0 0xc00040a020 0 {0 0} [0xc000210240] map[] 0 1 0xc000038420 false map[0xc000210240:map[0xc000210240:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/concurrent_ddl.test: ok! 8 test cases passed, take time 0.444434972 s
time="2024-04-26T19:38:10+08:00" level=info msg="run test [concurrent_ddl] ok"
time="2024-04-26T19:38:10+08:00" level=warning msg="Create new db&{0 0xc00040a0c0 0 {0 0} [0xc0002103f0] map[] 0 1 0xc0000386c0 false map[0xc0002103f0:map[0xc0002103f0:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/date_formats.test: ok! 99 test cases passed, take time 0.972110773 s
time="2024-04-26T19:38:11+08:00" level=info msg="run test [date_formats] ok"
time="2024-04-26T19:38:11+08:00" level=warning msg="Create new db&{0 0xc00040a400 0 {0 0} [0xc000211a70] map[] 0 1 0xc0000395c0 false map[0xc000211a70:map[0xc000211a70:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/drop.test: ok! 48 test cases passed, take time 1.919063488 s
time="2024-04-26T19:38:13+08:00" level=info msg="run test [drop] ok"
time="2024-04-26T19:38:13+08:00" level=warning msg="Create new db&{0 0xc00030e828 0 {0 0} [0xc00044f050] map[] 0 1 0xc0002ee480 false map[0xc00044f050:map[0xc00044f050:true]] map[] 0 0 0 0 <nil> 0 0 0 0 0x48dd80}"
./t/mysql_replace.test: ok! 16 test cases passed, take time 0.19093113 s
time="2024-04-26T19:38:13+08:00" level=info msg="run test [mysql_replace] ok"
Great, All tests passed
mysqltest end
table test.finish_mark not exists for 1-th check, retry later
table test.finish_mark not exists for 2-th check, retry later
table test.finish_mark not exists for 3-th check, retry later
table test.finish_mark not exists for 4-th check, retry later
table test.finish_mark not exists for 5-th check, retry later
table test.finish_mark not exists for 6-th check, retry later
table test.finish_mark not exists for 7-th check, retry later
table test.finish_mark not exists for 8-th check, retry later
table test.finish_mark not exists for 9-th check, retry later
table test.finish_mark not exists for 10-th check, retry later
table test.finish_mark not exists for 11-th check, retry later
table test.finish_mark not exists for 12-th check, retry later
table test.finish_mark not exists for 13-th check, retry later
table test.finish_mark not exists for 14-th check, retry later
table test.finish_mark not exists for 15-th check, retry later
table test.finish_mark not exists for 16-th check, retry later
table test.finish_mark not exists for 17-th check, retry later
table test.finish_mark not exists for 18-th check, retry later
table test.finish_mark not exists for 19-th check, retry later
table test.finish_mark not exists for 20-th check, retry later
table test.finish_mark not exists for 21-th check, retry later
table test.finish_mark not exists for 22-th check, retry later
table test.finish_mark not exists for 23-th check, retry later
table test.finish_mark not exists for 24-th check, retry later
table test.finish_mark not exists for 25-th check, retry later
table test.finish_mark not exists for 26-th check, retry later
table test.finish_mark not exists for 27-th check, retry later
table test.finish_mark not exists for 28-th check, retry later
table test.finish_mark not exists for 29-th check, retry later
table test.finish_mark not exists for 30-th check, retry later
table test.finish_mark not exists for 31-th check, retry later
table test.finish_mark not exists for 32-th check, retry later
table test.finish_mark not exists for 33-th check, retry later
table test.finish_mark not exists for 34-th check, retry later
table test.finish_mark not exists for 35-th check, retry later
table test.finish_mark not exists for 36-th check, retry later
table test.finish_mark not exists for 37-th check, retry later
table test.finish_mark not exists for 38-th check, retry later
table test.finish_mark not exists for 39-th check, retry later
table test.finish_mark not exists for 40-th check, retry later
table test.finish_mark not exists for 41-th check, retry later
table test.finish_mark not exists for 42-th check, retry later
table test.finish_mark not exists for 43-th check, retry later
table test.finish_mark not exists for 44-th check, retry later
table test.finish_mark not exists for 45-th check, retry later
table test.finish_mark not exists for 46-th check, retry later
table test.finish_mark not exists for 47-th check, retry later
table test.finish_mark not exists for 48-th check, retry later
table test.finish_mark not exists for 49-th check, retry later
table test.finish_mark not exists for 50-th check, retry later
table test.finish_mark not exists for 51-th check, retry later
table test.finish_mark not exists for 52-th check, retry later
table test.finish_mark not exists for 53-th check, retry later
table test.finish_mark not exists for 54-th check, retry later
table test.finish_mark not exists for 55-th check, retry later
table test.finish_mark not exists for 56-th check, retry later
table test.finish_mark exists
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Fri Apr 26 19:40:07 CST 2024] <<<<<< run test case tidb_mysql_test success! >>>>>>
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-6.5/pull_cdc_integration_test/tiflow/tests/integration_tests/tiflash/run.sh using Sink-Type: mysql... <<=================
The 1 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3ce8c0002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77110, start at 2024-04-26 19:40:28.963703558 +0800 CST m=+4.612189966 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.970 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.963 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.963 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v6.5.8-7-g0be447c11
Edition: Community
Git Commit Hash: 0be447c1167454e092fadf228aa2ab11cef3f6aa
Git Branch: HEAD
UTC Build Time: 2024-03-30 14:02:19
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Raft Proxy
Git Commit Hash: 37eb1f1a09576a984f07a04fe7d9f2373935657a
Git Commit Branch: HEAD
UTC Build Time: 2024-03-30 14:05:17
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/tiflash/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/tiflash/tiflash/log/error.log
Starting CDC state checker...
# github.com/pingcap/tiflow/tests/utils/cdc_state_checker
./cdc_monitor.go:82:52: cannot use initState (variable of type *cdcReactorState) as type orchestrator.ReactorState in argument to orchestrator.NewEtcdWorker:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:32:12: impossible type assertion: state.(*cdcReactorState)
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:34:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:40:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:46:10: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
./reactor.go:49:9: cannot use r.state (variable of type *cdcReactorState) as type orchestrator.ReactorState in return statement:
*cdcReactorState does not implement orchestrator.ReactorState (missing UpdatePendingChange method)
start tidb cluster failed
The 2 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3ce8c0002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77110, start at 2024-04-26 19:40:28.963703558 +0800 CST m=+4.612189966 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.970 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.963 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.963 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
The 3 times to try to start tidb cluster...
start tidb cluster in /tmp/tidb_cdc_test/tiflash
Starting Upstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Starting Downstream PD...
Release Version: v6.5.9-1-g0ecafcb36
Edition: Community
Git Commit Hash: 0ecafcb3657b51a1ae14f7ca79d5aa6175d70371
Git Branch: release-6.5
UTC Build Time: 2024-04-26 06:25:13
Verifying upstream PD is started...
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 6.5.9
Edition: Community
Git Commit Hash: 172a25c1447d3cca370becc0cd7ceda40cda65d5
Git Commit Branch: release-6.5
UTC Build Time: 2024-04-25 09:35:48
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features: pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile: dist_release
Starting Upstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v6.5.9-3-g7357987916
Edition: Community
Git Commit Hash: 7357987916b70e4721952d8998428db7bfb78adc
Git Branch: release-6.5
UTC Build Time: 2024-04-17 15:20:26
GoVersion: go1.19.13
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3cc500004 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77023, start at 2024-04-26 19:40:28.823168889 +0800 CST m=+4.571158579 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.828 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.820 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.820 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 110 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
tikv_gc_leader_uuid 63c68d3ce8c0002 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-release-6-5-pull-cdc-integration-test-164--zc3dq, pid:77110, start at 2024-04-26 19:40:28.963703558 +0800 CST m=+4.612189966 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240426-19:42:28.970 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 60m All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240426-19:40:28.963 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240426-19:30:28.963 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'normal'@'%'
start tidb cluster failed
+ pd_host=127.0.0.1
+ pd_port=2379
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.cli.81039.out cli tso query --pd=http://127.0.0.1:2379
+ set +x
+ tso='449349333210365953
PASS
coverage: 1.9% of statements in github.com/pingcap/tiflow/...'
+ echo 449349333210365953 PASS coverage: 1.9% of statements in github.com/pingcap/tiflow/...
+ awk -F ' ' '{print $1}'
+ set +x
[Fri Apr 26 19:41:12 CST 2024] <<<<<< START cdc server in tiflash case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ GO_FAILPOINTS=
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info'
+ [[ no != \n\o ]]
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.8108081082.out server --log-file /tmp/tidb_cdc_test/tiflash/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/tiflash/cdc_data --cluster-id default
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
> GET /debug/info HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 26 Apr 2024 11:41:16 GMT
< Content-Length: 612
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/afce39f7-6b22-4043-aefa-299a4c7d241e
{"id":"afce39f7-6b22-4043-aefa-299a4c7d241e","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a3575a790
afce39f7-6b22-4043-aefa-299a4c7d241e
/tidb/cdc/default/default/upstream/7362139255416327829
{"id":7362139255416327829,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/afce39f7-6b22-4043-aefa-299a4c7d241e
{"id":"afce39f7-6b22-4043-aefa-299a4c7d241e","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a3575a790
afce39f7-6b22-4043-aefa-299a4c7d241e
/tidb/cdc/default/default/upstream/7362139255416327829
{"id":7362139255416327829,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/afce39f7-6b22-4043-aefa-299a4c7d241e
{"id":"afce39f7-6b22-4043-aefa-299a4c7d241e","address":"127.0.0.1:8300","version":"v6.5.9-1-gc3f88235b"}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f1a3575a790
afce39f7-6b22-4043-aefa-299a4c7d241e
/tidb/cdc/default/default/upstream/7362139255416327829
{"id":7362139255416327829,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.tiflash.cli.81147.out cli changefeed create --start-ts=449349333210365953 --sink-uri=mysql://normal:123456@127.0.0.1:3306/
Create changefeed successfully!
ID: c2936e6b-ade8-4d50-bd81-349a0d1365d6
Info: {"upstream_id":7362139255416327829,"namespace":"default","id":"c2936e6b-ade8-4d50-bd81-349a0d1365d6","sink_uri":"mysql://normal:xxxxx@127.0.0.1:3306/","create_time":"2024-04-26T19:41:16.396191879+08:00","start_ts":449349333210365953,"config":{"memory_quota":1073741824,"case_sensitive":false,"enable_old_value":true,"force_replicate":false,"ignore_ineligible_table":false,"check_gc_safe_point":true,"enable_sync_point":false,"bdr_mode":false,"sync_point_interval":600000000000,"sync_point_retention":86400000000000,"filter":{"rules":["*.*"],"event_filters":null},"mounter":{"worker_num":16},"sink":{"protocol":"","schema_registry":"","csv":{"delimiter":",","quote":"\"","null":"\\N","include_commit_ts":false,"binary_encoding_method":"base64"},"column_selectors":null,"transaction_atomicity":"","encoder_concurrency":16,"terminator":"\r\n","date_separator":"day","enable_partition_separator":true,"file_index_width":0,"kafka_config":null,"advance_timeout":150},"consistent":{"level":"none","max_log_size":64,"flush_interval":2000,"meta_flush_interval":200,"encoding_worker_num":16,"flush_worker_num":8,"use_file_backend":false,"memory_usage":{"memory_quota_percentage":50,"event_cache_percentage":0}},"changefeed_error_stuck_duration":1800000000000,"sql_mode":"ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION","synced_status":{"synced_check_interval":300,"checkpoint_interval":15}},"state":"normal","creator_version":"v6.5.9-1-gc3f88235b","resolved_ts":449349333210365953,"checkpoint_ts":449349333210365953,"checkpoint_time":"2024-04-26 19:41:12.708"}
PASS
coverage: 2.4% of statements in github.com/pingcap/tiflow/...
+ set +x
table cdc_tiflash_test.multi_data_type not exists for 1-th check, retry later
table cdc_tiflash_test.multi_data_type exists
check diff failed 1-th time, retry later
check diff successfully
cdc.test: no process found
wait process cdc.test exit for 1-th time...
process cdc.test already exit
[Fri Apr 26 19:42:13 CST 2024] <<<<<< run test case tiflash success! >>>>>>
\033[0;36m<<< Run all test success >>>\033[0m
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-6.5-pull_cdc_integration_test-164/tiflow-cdc already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
![[Jenkins]](/jenkins/static/62b7ca6c/images/svgs/logo.svg){kind=logo}
