Skipping 486 KB..
Full LogTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTest
[Pipeline] {
[Pipeline] sh
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
[Pipeline] container
[Pipeline] {
[Pipeline] container
[Pipeline] {
[Pipeline] checkout
+ set +x
[Fri May 17 12:54:31 CST 2024] <<<<<< START kafka consumer in sink_retry case >>>>>>
The recommended git tool is: git
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] stage
[Pipeline] { (Test)
+ echo Waiting for zookeeper to be ready...
Waiting for zookeeper to be ready...
+ nc -z localhost 2181
+ echo Waiting for kafka to be ready...
Waiting for kafka to be ready...
+ nc -z localhost 9092
+ echo Waiting for kafka-broker to be ready...
Waiting for kafka-broker to be ready...
+ echo dump
+ nc localhost 2181
+ grep brokers
+ awk {$1=$1;print}
+ grep -F -w /brokers/ids/1
/brokers/ids/1
[Pipeline] }
[Pipeline] timeout
Timeout set to expire in 45 min
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 45 min
[Pipeline] {
[Pipeline] // timeout
[Pipeline] }
[Pipeline] withCredentials
Masking supported pattern matches of $TICDC_COVERALLS_TOKEN or $TICDC_CODECOV_TOKEN
[Pipeline] withCredentials
Masking supported pattern matches of $TICDC_COVERALLS_TOKEN or $TICDC_CODECOV_TOKEN
[Pipeline] // container
[Pipeline] sh
check diff failed 1-th time, retry later
[Pipeline] {
No credentials specified
Warning: JENKINS-30600: special launcher org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator$1@3c885bb; decorates RemoteLauncher[hudson.remoting.Channel@552bf865:JNLP4-connect connection from 10.233.90.211/10.233.90.211:55694] will be ignored (a typical symptom is the Git executable not being run inside a designated container)
[Pipeline] {
Cloning the remote Git repository
Using shallow clone with depth 1
+ rm -rf /tmp/tidb_cdc_test
+ mkdir -p /tmp/tidb_cdc_test
+ chmod +x ./tests/integration_tests/run_group.sh
+ ./tests/integration_tests/run_group.sh kafka G03
Run cases: row_format drop_many_tables processor_stop_delay partition_table
PROW_JOB_ID=9567169a-1f1e-46d8-b085-8594a711db9d
JENKINS_NODE_COOKIE=f0790608-cfc9-4e58-a4e4-d1aff5e47db5
BUILD_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/
GOLANG_VERSION=1.21.0
HOSTNAME=pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr
HUDSON_SERVER_COOKIE=83ef27fe9acccc92
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT=tcp://10.233.0.1:443
TERM=xterm
STAGE_NAME=Test
BUILD_TAG=jenkins-pingcap-tiflow-pull_cdc_integration_kafka_test-2016
KUBERNETES_SERVICE_PORT=443
GIT_PREVIOUS_COMMIT=73d1a8209bddec16d8d58403efcd7a20d12cf867
JOB_SPEC={"type":"presubmit","job":"pingcap/tiflow/pull_cdc_integration_kafka_test","buildid":"1791326932713869312","prowjobid":"9567169a-1f1e-46d8-b085-8594a711db9d","refs":{"org":"pingcap","repo":"tiflow","repo_link":"https://github.com/pingcap/tiflow","base_ref":"master","base_sha":"61efa5f2b377ffcd3f35eb5f61cae9648e5023d0","base_link":"https://github.com/pingcap/tiflow/commit/61efa5f2b377ffcd3f35eb5f61cae9648e5023d0","pulls":[{"number":11129,"author":"lidezhu","sha":"38605cb8d3446efa45535be6da8d5f37679fd973","title":"*(cdc): accept access denied error when query tidb_version","link":"https://github.com/pingcap/tiflow/pull/11129","commit_link":"https://github.com/pingcap/tiflow/pull/11129/commits/38605cb8d3446efa45535be6da8d5f37679fd973","author_link":"https://github.com/lidezhu"}]}}
KUBERNETES_SERVICE_HOST=10.233.0.1
WORKSPACE=/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test
JOB_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/
RUN_CHANGES_DISPLAY_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/display/redirect?page=changes
RUN_ARTIFACTS_DISPLAY_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/display/redirect?page=artifacts
FILE_SERVER_URL=http://fileserver.pingcap.net
JENKINS_HOME=/var/jenkins_home
GIT_COMMIT=73d1a8209bddec16d8d58403efcd7a20d12cf867
PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/_utils:/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/../bin:/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/../../bin:/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/../../scripts/bin
RUN_DISPLAY_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/display/redirect
GOPROXY=http://goproxy.apps.svc,https://proxy.golang.org,direct
POD_CONTAINER=golang
PWD=/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow
HUDSON_URL=https://do.pingcap.net/jenkins/
TICDC_COVERALLS_TOKEN=****
JOB_NAME=pingcap/tiflow/pull_cdc_integration_kafka_test
TZ=Asia/Shanghai
BUILD_DISPLAY_NAME=#2016
TEST_GROUP=G03
JENKINS_URL=https://do.pingcap.net/jenkins/
BUILD_ID=1791326932713869312
TICDC_CODECOV_TOKEN=****
GOLANG_DOWNLOAD_SHA256=d0398903a16ba2232b389fb31032ddf57cac34efda306a0eebac34f0965a0742
JOB_BASE_NAME=pull_cdc_integration_kafka_test
GIT_PREVIOUS_SUCCESSFUL_COMMIT=73d1a8209bddec16d8d58403efcd7a20d12cf867
RUN_TESTS_DISPLAY_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/display/redirect?page=tests
SHLVL=5
HOME=/home/jenkins
POD_LABEL=pingcap_tiflow_pull_cdc_integration_kafka_test_2016-brh4m
GOROOT=/usr/local/go
GIT_BRANCH=origin/main
KUBERNETES_PORT_443_TCP_PROTO=tcp
TINI_VERSION=v0.19.0
CI=true
KUBERNETES_SERVICE_PORT_HTTPS=443
WORKSPACE_TMP=/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test@tmp
EXECUTOR_NUMBER=0
JENKINS_SERVER_COOKIE=durable-8f1d433d6527e85b4c28b432b07fa0c56dd3090c9176381d29a5b4531676247a
NODE_LABELS=pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr pingcap_tiflow_pull_cdc_integration_kafka_test_2016-brh4m
GIT_URL=https://github.com/PingCAP-QE/ci.git
HUDSON_HOME=/var/jenkins_home
CLASSPATH=
NODE_NAME=pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr
GOPATH=/go
JOB_DISPLAY_URL=https://do.pingcap.net/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/display/redirect
BUILD_NUMBER=2016
KUBERNETES_PORT_443_TCP_ADDR=10.233.0.1
KUBERNETES_PORT_443_TCP=tcp://10.233.0.1:443
GOLANG_DOWNLOAD_URL=https://dl.google.com/go/go1.21.0.linux-amd64.tar.gz
_=/usr/bin/env
find: '/tmp/tidb_cdc_test/*/*': No such file or directory
=================>> Running test /home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/row_format/run.sh using Sink-Type: kafka... <<=================
The 1 times to try to start tidb cluster...
Cloning repository https://github.com/PingCAP-QE/ci.git
> git init /home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test # timeout=10
Fetching upstream changes from https://github.com/PingCAP-QE/ci.git
> git --version # timeout=10
> git --version # 'git version 2.39.2'
> git fetch --tags --force --progress --depth=1 -- https://github.com/PingCAP-QE/ci.git +refs/heads/*:refs/remotes/origin/* # timeout=5
table ddl_manager.finish_mark not exists for 21-th check, retry later
[Pipeline] dir
Running in /home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow
[Pipeline] {
[Pipeline] dir
Running in /home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow
[Pipeline] {
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139b61880006 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-jwdgn-lsc23, pid:1293, start at 2024-05-17 12:54:32.29418909 +0800 CST m=+5.160800577 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:32.300 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:32.290 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:32.290 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139b61880006 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-jwdgn-lsc23, pid:1293, start at 2024-05-17 12:54:32.29418909 +0800 CST m=+5.160800577 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:32.300 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:32.290 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:32.290 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139b63140014 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-jwdgn-lsc23, pid:1377, start at 2024-05-17 12:54:32.41810116 +0800 CST m=+5.227425563 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:32.425 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:32.389 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:32.389 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v8.2.0-alpha-28-g961f9dded
Edition: Community
Git Commit Hash: 961f9dded38b814bb41b33c691ab58f3f090a0d9
Git Branch: HEAD
UTC Build Time: 2024-05-15 03:49:49
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Compiler: clang++ 13.0.0
Raft Proxy
Git Commit Hash: 7dc50b4eb06124e31f03adb06c20ff7ab61c5f79
Git Commit Branch: HEAD
UTC Build Time: 2024-05-15 03:53:57
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Enable Features: external-jemalloc portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure openssl-vendored portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure openssl-vendored
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/kv_client_stream_reconnect/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/kv_client_stream_reconnect/tiflash/log/error.log
arg matches is ArgMatches { args: {"engine-addr": MatchedArg { occurs: 1, indices: [2], vals: ["127.0.0.1:9500"] }, "addr": MatchedArg { occurs: 1, indices: [20], vals: ["127.0.0.1:9000"] }, "data-dir": MatchedArg { occurs: 1, indices: [6], vals: ["/tmp/tidb_cdc_test/kv_client_stream_reconnect/tiflash/db/proxy"] }, "advertise-addr": MatchedArg { occurs: 1, indices: [4], vals: ["127.0.0.1:9000"] }, "config": MatchedArg { occurs: 1, indices: [8], vals: ["/tmp/tidb_cdc_test/kv_client_stream_reconnect/tiflash-proxy.toml"] }, "engine-git-hash": MatchedArg { occurs: 1, indices: [10], vals: ["961f9dded38b814bb41b33c691ab58f3f090a0d9"] }, "engine-label": MatchedArg { occurs: 1, indices: [14], vals: ["tiflash"] }, "engine-version": MatchedArg { occurs: 1, indices: [12], vals: ["v8.2.0-alpha-28-g961f9dded"] }, "log-file": MatchedArg { occurs: 1, indices: [18], vals: ["/tmp/tidb_cdc_test/kv_client_stream_reconnect/tiflash/log/proxy.log"] }, "pd-endpoints": MatchedArg { occurs: 1, indices: [16], vals: ["127.0.0.1:2379"] }}, subcommand: None, usage: Some("USAGE:\n TiFlash Proxy [FLAGS] [OPTIONS] --engine-git-hash <engine-git-hash> --engine-label <engine-label> --engine-version <engine-version>") }
[Pipeline] cache
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
* Server auth using Basic with user 'ticdc'
> GET /debug/info HTTP/1.1
> Authorization: Basic dGljZGM6dGljZGNfc2VjcmV0
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 17 May 2024 04:54:33 GMT
< Content-Length: 815
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f1857546-36f7-4bcd-849c-d920894a08de
{"id":"f1857546-36f7-4bcd-849c-d920894a08de","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921669}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0bd3
f1857546-36f7-4bcd-849c-d920894a08de
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f1857546-36f7-4bcd-849c-d920894a08de
{"id":"f1857546-36f7-4bcd-849c-d920894a08de","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921669}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0bd3
f1857546-36f7-4bcd-849c-d920894a08de
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f1857546-36f7-4bcd-849c-d920894a08de
{"id":"f1857546-36f7-4bcd-849c-d920894a08de","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921669}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0bd3
f1857546-36f7-4bcd-849c-d920894a08de
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
[Fri May 17 12:54:34 CST 2024] <<<<<< START kafka consumer in changefeed_reconstruct case >>>>>>
***************** properties *****************
"mysql.port"="4000"
"dotransactions"="false"
"operationcount"="0"
"threadcount"="4"
"scanproportion"="0"
"readproportion"="0"
"mysql.user"="root"
"recordcount"="50"
"mysql.db"="changefeed_reconstruct"
"mysql.host"="127.0.0.1"
"readallfields"="true"
"requestdistribution"="uniform"
"workload"="core"
"insertproportion"="0"
"updateproportion"="0"
**********************************************
Run finished, takes 18.009504ms
INSERT - Takes(s): 0.0, Count: 48, OPS: 3421.0, Avg(us): 1347, Min(us): 884, Max(us): 3839, 95th(us): 4000, 99th(us): 4000
table changefeed_reconstruct.usertable not exists for 1-th check, retry later
check diff failed 2-th time, retry later
table ddl_manager.finish_mark not exists for 22-th check, retry later
> git config remote.origin.url https://github.com/PingCAP-QE/ci.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
Checking out Revision 73d1a8209bddec16d8d58403efcd7a20d12cf867 (origin/main)
Commit message: "update utf go build image (#2965)"
[Fri May 17 12:54:35 CST 2024] <<<<<< START cdc server in kv_client_stream_reconnect case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL'
+ GO_FAILPOINTS='github.com/pingcap/tiflow/cdc/kv/kvClientForceReconnect=return(true)'
+ [[ no != \n\o ]]
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.kv_client_stream_reconnect.28102812.out server --log-file /tmp/tidb_cdc_test/kv_client_stream_reconnect/cdc.log --log-level debug --data-dir /tmp/tidb_cdc_test/kv_client_stream_reconnect/cdc_data --cluster-id default --addr 127.0.0.1:8300 --pd http://127.0.0.1:2379
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
start tidb cluster in /tmp/tidb_cdc_test/row_format
Starting Upstream PD...
Release Version: v8.2.0-alpha-33-g79083e902
Edition: Community
Git Commit Hash: 79083e90263050e5bbfabca9aae3e63710cb1aca
Git Branch: master
UTC Build Time: 2024-05-17 04:28:56
Starting Downstream PD...
> git rev-parse origin/main^{commit} # timeout=10
> git config core.sparsecheckout # timeout=10
> git checkout -f 73d1a8209bddec16d8d58403efcd7a20d12cf867 # timeout=10
Release Version: v8.2.0-alpha-33-g79083e902
Edition: Community
Git Commit Hash: 79083e90263050e5bbfabca9aae3e63710cb1aca
Git Branch: master
UTC Build Time: 2024-05-17 04:28:56
Verifying upstream PD is started...
check diff successfully
table changefeed_reconstruct.usertable not exists for 2-th check, retry later
table ddl_manager.finish_mark not exists for 23-th check, retry later
Aborted by Jenkins Admin
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
* Server auth using Basic with user 'ticdc'
> GET /debug/info HTTP/1.1
> Authorization: Basic dGljZGM6dGljZGNfc2VjcmV0
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 17 May 2024 04:54:38 GMT
< Content-Length: 815
< Content-Type: text/plain; charset=utf-8
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
{"id":"d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921675}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e6b052c4
d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
/tidb/cdc/default/default/upstream/7369827431006411482
{"id":7369827431006411482,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
{"id":"d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921675}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e6b052c4
d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
/tidb/cdc/default/default/upstream/7369827431006411482
{"id":7369827431006411482,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'failed to get info:'
+ echo '
*** owner info ***:
*** processors info ***:
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
{"id":"d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921675}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e6b052c4
d88bf96e-7bf1-4b8b-8cef-f4e445ee9bf7
/tidb/cdc/default/default/upstream/7369827431006411482
{"id":7369827431006411482,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
[Fri May 17 12:54:38 CST 2024] <<<<<< START kafka consumer in kv_client_stream_reconnect case >>>>>>
table changefeed_reconstruct.usertable exists
check diff successfully
wait process 5873 exit for 1-th time...
table ddl_manager.finish_mark not exists for 24-th check, retry later
wait process 5873 exit for 2-th time...
/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/tests/integration_tests/_utils/kill_cdc_pid: line 19: kill: (5873) - No such process
wait process 5873 exit for 3-th time...
process 5873 already exit
check_no_capture http://127.0.0.1:2379
parse error: Invalid numeric literal at line 1, column 6
run task successfully
[Fri May 17 12:54:40 CST 2024] <<<<<< START cdc server in changefeed_reconstruct case >>>>>>
+ [[ '' == \t\r\u\e ]]
+ set +e
+ get_info_fail_msg='failed to get info:'
+ etcd_info_msg='etcd info'
+ '[' -z '' ']'
+ curl_status_cmd='curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL'
+ GO_FAILPOINTS=
+ [[ no != \n\o ]]
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.changefeed_reconstruct.61376139.out server --log-file /tmp/tidb_cdc_test/changefeed_reconstruct/cdcserver2.log --log-level debug --data-dir /tmp/tidb_cdc_test/changefeed_reconstruct/cdc_dataserver2 --cluster-id default --addr 127.0.0.1:8300
+ (( i = 0 ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connection refused
* Failed connect to 127.0.0.1:8300; Connection refused
* Closing connection 0
+ res=
+ echo ''
+ grep -q 'failed to get info:'
+ echo ''
+ grep -q 'etcd info'
+ '[' 0 -eq 50 ']'
+ sleep 3
Verifying downstream PD is started...
Starting Upstream TiKV...
TiKV
Release Version: 8.2.0-alpha
Edition: Community
Git Commit Hash: 3c2cbcf24e326ea917736538b05605c8143b6e1d
Git Commit Branch: master
UTC Build Time: 2024-05-16 09:05:19
Rust Version: rustc 1.77.0-nightly (89e2160c4 2023-12-27)
Enable Features: memory-engine pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine trace-async-tasks openssl-vendored
Profile: dist_release
Starting Downstream TiKV...
TiKV
Release Version: 8.2.0-alpha
Edition: Community
Git Commit Hash: 3c2cbcf24e326ea917736538b05605c8143b6e1d
Git Commit Branch: master
UTC Build Time: 2024-05-16 09:05:19
Rust Version: rustc 1.77.0-nightly (89e2160c4 2023-12-27)
Enable Features: memory-engine pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine trace-async-tasks openssl-vendored
Profile: dist_release
table ddl_manager.finish_mark not exists for 25-th check, retry later
Starting Upstream TiDB...
Release Version: v8.2.0-alpha-181-g491740bb83
Edition: Community
Git Commit Hash: 491740bb837bcdb5797da44954f75953b4bfe849
Git Branch: master
UTC Build Time: 2024-05-17 01:42:51
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: unistore
Starting Downstream TiDB...
Release Version: v8.2.0-alpha-181-g491740bb83
Edition: Community
Git Commit Hash: 491740bb837bcdb5797da44954f75953b4bfe849
Git Branch: master
UTC Build Time: 2024-05-17 01:42:51
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: unistore
Verifying Upstream TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
Cache restored successfully (ws/jenkins-pingcap-tiflow-pull_cdc_integration_kafka_test-2016/tiflow-cdc)
3672347648 bytes in 6.73 secs (545334681 bytes/sec)
[Pipeline] {
[Pipeline] cache
+ (( i++ ))
+ (( i <= 50 ))
++ curl -vsL --max-time 20 http://127.0.0.1:8300/debug/info --user ticdc:ticdc_secret -vsL
* About to connect() to 127.0.0.1 port 8300 (#0)
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 8300 (#0)
* Server auth using Basic with user 'ticdc'
> GET /debug/info HTTP/1.1
> Authorization: Basic dGljZGM6dGljZGNfc2VjcmV0
> User-Agent: curl/7.29.0
> Host: 127.0.0.1:8300
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 17 May 2024 04:54:43 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
<
{ [data not shown]
* Connection #0 to host 127.0.0.1 left intact
+ res='
*** owner info ***:
*** processors info ***:
changefeedID: default/c10e3a00-846a-4a85-beb1-fd05f10974f3
{UpstreamID:7369827386320086158 Namespace:default ID:c10e3a00-846a-4a85-beb1-fd05f10974f3 SinkURI:kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol&partition-num=4&kafka-version=2.4.1&max-message-bytes=10485760 CreateTime:2024-05-17 12:54:34.183859574 +0800 CST StartTs:449818571323211778 TargetTs:0 AdminJobType:noop Engine:unified SortDir: Config:0xc003a3bd40 State:normal Error:<nil> Warning:<nil> CreatorVersion:v8.2.0-alpha-47-g38605cb8d Epoch:449818571349426180}
{CheckpointTs:449818573525221378 MinTableBarrierTs:449818573525221378 AdminJobType:noop}
span: {table_id:106,start_key:7480000000000000ff6a5f720000000000fa,end_key:7480000000000000ff6a5f730000000000fa}, resolvedTs: 449818573525221378, checkpointTs: 449818573525221378, state: Replicating
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f57c3d21-745f-4131-a441-5ef82155f4fd
{"id":"f57c3d21-745f-4131-a441-5ef82155f4fd","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921680}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0c71
f57c3d21-745f-4131-a441-5ef82155f4fd
/tidb/cdc/default/default/changefeed/info/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"upstream-id":7369827386320086158,"namespace":"default","changefeed-id":"c10e3a00-846a-4a85-beb1-fd05f10974f3","sink-uri":"kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol\u0026partition-num=4\u0026kafka-version=2.4.1\u0026max-message-bytes=10485760","create-time":"2024-05-17T12:54:34.183859574+08:00","start-ts":449818571323211778,"target-ts":0,"admin-job-type":0,"sort-engine":"","sort-dir":"","config":{"memory-quota":1073741824,"case-sensitive":false,"force-replicate":false,"check-gc-safe-point":true,"enable-sync-point":false,"enable-table-monitor":false,"ignore-ineligible-table":false,"bdr-mode":false,"sync-point-interval":600000000000,"sync-point-retention":86400000000000,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"event-filters":null},"mounter":{"worker-num":16},"sink":{"protocol":"open-protocol","csv":{"delimiter":",","quote":"\"","null":"\\N","include-commit-ts":false,"binary-encoding-method":"base64","output-old-value":false,"output-handle-key":false},"encoder-concurrency":32,"terminator":"\r\n","date-separator":"day","enable-partition-separator":true,"enable-kafka-sink-v2":false,"only-output-updated-columns":false,"delete-only-output-handle-key-columns":false,"content-compatible":false,"advance-timeout-in-sec":150,"send-bootstrap-interval-in-sec":120,"send-bootstrap-in-msg-count":10000,"send-bootstrap-to-all-partition":true,"debezium-disable-schema":false,"open":{"output-old-value":true},"debezium":{"output-old-value":true}},"consistent":{"level":"none","max-log-size":64,"flush-interval":2000,"meta-flush-interval":200,"encoding-worker-num":16,"flush-worker-num":8,"storage":"","use-file-backend":false,"compression":"","memory-usage":{"memory-quota-percentage":50}},"scheduler":{"enable-table-across-nodes":false,"region-threshold":100000,"write-key-threshold":0,"region-per-span":0},"integrity":{"integrity-check-level":"none","corruption-handle-level":"warn"},"changefeed-error-stuck-duration":1800000000000,"synced-status":{"synced-check-interval":300,"checkpoint-interval":15},"sql-mode":""},"state":"normal","error":null,"warning":null,"creator-version":"v8.2.0-alpha-47-g38605cb8d","epoch":449818571349426180}
/tidb/cdc/default/default/changefeed/status/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":449818573525221378,"min-table-barrier-ts":449818573525221378,"admin-job-type":0}
/tidb/cdc/default/default/task/position/f57c3d21-745f-4131-a441-5ef82155f4fd/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":0,"resolved-ts":0,"count":0,"error":null,"warning":null}
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
changefeedID: default/c10e3a00-846a-4a85-beb1-fd05f10974f3
{UpstreamID:7369827386320086158 Namespace:default ID:c10e3a00-846a-4a85-beb1-fd05f10974f3 SinkURI:kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol&partition-num=4&kafka-version=2.4.1&max-message-bytes=10485760 CreateTime:2024-05-17 12:54:34.183859574 +0800 CST StartTs:449818571323211778 TargetTs:0 AdminJobType:noop Engine:unified SortDir: Config:0xc003a3bd40 State:normal Error:<nil> Warning:<nil> CreatorVersion:v8.2.0-alpha-47-g38605cb8d Epoch:449818571349426180}
{CheckpointTs:449818573525221378 MinTableBarrierTs:449818573525221378 AdminJobType:noop}
span: {table_id:106,start_key:7480000000000000ff6a5f720000000000fa,end_key:7480000000000000ff6a5f730000000000fa}, resolvedTs: 449818573525221378, checkpointTs: 449818573525221378, state: Replicating
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f57c3d21-745f-4131-a441-5ef82155f4fd
{"id":"f57c3d21-745f-4131-a441-5ef82155f4fd","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921680}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0c71
f57c3d21-745f-4131-a441-5ef82155f4fd
/tidb/cdc/default/default/changefeed/info/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"upstream-id":7369827386320086158,"namespace":"default","changefeed-id":"c10e3a00-846a-4a85-beb1-fd05f10974f3","sink-uri":"kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol\u0026partition-num=4\u0026kafka-version=2.4.1\u0026max-message-bytes=10485760","create-time":"2024-05-17T12:54:34.183859574+08:00","start-ts":449818571323211778,"target-ts":0,"admin-job-type":0,"sort-engine":"","sort-dir":"","config":{"memory-quota":1073741824,"case-sensitive":false,"force-replicate":false,"check-gc-safe-point":true,"enable-sync-point":false,"enable-table-monitor":false,"ignore-ineligible-table":false,"bdr-mode":false,"sync-point-interval":600000000000,"sync-point-retention":86400000000000,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"event-filters":null},"mounter":{"worker-num":16},"sink":{"protocol":"open-protocol","csv":{"delimiter":",","quote":"\"","null":"\\N","include-commit-ts":false,"binary-encoding-method":"base64","output-old-value":false,"output-handle-key":false},"encoder-concurrency":32,"terminator":"\r\n","date-separator":"day","enable-partition-separator":true,"enable-kafka-sink-v2":false,"only-output-updated-columns":false,"delete-only-output-handle-key-columns":false,"content-compatible":false,"advance-timeout-in-sec":150,"send-bootstrap-interval-in-sec":120,"send-bootstrap-in-msg-count":10000,"send-bootstrap-to-all-partition":true,"debezium-disable-schema":false,"open":{"output-old-value":true},"debezium":{"output-old-value":true}},"consistent":{"level":"none","max-log-size":64,"flush-interval":2000,"meta-flush-interval":200,"encoding-worker-num":16,"flush-worker-num":8,"storage":"","use-file-backend":false,"compression":"","memory-usage":{"memory-quota-percentage":50}},"scheduler":{"enable-table-across-nodes":false,"region-threshold":100000,"write-key-threshold":0,"region-per-span":0},"integrity":{"integrity-check-level":"none","corruption-handle-level":"warn"},"changefeed-error-stuck-duration":1800000000000,"synced-status":{"synced-check-interval":300,"checkpoint-interval":15},"sql-mode":""},"state":"normal","error":null,"warning":null,"creator-version":"v8.2.0-alpha-47-g38605cb8d","epoch":449818571349426180}
/tidb/cdc/default/default/changefeed/status/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":449818573525221378,"min-table-barrier-ts":449818573525221378,"admin-job-type":0}
/tidb/cdc/default/default/task/position/f57c3d21-745f-4131-a441-5ef82155f4fd/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":0,"resolved-ts":0,"count":0,"error":null,"warning"+ grep -q 'failed to get info:'
:null}
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ echo '
*** owner info ***:
*** processors info ***:
changefeedID: default/c10e3a00-846a-4a85-beb1-fd05f10974f3
{UpstreamID:7369827386320086158 Namespace:default ID:c10e3a00-846a-4a85-beb1-fd05f10974f3 SinkURI:kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol&partition-num=4&kafka-version=2.4.1&max-message-bytes=10485760 CreateTime:2024-05-17 12:54:34.183859574 +0800 CST StartTs:449818571323211778 TargetTs:0 AdminJobType:noop Engine:unified SortDir: Config:0xc003a3bd40 State:normal Error:<nil> Warning:<nil> CreatorVersion:v8.2.0-alpha-47-g38605cb8d Epoch:449818571349426180}
{CheckpointTs:449818573525221378 MinTableBarrierTs:449818573525221378 AdminJobType:noop}
span: {table_id:106,start_key:7480000000000000ff6a5f720000000000fa,end_key:7480000000000000ff6a5f730000000000fa}, resolvedTs: 449818573525221378, checkpointTs: 449818573525221378, state: Replicating
*** etcd info ***:
/tidb/cdc/default/__cdc_meta__/capture/f57c3d21-745f-4131-a441-5ef82155f4fd
{"id":"f57c3d21-745f-4131-a441-5ef82155f4fd","address":"127.0.0.1:8300","version":"v8.2.0-alpha-47-g38605cb8d","git-hash":"38605cb8d3446efa45535be6da8d5f37679fd973","deploy-path":"/home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test/tiflow/bin/cdc.test","start-timestamp":1715921680}
/tidb/cdc/default/__cdc_meta__/meta/meta-version
1
/tidb/cdc/default/__cdc_meta__/owner/22318f84e68c0c71
f57c3d21-745f-4131-a441-5ef82155f4fd
/tidb/cdc/default/default/changefeed/info/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"upstream-id":7369827386320086158,"namespace":"default","changefeed-id":"c10e3a00-846a-4a85-beb1-fd05f10974f3","sink-uri":"kafka://127.0.0.1:9092/ticdc-changefeed-reconstruct-5206?protocol=open-protocol\u0026partition-num=4\u0026kafka-version=2.4.1\u0026max-message-bytes=10485760","create-time":"2024-05-17T12:54:34.183859574+08:00","start-ts":449818571323211778,"target-ts":0,"admin-job-type":0,"sort-engine":"","sort-dir":"","config":{"memory-quota":1073741824,"case-sensitive":false,"force-replicate":false,"check-gc-safe-point":true,"enable-sync-point":false,"enable-table-monitor":false,"ignore-ineligible-table":false,"bdr-mode":false,"sync-point-interval":600000000000,"sync-point-retention":86400000000000,"filter":{"rules":["*.*"],"ignore-txn-start-ts":null,"event-filters":null},"mounter":{"worker-num":16},"sink":{"protocol":"open-protocol","csv":{"delimiter":",","quote":"\"","null":"\\N","include-commit-ts":false,"binary-encoding-method":"base64","output-old-value":false,"output-handle-key":false},"encoder-concurrency":32,"terminator":"\r\n","date-separator":"day","enable-partition-separator":true,"enable-kafka-sink-v2":false,"only-output-updated-columns":false,"delete-only-output-handle-key-columns":false,"content-compatible":false,"advance-timeout-in-sec":150,"send-bootstrap-interval-in-sec":120,"send-bootstrap-in-msg-count":10000,"send-bootstrap-to-all-partition":true,"debezium-disable-schema":false,"open":{"output-old-value":true},"debezium":{"output-old-value":true}},"consistent":{"level":"none","max-log-size":64,"flush-interval":2000,"meta-flush-interval":200,"encoding-worker-num":16,"flush-worker-num":8,"storage":"","use-file-backend":false,"compression":"","memory-usage":{"memory-quota-percentage":50}},"scheduler":{"enable-table-across-nodes":false,"region-threshold":100000,"write-key-threshold":0,"region-per-span":0},"integrity":{"integrity-check-level":"none","corruption-handle-level":"warn"},"changefeed-error-stuck-duration":1800000000000,"synced-status":{"synced-check-interval":300,"checkpoint-interval":15},"sql-mode":""},"state":"normal","error":null,"warning":null,"creator-version":"v8.2.0-alpha-47-g38605cb8d","epoch":449818571349426180}
/tidb/cdc/default/default/changefeed/status/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":449818573525221378,"min-table-barrier-ts":449818573525221378,"admin-job-type":0}
/tidb/cdc/default/default/task/position/f57c3d21-745f-4131-a441-5ef82155f4fd/c10e3a00-846a-4a85-beb1-fd05f10974f3
{"checkpoint-ts":0,"resolved-ts":0,"count":0,"error":null,"warning":null}
/tidb/cdc/default/default/upstream/7369827386320086158
{"id":7369827386320086158,"pd-endpoints":"http://127.0.0.1:2379,http://127.0.0.1:2379","key-path":"","cert-path":"","ca-path":"","cert-allowed-cn":null}'
+ grep -q 'etcd info'
+ break
+ set +x
cdc.test cli capture list --pd=http://127.0.0.1:2379 2>&1 | grep id
table ddl_manager.finish_mark not exists for 26-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
"id": "f57c3d21-745f-4131-a441-5ef82155f4fd",
"cluster-id": "default"
run task successfully
table ddl_manager.finish_mark not exists for 27-th check, retry later
capture_id: f57c3d21-745f-4131-a441-5ef82155f4fd
check_processor_table_count http://127.0.0.1:2379 c10e3a00-846a-4a85-beb1-fd05f10974f3 f57c3d21-745f-4131-a441-5ef82155f4fd 1
run task successfully
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
check_processor_table_count http://127.0.0.1:2379 c10e3a00-846a-4a85-beb1-fd05f10974f3 f57c3d21-745f-4131-a441-5ef82155f4fd 0
table count 1 does equal to expected count 0
run task failed 1-th time, retry later
table ddl_manager.finish_mark not exists for 28-th check, retry later
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
check_processor_table_count http://127.0.0.1:2379 c10e3a00-846a-4a85-beb1-fd05f10974f3 f57c3d21-745f-4131-a441-5ef82155f4fd 0
run task successfully
***************** properties *****************
"mysql.port"="4000"
"recordcount"="50"
"readallfields"="true"
"threadcount"="4"
"scanproportion"="0"
"insertproportion"="0"
"mysql.host"="127.0.0.1"
"operationcount"="0"
"readproportion"="0"
"mysql.user"="root"
"requestdistribution"="uniform"
"updateproportion"="0"
"mysql.db"="changefeed_reconstruct"
"dotransactions"="false"
"workload"="core"
**********************************************
Run finished, takes 16.098081ms
INSERT - Takes(s): 0.0, Count: 48, OPS: 3805.5, Avg(us): 1275, Min(us): 892, Max(us): 3540, 95th(us): 4000, 99th(us): 4000
table changefeed_reconstruct.usertable not exists for 1-th check, retry later
table ddl_manager.finish_mark not exists for 29-th check, retry later
Agent pingcap-tiflow-pull-cdc-integration-kafka-test-2016-wg6j1-gv7fk is provisioned from template pingcap_tiflow_pull_cdc_integration_kafka_test_2016-wg6j1-962nk
---
apiVersion: "v1"
kind: "Pod"
metadata:
annotations:
buildUrl: "http://jenkins.apps.svc.cluster.local:8080/jenkins/job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/"
runUrl: "job/pingcap/job/tiflow/job/pull_cdc_integration_kafka_test/2016/"
labels:
jenkins/jenkins-jenkins-agent: "true"
jenkins/label-digest: "f24fbe1e3d461a1aa9d9fd77c62f6352fa24885b"
jenkins/label: "pingcap_tiflow_pull_cdc_integration_kafka_test_2016-wg6j1"
name: "pingcap-tiflow-pull-cdc-integration-kafka-test-2016-wg6j1-gv7fk"
namespace: "jenkins-tiflow"
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "kubernetes.io/arch"
operator: "In"
values:
- "amd64"
containers:
- image: "wurstmeister/zookeeper"
imagePullPolicy: "IfNotPresent"
name: "zookeeper"
resources:
limits:
cpu: "2000m"
memory: "4Gi"
requests:
cpu: "2000m"
memory: "4Gi"
tty: true
volumeMounts:
- mountPath: "/tmp"
name: "volume-0"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- args:
- "cat"
image: "hub.pingcap.net/jenkins/golang-tini:1.21"
imagePullPolicy: "Always"
name: "golang"
resources:
limits:
cpu: "12"
memory: "32Gi"
requests:
cpu: "12"
memory: "32Gi"
tty: true
volumeMounts:
- mountPath: "/tmp"
name: "volume-0"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "KAFKA_CREATE_TOPICS"
value: "big-message-test:1:1"
- name: "KAFKA_BROKER_ID"
value: "1"
- name: "KAFKA_SSL_KEYSTORE_PASSWORD"
value: "test1234"
- name: "KAFKA_ZOOKEEPER_CONNECT"
value: "localhost:2181"
- name: "KAFKA_MESSAGE_MAX_BYTES"
value: "11534336"
- name: "KAFKA_REPLICA_FETCH_MAX_BYTES"
value: "11534336"
- name: "KAFKA_ADVERTISED_LISTENERS"
value: "SSL://127.0.0.1:9093,PLAINTEXT://127.0.0.1:9092"
- name: "ZK"
value: "zk"
- name: "KAFKA_SSL_KEYSTORE_LOCATION"
value: "/tmp/kafka.server.keystore.jks"
- name: "KAFKA_SSL_KEY_PASSWORD"
value: "test1234"
- name: "KAFKA_SSL_TRUSTSTORE_PASSWORD"
value: "test1234"
- name: "KAFKA_LISTENERS"
value: "SSL://127.0.0.1:9093,PLAINTEXT://127.0.0.1:9092"
- name: "KAFKA_SSL_TRUSTSTORE_LOCATION"
value: "/tmp/kafka.server.truststore.jks"
- name: "RACK_COMMAND"
value: "curl -sfL https://github.com/pingcap/tiflow/raw/6e62afcfecc4e3965d8818784327d4bf2600d9fa/tests/_certificates/kafka.server.keystore.jks\
\ -o /tmp/kafka.server.keystore.jks && curl -sfL https://github.com/pingcap/tiflow/raw/6e62afcfecc4e3965d8818784327d4bf2600d9fa/tests/_certificates/kafka.server.truststore.jks\
\ -o /tmp/kafka.server.truststore.jks"
image: "wurstmeister/kafka:2.12-2.4.1"
imagePullPolicy: "IfNotPresent"
name: "kafka"
resources:
limits:
cpu: "4000m"
memory: "6Gi"
requests:
cpu: "4000m"
memory: "6Gi"
tty: true
volumeMounts:
- mountPath: "/tmp"
name: "volume-0"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "KAFKA_SERVER"
value: "127.0.0.1:9092"
- name: "ZOOKEEPER_SERVER"
value: "127.0.0.1:2181"
- name: "DOWNSTREAM_DB_HOST"
value: "127.0.0.1"
- name: "USE_FLAT_MESSAGE"
value: "true"
- name: "DOWNSTREAM_DB_PORT"
value: "3306"
- name: "DB_NAME"
value: "test"
image: "rustinliu/ticdc-canal-json-adapter:latest"
imagePullPolicy: "IfNotPresent"
name: "canal-adapter"
resources:
requests:
cpu: "200m"
memory: "4Gi"
tty: true
volumeMounts:
- mountPath: "/tmp"
name: "volume-0"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- image: "hub.pingcap.net/jenkins/network-multitool"
name: "net-tool"
resources:
limits:
memory: "128Mi"
cpu: "100m"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- image: "hub.pingcap.net/jenkins/python3-requests:latest"
name: "report"
resources:
requests:
cpu: "200m"
memory: "4Gi"
tty: true
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "MYSQL_ROOT_PASSWORD"
value: ""
- name: "MYSQL_USER"
value: "mysqluser"
- name: "MYSQL_PASSWORD"
value: "mysqlpw"
- name: "MYSQL_ALLOW_EMPTY_PASSWORD"
value: "yes"
- name: "MYSQL_TCP_PORT"
value: "3310"
image: "quay.io/debezium/example-mysql:2.4"
imagePullPolicy: "IfNotPresent"
name: "mysql"
resources:
requests:
cpu: "200m"
memory: "4Gi"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "BOOTSTRAP_SERVERS"
value: "127.0.0.1:9092"
- name: "GROUP_ID"
value: "1"
- name: "CONFIG_STORAGE_TOPIC"
value: "my_connect_configs"
- name: "OFFSET_STORAGE_TOPIC"
value: "my_connect_offsets"
- name: "STATUS_STORAGE_TOPIC"
value: "my_connect_statuses"
image: "quay.io/debezium/connect:2.4"
name: "connect"
resources:
requests:
cpu: "200m"
memory: "4Gi"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
- env:
- name: "JENKINS_SECRET"
value: "********"
- name: "JENKINS_TUNNEL"
value: "jenkins-agent.apps.svc.cluster.local:50000"
- name: "JENKINS_AGENT_NAME"
value: "pingcap-tiflow-pull-cdc-integration-kafka-test-2016-wg6j1-gv7fk"
- name: "JENKINS_NAME"
value: "pingcap-tiflow-pull-cdc-integration-kafka-test-2016-wg6j1-gv7fk"
- name: "JENKINS_AGENT_WORKDIR"
value: "/home/jenkins/agent"
- name: "JENKINS_URL"
value: "http://jenkins.apps.svc.cluster.local:8080/jenkins/"
image: "jenkins/inbound-agent:3206.vb_15dcf73f6a_9-2"
name: "jnlp"
resources:
requests:
memory: "256Mi"
cpu: "100m"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
restartPolicy: "Never"
securityContext:
fsGroup: 1000
volumes:
- emptyDir: {}
name: "volume-0"
- emptyDir:
medium: ""
name: "workspace-volume"
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139c50bc0015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr, pid:1410, start at 2024-05-17 12:54:47.628196506 +0800 CST m=+5.279611355 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:47.637 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:47.599 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:47.599 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139c50bc0015 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr, pid:1410, start at 2024-05-17 12:54:47.628196506 +0800 CST m=+5.279611355 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:47.637 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:47.599 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:47.599 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Verifying Downstream TiDB is started...
VARIABLE_NAME VARIABLE_VALUE COMMENT
bootstrapped True Bootstrap flag. Do not delete.
tidb_server_version 198 Bootstrap version. Do not delete.
system_tz Asia/Shanghai TiDB Global System Timezone.
new_collation_enabled True If the new collations are enabled. Do not edit it.
ddl_table_version 3 DDL Table Version. Do not delete.
tikv_gc_leader_uuid 63e139c52400007 Current GC worker leader UUID. (DO NOT EDIT)
tikv_gc_leader_desc host:pingcap-tiflow-pull-cdc-integration-kafka-test-2016-brh4m-b5qcr, pid:1488, start at 2024-05-17 12:54:47.704776948 +0800 CST m=+5.300515246 Host name and pid of current GC leader. (DO NOT EDIT)
tikv_gc_leader_lease 20240517-12:56:47.714 +0800 Current GC worker leader lease. (DO NOT EDIT)
tikv_gc_auto_concurrency true Let TiDB pick the concurrency automatically. If set false, tikv_gc_concurrency will be used
tikv_gc_enable true Current GC enable status
tikv_gc_run_interval 10m0s GC run interval, at least 10m, in Go format.
tikv_gc_life_time 10m0s All versions within life time will not be collected by GC, at least 10m, in Go format.
tikv_gc_last_run_time 20240517-12:54:47.696 +0800 The time when last GC starts. (DO NOT EDIT)
tikv_gc_safe_point 20240517-12:44:47.696 +0800 All versions after safe point can be accessed. (DO NOT EDIT)
Starting Upstream TiFlash...
TiFlash
Release Version: v8.2.0-alpha-28-g961f9dded
Edition: Community
Git Commit Hash: 961f9dded38b814bb41b33c691ab58f3f090a0d9
Git Branch: HEAD
UTC Build Time: 2024-05-15 03:49:49
Enable Features: jemalloc sm4(GmSSL) avx2 avx512 unwind thinlto
Profile: RELWITHDEBINFO
Compiler: clang++ 13.0.0
Raft Proxy
Git Commit Hash: 7dc50b4eb06124e31f03adb06c20ff7ab61c5f79
Git Commit Branch: HEAD
UTC Build Time: 2024-05-15 03:53:57
Rust Version: rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Storage Engine: tiflash
Prometheus Prefix: tiflash_proxy_
Profile: release
Enable Features: external-jemalloc portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure openssl-vendored portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure openssl-vendored
Verifying Upstream TiFlash is started...
Logging trace to /tmp/tidb_cdc_test/row_format/tiflash/log/server.log
Logging errors to /tmp/tidb_cdc_test/row_format/tiflash/log/error.log
arg matches is ArgMatches { args: {"pd-endpoints": MatchedArg { occurs: 1, indices: [16], vals: ["127.0.0.1:2379"] }, "engine-addr": MatchedArg { occurs: 1, indices: [2], vals: ["127.0.0.1:9500"] }, "engine-git-hash": MatchedArg { occurs: 1, indices: [10], vals: ["961f9dded38b814bb41b33c691ab58f3f090a0d9"] }, "engine-label": MatchedArg { occurs: 1, indices: [14], vals: ["tiflash"] }, "log-file": MatchedArg { occurs: 1, indices: [18], vals: ["/tmp/tidb_cdc_test/row_format/tiflash/log/proxy.log"] }, "addr": MatchedArg { occurs: 1, indices: [20], vals: ["127.0.0.1:9000"] }, "config": MatchedArg { occurs: 1, indices: [8], vals: ["/tmp/tidb_cdc_test/row_format/tiflash-proxy.toml"] }, "advertise-addr": MatchedArg { occurs: 1, indices: [4], vals: ["127.0.0.1:9000"] }, "data-dir": MatchedArg { occurs: 1, indices: [6], vals: ["/tmp/tidb_cdc_test/row_format/tiflash/db/proxy"] }, "engine-version": MatchedArg { occurs: 1, indices: [12], vals: ["v8.2.0-alpha-28-g961f9dded"] }}, subcommand: None, usage: Some("USAGE:\n TiFlash Proxy [FLAGS] [OPTIONS] --engine-git-hash <engine-git-hash> --engine-label <engine-label> --engine-version <engine-version>") }
Running on pingcap-tiflow-pull-cdc-integration-kafka-test-2016-wg6j1-gv7fk in /home/jenkins/agent/workspace/pingcap/tiflow/pull_cdc_integration_kafka_test
Cache restored successfully (ws/jenkins-pingcap-tiflow-pull_cdc_integration_kafka_test-2016/tiflow-cdc)
3672347648 bytes in 5.75 secs (639052532 bytes/sec)
[Pipeline] {
[Pipeline] container
[Pipeline] {
[Pipeline] container
[Pipeline] {
Sending interrupt signal to process
Killing processes
table ddl_manager.finish_mark not exists for 30-th check, retry later
table changefeed_reconstruct.usertable not exists for 2-th check, retry later
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
{"level":"warn","ts":1715921692.8527195,"caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc00226b500/127.0.0.1:2379","attempt":0,"error":"rpc error: code = Unavailable desc = error reading from server: EOF"}
script returned exit code 143
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
+ pd_host=127.0.0.1
+ pd_port=2379
+ is_tls=false
+ '[' false == true ']'
++ run_cdc_cli tso query --pd=http://127.0.0.1:2379
+ cdc.test -test.coverprofile=/tmp/tidb_cdc_test/cov.row_format.cli.2815.out cli tso query --pd=http://127.0.0.1:2379
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
Click here to forcibly terminate running steps
{"level":"warn","ts":1715921693.123908,"caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0021bb500/127.0.0.1:2379","attempt":0,"error":"rpc error: code = Unavailable desc = error reading from server: EOF"}
script returned exit code 143
script returned exit code 143
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
table ddl_manager.finish_mark not exists for 31-th check, retry later
script returned exit code 143
kill finished with exit code 0
{"level":"warn","ts":1715921694.0993383,"caller":"v3@v3.5.12/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc002381340/127.0.0.1:2379","attempt":0,"error":"rpc error: code = Unavailable desc = error reading from server: EOF"}
script returned exit code 143
[Pipeline] }
[Pipeline] }
[Pipeline] }
script returned exit code 143
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // container
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G01'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G02'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G09'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G10'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G11'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G12'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G13'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G15'
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G00'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G03'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G04'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G05'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G06'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G07'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G08'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G14'
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 146f93d7-a6f3-49d8-8f2c-5e394d254831
Failed in branch Matrix - TEST_GROUP = 'G01'
Finished: ABORTED
![[Jenkins]](/jenkins/static/62b7ca6c/images/svgs/logo.svg){kind=logo}
