Skipping 493 KB.. Full Log

 > git checkout -f 8c60e558a9356c5a25645eebaade802e946e45b6 # timeout=10
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/source1.yaml"
wait for rpc addr 127.0.0.1:8261 alive the 2-th time
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/case_sensitive/source2.yaml"
rpc addr 127.0.0.1:8261 is alive
[Sun Apr 28 11:09:03 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-task-no-validator.yaml --remove-meta"
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
dmctl test cmd: "start-task /tmp/dm_test/case_sensitive/dm-task.yaml --remove-meta"
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/shardddl3/source1.yaml"
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
check diff successfully
wait pattern dm-worker1.toml exit...
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
wait pattern dm-worker1.toml exit...
pattern dm-worker1.toml already exit
[Sun Apr 28 11:09:05 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/case_sensitive/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
make: *** [dm_integration_test_in_group] Terminated
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/start_task/../_utils/test_prepare: line 232:   635 Terminated              sleep 2
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
script returned exit code 143
kill finished with exit code 0
[Pipeline] }
[Pipeline] // cache
[Pipeline] }
make: *** [dm_integration_test_in_group] Terminated
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/case_sensitive/../_utils/test_prepare: line 4: 12896 Terminated              check_rpc_alive $cur/../bin/check_worker_online 127.0.0.1:$WORKER1_PORT
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
script returned exit code 143
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
[Pipeline] // withCredentials
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] withEnv
[Pipeline] {
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] container
[Pipeline] {
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // timeout
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // cache
[Pipeline] }
[Pipeline] // cache
[Pipeline] }
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] timeout
Timeout set to expire in 40 min
[Pipeline] {
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] withCredentials
Masking supported pattern matches of $DM_CODECOV_TOKEN or $DM_COVERALLS_TOKEN
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] {
[Pipeline] // container
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // node
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] container
[Pipeline] {
[Pipeline] // stage
[Pipeline] }
[Pipeline] sh
[Pipeline] // stage
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] // container
+ mkdir /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/mysql-ssl
+ cp -r /var/lib/mysql/ca-key.pem /var/lib/mysql/ca.pem /var/lib/mysql/client-cert.pem /var/lib/mysql/client-key.pem /var/lib/mysql/private_key.pem /var/lib/mysql/public_key.pem /var/lib/mysql/server-cert.pem /var/lib/mysql/server-key.pem /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/mysql-ssl/
+ ls -alh /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/mysql-ssl/
total 40K
drwxr-sr-x.  2 root 1000 4.0K Apr 28 03:09 .
drwxr-sr-x. 17 1000 1000 4.0K Apr 28 03:09 ..
-rw-------.  1 root 1000 1.7K Apr 28 03:09 ca-key.pem
-rw-r--r--.  1 root 1000 1.1K Apr 28 03:09 ca.pem
-rw-r--r--.  1 root 1000 1.1K Apr 28 03:09 client-cert.pem
-rw-------.  1 root 1000 1.7K Apr 28 03:09 client-key.pem
-rw-------.  1 root 1000 1.7K Apr 28 03:09 private_key.pem
-rw-r--r--.  1 root 1000  451 Apr 28 03:09 public_key.pem
-rw-r--r--.  1 root 1000 1.1K Apr 28 03:09 server-cert.pem
-rw-------.  1 root 1000 1.7K Apr 28 03:09 server-key.pem
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] dir
Running in /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow
[Pipeline] {
[Pipeline] cache
Cache restored successfully (ws/jenkins-pingcap-tiflow-release-7.1-pull_dm_integration_test-192/tiflow-dm)
2783003136 bytes in 19.61 secs (141903081 bytes/sec)
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 10 min
[Pipeline] {
[Pipeline] sh
[Pipeline] // stage
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
+ pwd
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow
+ ls -alh
total 408K
drwxr-sr-x. 18 jenkins jenkins 4.0K Apr 28 11:09 .
drwxr-sr-x. 19 jenkins jenkins 4.0K Apr 28 11:09 ..
-rw-r--r--.  1 jenkins jenkins  858 Apr 28 10:46 .codecov.yml
lrwxrwxrwx.  1 jenkins jenkins   10 Apr 28 11:09 .dockerignore -> .gitignore
-rw-r--r--.  1 jenkins jenkins  162 Apr 28 10:46 .editorconfig
drwxr-sr-x.  7 jenkins jenkins 4.0K Apr 28 11:09 .git
drwxr-sr-x.  4 jenkins jenkins 4.0K Apr 28 11:09 .github
-rw-r--r--.  1 jenkins jenkins  925 Apr 28 10:46 .gitignore
-rw-r--r--.  1 jenkins jenkins 2.9K Apr 28 10:46 .golangci.yml
-rw-r--r--.  1 jenkins jenkins  156 Apr 28 10:46 CODE_OF_CONDUCT.md
-rw-r--r--.  1 jenkins jenkins 3.5K Apr 28 10:46 CONTRIBUTING.md
-rw-r--r--.  1 jenkins jenkins  12K Apr 28 10:46 LICENSE
-rw-r--r--.  1 jenkins jenkins  24K Apr 28 10:46 Makefile
-rw-r--r--.  1 jenkins jenkins 2.7K Apr 28 10:46 Makefile.engine
-rw-r--r--.  1 jenkins jenkins  693 Apr 28 10:46 OWNERS
-rw-r--r--.  1 jenkins jenkins 1.5K Apr 28 10:46 README.md
-rw-r--r--.  1 jenkins jenkins 2.1K Apr 28 10:46 README_DM.md
-rw-r--r--.  1 jenkins jenkins 5.2K Apr 28 10:46 README_Engine.md
-rw-r--r--.  1 jenkins jenkins 2.8K Apr 28 10:46 README_TiCDC.md
-rw-r--r--.  1 jenkins jenkins 1.5K Apr 28 10:46 SECURITY.md
drwxr-sr-x.  3 jenkins jenkins 4.0K Apr 28 11:09 bin
drwxr-sr-x. 18 jenkins jenkins 4.0K Apr 28 11:09 cdc
drwxr-sr-x. 10 jenkins jenkins 4.0K Apr 28 11:09 cmd
drwxr-sr-x.  4 jenkins jenkins 4.0K Apr 28 11:09 deployments
drwxr-sr-x. 26 jenkins jenkins 4.0K Apr 28 11:09 dm
drwxr-sr-x.  7 jenkins jenkins 4.0K Apr 28 11:09 docs
drwxr-sr-x. 13 jenkins jenkins 4.0K Apr 28 11:09 engine
-rwxr-xr-x.  1 jenkins jenkins  25K Apr 28 10:46 errors.toml
drwxr-sr-x.  3 jenkins jenkins 4.0K Apr 28 11:09 examples
-rw-r--r--.  1 jenkins jenkins  17K Apr 28 10:46 go.mod
-rw-r--r--.  1 jenkins jenkins 190K Apr 28 10:46 go.sum
drwxr-sr-x.  4 jenkins jenkins 4.0K Apr 28 11:09 metrics
drwxr-sr-x. 44 jenkins jenkins 4.0K Apr 28 11:09 pkg
drwxr-sr-x.  5 jenkins jenkins 4.0K Apr 28 11:09 proto
drwxr-sr-x.  3 jenkins jenkins 4.0K Apr 28 11:09 scripts
drwxr-sr-x.  5 jenkins jenkins 4.0K Apr 28 11:09 tests
-rw-r--r--.  1 jenkins jenkins  582 Apr 28 10:46 third-party-license.txt
drwxr-sr-x.  4 jenkins jenkins 4.0K Apr 28 11:09 tools
+ set +e
+ for i in '{1..90}'
+ mysqladmin ping -h127.0.0.1 -P 3306 -p123456 -uroot --silent
mysqld is alive
+ '[' 0 -eq 0 ']'
+ set -e
+ break
+ set +e
+ for i in '{1..90}'
+ mysqladmin ping -h127.0.0.1 -P 3307 -p123456 -uroot --silent
mysqld is alive
+ '[' 0 -eq 0 ']'
+ set -e
+ break
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G06'
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // timeout
[Pipeline] sh
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
+ '[' TLS_GROUP == G02 ']'
+ echo 'run G02 test'
run G02 test
+ export PATH=/usr/local/go/bin:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/usr/local/go/bin:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ mkdir -p ./dm/tests/bin
+ cp -r ./bin/dm-test-tools/check_exit_safe_binlog ./bin/dm-test-tools/check_master_http_apis ./bin/dm-test-tools/check_master_online ./bin/dm-test-tools/check_master_online_http ./bin/dm-test-tools/check_worker_online ./dm/tests/bin/
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
+ make dm_integration_test_in_group GROUP=G02
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/tidb-server
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/sync_diff_inspector
/usr/bin/mysql
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/minio
install python requirments for test
pip install --user -q -r ./dm/tests/requirements.txt
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G00'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G01'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G03'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G05'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G08'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G10'
[Pipeline] // stage
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G11'
[Pipeline] // stage
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G09'
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/dm-master.test
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/dm-worker.test
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/bin/dm-syncer.test
cd dm && ln -sf ../bin .
cd dm && ./tests/run_group.sh G02
Run cases: handle_error handle_error_2 handle_error_3
...
rm: cannot remove '/tmp/dm_test/tidb.toml': No such file or directory
Starting TiDB on port 4000
Verifying TiDB is started...
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
VARIABLE_NAME	VARIABLE_VALUE	COMMENT
bootstrapped	True	Bootstrap flag. Do not delete.
tidb_server_version	146	Bootstrap version. Do not delete.
system_tz	Asia/Shanghai	TiDB Global System Timezone.
new_collation_enabled	True	If the new collations are enabled. Do not edit it.
version()
5.7.26-log
version()
8.0.21
start running case: [handle_error] script: [/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/run.sh]
Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/run.sh...
Verbose mode = false
0 dm-master alive
0 dm-worker alive
0 dm-syncer alive
process dm-master.test already exit
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:09:43 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
rpc addr 127.0.0.1:8261 is alive
[Sun Apr 28 11:09:44 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error/source1.yaml"
[Sun Apr 28 11:09:45 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error/source2.yaml"
[Sun Apr 28 11:09:48 CST 2024] <<<<<< start DM-INJECT_DDL_ERROR  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -s mysql-replica-01 alter table handle_error.tb1 add column c int default 100; alter table handle_error.tb1 add primary key (c);"
dmctl test cmd: "binlog replace test -s mysql-replica-02 alter table handle_error.tb2 add column c int default 100; alter table handle_error.tb2 add primary key (c);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog inject test -s mysql-replica-01 alter table handle_error.tb1 drop primary key; alter table handle_error.tb1 add unique(c);"
dmctl test cmd: "binlog inject test -s mysql-replica-02 alter table handle_error.tb2 drop primary key; alter table handle_error.tb2 add unique(c);"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:09:52 CST 2024] <<<<<< finish DM-INJECT_DDL_ERROR  >>>>>>
[Sun Apr 28 11:09:52 CST 2024] <<<<<< start DM-INJECT_DDL_ERROR_SHARDING_BASE pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "unlock-ddl-lock test-`handle_error`.`tb` --force-remove"
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "unlock-ddl-lock test-`handle_error`.`tb` --force-remove"
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=4 expected=4
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:18 CST 2024] <<<<<< finish DM-INJECT_DDL_ERROR_SHARDING_BASE pessimistic >>>>>>
[Sun Apr 28 11:10:18 CST 2024] <<<<<< start DM-INJECT_DDL_ERROR_SHARDING_BASE optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c int default 100; alter table handle_error.tb1 add primary key (c);"
dmctl test cmd: "query-status test"
got=4 expected=4
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:25 CST 2024] <<<<<< finish DM-INJECT_DDL_ERROR_SHARDING_BASE optimistic >>>>>>
[Sun Apr 28 11:10:25 CST 2024] <<<<<< start DM-INJECT_DDL_ERROR_SHARDING_BASE2 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20); alter table handle_error.tb1 add primary key (c);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -s mysql-replica-01,mysql-replica-02 alter table handle_error.tb2 add column c varchar(20); alter table handle_error.tb2 add primary key (c);"
dmctl test cmd: "query-status test"
got=4 expected=4
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:30 CST 2024] <<<<<< finish DM-INJECT_DDL_ERROR_SHARDING_BASE2 pessimistic >>>>>>
[Sun Apr 28 11:10:30 CST 2024] <<<<<< start DM-INJECT_DDL_ERROR_SHARDING_BASE2 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20); alter table handle_error.tb1 add primary key (c);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb2 add column c varchar(20); alter table handle_error.tb2 add primary key (c);"
check log contain failed 1-th time, retry later
dmctl test cmd: "query-status test"
got=4 expected=4
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:35 CST 2024] <<<<<< finish DM-INJECT_DDL_ERROR_SHARDING_BASE2 optimistic >>>>>>
[Sun Apr 28 11:10:35 CST 2024] <<<<<< start DM-LIST_ERROR  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog list test"
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:14435"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog list test"
dmctl test cmd: "binlog inject test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:14627 alter table handle_error.tb1 drop primary key; alter table handle_error.tb1 add unique (c);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog list test"
dmctl test cmd: "binlog skip test"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:40 CST 2024] <<<<<< finish DM-LIST_ERROR  >>>>>>
[Sun Apr 28 11:10:40 CST 2024] <<<<<< start DM-SKIP_ERROR  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:43 CST 2024] <<<<<< finish DM-SKIP_ERROR  >>>>>>
[Sun Apr 28 11:10:43 CST 2024] <<<<<< start DM-REPLACE_ERROR  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column new_col text, add column c int; alter table handle_error.tb1 add unique(c);"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:47 CST 2024] <<<<<< finish DM-REPLACE_ERROR  >>>>>>
[Sun Apr 28 11:10:47 CST 2024] <<<<<< start DM-CROSS_DDL_SHARDING pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=4 expected=4
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:51 CST 2024] <<<<<< finish DM-CROSS_DDL_SHARDING pessimistic >>>>>>
[Sun Apr 28 11:10:51 CST 2024] <<<<<< start DM-CROSS_DDL_SHARDING optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=4 expected=4
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:53 CST 2024] <<<<<< finish DM-CROSS_DDL_SHARDING optimistic >>>>>>
[Sun Apr 28 11:10:53 CST 2024] <<<<<< start DM-REPLACE_ERROR_MULTIPLE pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test "alter table handle_error.tb1 add column a varchar(20); alter table handle_error.tb1 add unique idx(a); alter table handle_error.tb1 add column b varchar(20); alter table handle_error.tb1 add unique idx2(b);""
got=3 expected=3
dmctl test cmd: "query-status test"
got=4 expected=4
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:10:57 CST 2024] <<<<<< finish DM-REPLACE_ERROR_MULTIPLE pessimistic >>>>>>
[Sun Apr 28 11:10:57 CST 2024] <<<<<< start DM-REPLACE_ERROR_MULTIPLE optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test "alter table handle_error.tb1 add column a varchar(20); alter table handle_error.tb1 add unique idx(a); alter table handle_error.tb1 add column b varchar(20); alter table handle_error.tb1 add unique idx2(b);""
got=3 expected=3
dmctl test cmd: "query-status test"
got=4 expected=4
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:01 CST 2024] <<<<<< finish DM-REPLACE_ERROR_MULTIPLE optimistic >>>>>>
[Sun Apr 28 11:11:01 CST 2024] <<<<<< start DM-EXEC_ERROR_SKIP pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=0 expected=1
command: query-status test Error 1062 (23000): Duplicate  count: 0 != expected: 1, failed the 0-th time, will retry again
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:05 CST 2024] <<<<<< finish DM-EXEC_ERROR_SKIP pessimistic >>>>>>
[Sun Apr 28 11:11:05 CST 2024] <<<<<< start DM-EXEC_ERROR_SKIP optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=0 expected=1
command: query-status test Error 1062 (23000): Duplicate  count: 0 != expected: 1, failed the 0-th time, will retry again
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:09 CST 2024] <<<<<< finish DM-EXEC_ERROR_SKIP optimistic >>>>>>
[Sun Apr 28 11:11:09 CST 2024] <<<<<< start DM-SKIP_INCOMPATIBLE_DDL  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=2 expected=2
/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/run.sh: line 556: hello: command not found
dmctl test cmd: "query-status test"
got=2 expected=2
got=1 expected=1
dmctl test cmd: "binlog skip test"
check diff successfully
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:11 CST 2024] <<<<<< finish DM-SKIP_INCOMPATIBLE_DDL  >>>>>>
[Sun Apr 28 11:11:11 CST 2024] <<<<<< start DM-4202  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:35724"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:13 CST 2024] <<<<<< finish DM-4202  >>>>>>
[Sun Apr 28 11:11:14 CST 2024] <<<<<< start DM-4204  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:19 CST 2024] <<<<<< finish DM-4204  >>>>>>
[Sun Apr 28 11:11:20 CST 2024] <<<<<< start DM-4206 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:39469 -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:39276 -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:36408 -s mysql-replica-02"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:36627 -s mysql-replica-02"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:23 CST 2024] <<<<<< finish DM-4206 pessimistic >>>>>>
[Sun Apr 28 11:11:23 CST 2024] <<<<<< start DM-4206 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:41438 -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:41245 -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:38566 -s mysql-replica-02"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:38785 -s mysql-replica-02"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:27 CST 2024] <<<<<< finish DM-4206 optimistic >>>>>>
[Sun Apr 28 11:11:28 CST 2024] <<<<<< start DM-4207 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:43214"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:40724"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:31 CST 2024] <<<<<< finish DM-4207 pessimistic >>>>>>
[Sun Apr 28 11:11:31 CST 2024] <<<<<< start DM-4207 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:44990"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:42663"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:34 CST 2024] <<<<<< finish DM-4207 optimistic >>>>>>
[Sun Apr 28 11:11:34 CST 2024] <<<<<< start DM-4207 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:47024"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:44602"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:37 CST 2024] <<<<<< finish DM-4207 pessimistic >>>>>>
[Sun Apr 28 11:11:37 CST 2024] <<<<<< start DM-4207 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:49058"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:46541"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:39 CST 2024] <<<<<< finish DM-4207 optimistic >>>>>>
[Sun Apr 28 11:11:40 CST 2024] <<<<<< start DM-4209 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -s mysql-replica-02"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:44 CST 2024] <<<<<< finish DM-4209 pessimistic >>>>>>
[Sun Apr 28 11:11:44 CST 2024] <<<<<< start DM-4209 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -s mysql-replica-02"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:47 CST 2024] <<<<<< finish DM-4209 optimistic >>>>>>
[Sun Apr 28 11:11:48 CST 2024] <<<<<< start DM-4211  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test --binlog-pos dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:54386 alter table handle_error.tb1 add column c int;"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:50 CST 2024] <<<<<< finish DM-4211  >>>>>>
[Sun Apr 28 11:11:51 CST 2024] <<<<<< start DM-4213  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c int;alter table handle_error.tb1 add unique(c);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:11:53 CST 2024] <<<<<< finish DM-4213  >>>>>>
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
wait process dm-master.test exit...
process dm-master.test already exit
wait process dm-worker.test exit...
wait process dm-worker.test exit...
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:11:58 CST 2024] <<<<<< test case handle_error success! >>>>>>
start running case: [handle_error_2] script: [/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/run.sh]
Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/run.sh...
Verbose mode = false
0 dm-master alive
0 dm-worker alive
0 dm-syncer alive
process dm-master.test already exit
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:11:58 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
rpc addr 127.0.0.1:8261 is alive
[Sun Apr 28 11:11:59 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error_2/source1.yaml"
[Sun Apr 28 11:12:01 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error_2/source2.yaml"
[Sun Apr 28 11:12:03 CST 2024] <<<<<< start DM-4215 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "unlock-ddl-lock test-`handle_error`.`tb` --force-remove"
dmctl test cmd: "binlog skip test -s mysql-replica-01"
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "unlock-ddl-lock test-`handle_error`.`tb` --force-remove"
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:30 CST 2024] <<<<<< finish DM-4215 pessimistic >>>>>>
[Sun Apr 28 11:12:30 CST 2024] <<<<<< start DM-4215 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:34 CST 2024] <<<<<< finish DM-4215 optimistic >>>>>>
[Sun Apr 28 11:12:35 CST 2024] <<<<<< start DM-4216 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:5043 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:5548 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:38 CST 2024] <<<<<< finish DM-4216 pessimistic >>>>>>
[Sun Apr 28 11:12:38 CST 2024] <<<<<< start DM-4216 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:6850 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:7521 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:40 CST 2024] <<<<<< finish DM-4216 optimistic >>>>>>
[Sun Apr 28 11:12:40 CST 2024] <<<<<< start DM-4216 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:8915 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:9494 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:43 CST 2024] <<<<<< finish DM-4216 pessimistic >>>>>>
[Sun Apr 28 11:12:43 CST 2024] <<<<<< start DM-4216 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:10980 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:11467 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:45 CST 2024] <<<<<< finish DM-4216 optimistic >>>>>>
[Sun Apr 28 11:12:46 CST 2024] <<<<<< start DM-4219 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -s mysql-replica-01 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -s mysql-replica-02 alter table handle_error.tb1 add column c varchar(20);"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:51 CST 2024] <<<<<< finish DM-4219 pessimistic >>>>>>
[Sun Apr 28 11:12:51 CST 2024] <<<<<< start DM-4219 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -s mysql-replica-01 alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -s mysql-replica-02 alter table handle_error.tb1 add column c varchar(20);"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:12:55 CST 2024] <<<<<< finish DM-4219 optimistic >>>>>>
[Sun Apr 28 11:12:56 CST 2024] <<<<<< start DM-4220 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:16594"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:17605"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:16787"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:17824"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:16787"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:17824"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:16787"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:17824"
dmctl test cmd: "binlog revert test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:17824"
dmctl test cmd: "binlog revert test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:02 CST 2024] <<<<<< finish DM-4220 pessimistic >>>>>>
[Sun Apr 28 11:13:02 CST 2024] <<<<<< start DM-4220 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:19218"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:20496"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:19411"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:20715"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:19411"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:20715"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:19411"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:20715"
dmctl test cmd: "binlog revert test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:20715"
dmctl test cmd: "binlog revert test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:09 CST 2024] <<<<<< finish DM-4220 optimistic >>>>>>
[Sun Apr 28 11:13:10 CST 2024] <<<<<< start DM-4185 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:21842"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:23387"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:13 CST 2024] <<<<<< finish DM-4185 pessimistic >>>>>>
[Sun Apr 28 11:13:13 CST 2024] <<<<<< start DM-4185 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:23811"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:25545"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:18 CST 2024] <<<<<< finish DM-4185 optimistic >>>>>>
[Sun Apr 28 11:13:18 CST 2024] <<<<<< start DM-4185 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:26038"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:27703"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:21 CST 2024] <<<<<< finish DM-4185 pessimistic >>>>>>
[Sun Apr 28 11:13:21 CST 2024] <<<<<< start DM-4185 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:28265"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:29861"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:24 CST 2024] <<<<<< finish DM-4185 optimistic >>>>>>
[Sun Apr 28 11:13:25 CST 2024] <<<<<< start DM-4201  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:30234"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:27 CST 2024] <<<<<< finish DM-4201  >>>>>>
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
wait process dm-master.test exit...
process dm-master.test already exit
wait process dm-worker.test exit...
wait process dm-worker.test exit...
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:13:32 CST 2024] <<<<<< test case handle_error_2 success! >>>>>>
start running case: [handle_error_3] script: [/home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/run.sh]
Running test /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/run.sh...
Verbose mode = false
0 dm-master alive
0 dm-worker alive
0 dm-syncer alive
process dm-master.test already exit
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:13:32 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
wait for rpc addr 127.0.0.1:8261 alive the 2-th time
rpc addr 127.0.0.1:8261 is alive
[Sun Apr 28 11:13:34 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error_3/source1.yaml"
[Sun Apr 28 11:13:36 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/handle_error_3/source2.yaml"
[Sun Apr 28 11:13:38 CST 2024] <<<<<< start DM-4189 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:1559 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:1735 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:41 CST 2024] <<<<<< finish DM-4189 pessimistic >>>>>>
[Sun Apr 28 11:13:41 CST 2024] <<<<<< start DM-4189 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:3856 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:4247 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:44 CST 2024] <<<<<< finish DM-4189 optimistic >>>>>>
[Sun Apr 28 11:13:44 CST 2024] <<<<<< start DM-4189 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:6411 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:6759 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:47 CST 2024] <<<<<< finish DM-4189 pessimistic >>>>>>
[Sun Apr 28 11:13:47 CST 2024] <<<<<< start DM-4189 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:8966 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:9271 alter table handle_error.tb1 add column d varchar(20);"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c varchar(20);"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:50 CST 2024] <<<<<< finish DM-4189 optimistic >>>>>>
[Sun Apr 28 11:13:51 CST 2024] <<<<<< start DM-4210  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:11239 alter table handle_error.tb1 add column d int;alter table handle_error.tb1 add unique(d);"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column e int unique;"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c int;alter table handle_error.tb1 add unique(c);"
dmctl test cmd: "query-status test"
got=2 expected=2
run tidb sql failed 1-th time, retry later
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:13:56 CST 2024] <<<<<< finish DM-4210  >>>>>>
[Sun Apr 28 11:13:57 CST 2024] <<<<<< start DM-4193 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:13027"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:12152"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:13027"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:12152"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:13413"
dmctl test cmd: "binlog revert test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:13413"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:13413"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:12590"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:14:01 CST 2024] <<<<<< finish DM-4193 pessimistic >>>>>>
[Sun Apr 28 11:14:01 CST 2024] <<<<<< start DM-4193 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:15189"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:14529"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:15189"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:14529"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:15575"
dmctl test cmd: "binlog revert test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:15575"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:15575"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:14967"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
got=3 expected=3
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:14:05 CST 2024] <<<<<< finish DM-4193 optimistic >>>>>>
[Sun Apr 28 11:14:06 CST 2024] <<<<<< start DM-4230  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column d int unique;"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog revert test"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c int;"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:14:09 CST 2024] <<<<<< finish DM-4230  >>>>>>
[Sun Apr 28 11:14:10 CST 2024] <<<<<< start DM-4177  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip wrong-task"
dmctl test cmd: "binlog skip"
dmctl test cmd: "binlog skip test -b dm-it-77aa782e-7c6f-4914-b8ab-6261a84f4efb-ql0cr-znn5q-bin|000001.000001:19397 -s wrong-source"
dmctl test cmd: "binlog skip test -b mysql-bin|1111 -s wrong-source"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "binlog skip test"
dmctl test cmd: "binlog replace test "alter table handle_error.tb1 add column c int;""
dmctl test cmd: "binlog skip test -b mysql-bin.000000:00000"
dmctl test cmd: "binlog aaa test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:14:13 CST 2024] <<<<<< finish DM-4177  >>>>>>
dmctl test cmd: "binlog skip test"
dmctl test cmd: "binlog replace test alter table handle_error.tb1 add column c int"
dmctl test cmd: "binlog revert test"
[Sun Apr 28 11:14:14 CST 2024] <<<<<< start DM-4231  >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/release-7.1/pull_dm_integration_test/tiflow/dm/tests/handle_error_3/conf/single-source-no-sharding.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test "alter table handle_error.tb1 add column c int; alter table handle_error.tb1 add column d int unique;""
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
dmctl test cmd: "stop-task test"
[Sun Apr 28 11:14:17 CST 2024] <<<<<< finish DM-4231  >>>>>>
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
wait process dm-master.test exit...
process dm-master.test already exit
wait process dm-worker.test exit...
wait process dm-worker.test exit...
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
[Sun Apr 28 11:14:22 CST 2024] <<<<<< test case handle_error_3 success! >>>>>>
[Pipeline] }
Cache not saved (ws/jenkins-pingcap-tiflow-release-7.1-pull_dm_integration_test-192/tiflow-dm already exists)
[Pipeline] // cache
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
ERROR: script returned exit code 2
Finished: FAILURE