Skipping 485 KB..
Full LogTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTestTest
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-task-standalone.yaml --remove-meta"
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/shardddl1/source2.yaml"
rpc addr 127.0.0.1:8261 is alive
dmctl test cmd: "stop-task test"
dmctl test cmd: "query-status test2"
dmctl test cmd: "stop-task test2"
rpc addr 127.0.0.1:8361 is alive
rpc addr 127.0.0.1:8461 is alive
start worker and operate mysql config to worker
[Sat May 4 14:58:56 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/ha_cases_1/conf/dm-worker1.toml >>>>>>
[Sat May 4 14:58:57 CST 2024] <<<<<< finish DM-4219 pessimistic >>>>>>
[Sat May 4 14:58:57 CST 2024] <<<<<< start DM-4219 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-task.yaml --remove-meta"
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "config source mysql-replica-01"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "validation status test"
got=1 expected=1
got=0 expected=1
command: validation status test pendingRowsStatus": "insert\/update\/delete: 0\/0\/0 count: 0 != expected: 1, failed the 0-th time, will retry again
start prepare_data
dmctl test cmd: "query-status -s mysql-replica-01"
dmctl test cmd: "query-status -s mysql-replica-01"
dmctl test cmd: "query-status"
check diff successfully
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
reset go failpoints, and need restart dm-worker
then resume task, task will recover success
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/ha_cases_1/source1.yaml"
dmctl test cmd: "query-status test"
check diff successfully
dmctl test cmd: "validation status test"
got=0 expected=1
command: validation status test "processedRowsStatus": "insert\/update\/delete: 4\/1\/1" count: 0 != expected: 1, failed the 0-th time, will retry again
wait process dm-master.test exit...
got=2 expected=2
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog replace test -s mysql-replica-01 alter table handle_error.tb1 add column c varchar(20);"
wait process dm-worker.test exit...
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog replace test -s mysql-replica-02 alter table handle_error.tb1 add column c varchar(20);"
wait process dm-worker exit...
dmctl test cmd: "stop-task test"
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/ha_cases_1/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
[Sat May 4 14:58:59 CST 2024] <<<<<< finish DM-4219 optimistic >>>>>>
wait process dm-master.test exit...
process dm-master.test already exit
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
wait process dm-worker.test exit...
wait process dm-worker exit...
process dm-worker.test already exit
process dm-worker already exit
--> full mode, check we validate different data types(gtid=false, relay=true)
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl1/conf/dm-worker1.toml >>>>>>
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/relay_interrupt/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl1/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "start-task /tmp/dm_test/start_task/dm-task.yaml --start-time '2024-05-04 08:58:57'"
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /tmp/dm_test/tls/dm-master1.toml >>>>>>
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-MASTER on port 8361, config: /tmp/dm_test/tls/dm-master2.toml >>>>>>
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-MASTER on port 8461, config: /tmp/dm_test/tls/dm-master3.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/ha_cases_1/source2.yaml"
[Sat May 4 14:59:00 CST 2024] <<<<<< start DM-4220 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "list-member"
[Sat May 4 14:58:59 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
wait process dm-worker.test exit...
wait process dm-master.test exit...
rpc addr 127.0.0.1:8262 is alive
rpc addr 127.0.0.1:8262 is alive
rpc addr 127.0.0.1:8263 is alive
[Sat May 4 14:59:00 CST 2024] <<<<<< start DM-COMPACT >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl1/conf/single-source-no-sharding.yaml --remove-meta"
got=1 expected=1
got=1 expected=1
got=1 expected=2
command: validation status test pendingRowsStatus": "insert\/update\/delete: 0\/0\/0 count: 1 != expected: 2, failed the 1-th time, will retry again
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:01 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker2.toml >>>>>>
rpc addr 127.0.0.1:8261 is alive
rpc addr 127.0.0.1:8361 is alive
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
rpc addr 127.0.0.1:8461 is alive
[Sat May 4 14:59:01 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /tmp/dm_test/tls/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
wait process dm-master.test exit...
process dm-master.test already exit
dmctl test cmd: "stop-task test"
dmctl test cmd: "stop-relay -s mysql-replica-01"
dmctl test cmd: "query-status -s mysql-replica-01"
[Sat May 4 14:59:01 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
start DM task
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/ha_cases_1/conf/dm-task.yaml "
dmctl test cmd: "query-status test"
got=2 expected=2
got=2 expected=2
wait process dm-worker.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
check diff successfully
dmctl test cmd: "stop-task test"
dmctl test cmd: "query-status test"
got=1 expected=1
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/incremental_mode/source1.yaml"
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/tls/source1.yaml"
wait process dm-worker.test exit...
[Sat May 4 14:59:02 CST 2024] <<<<<< finish DM-071 pessimistic >>>>>>
dmctl test cmd: "operate-source create /tmp/dm_test/incremental_mode/source2.yaml"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:16594"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:17605"
dmctl test cmd: "query-status test"
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "transfer-source mysql-replica-01 worker2"
dmctl test cmd: "start-task /tmp/dm_test/start_task/dm-task.yaml --start-time '2024-05-04 08:58:57'"
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
dmctl test cmd: "query-status test"
[Sat May 4 14:59:02 CST 2024] <<<<<< test case import_v10x success! >>>>>>
start running case: [sharding2] script: [/home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/run.sh]
Running test /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/run.sh...
Verbose mode = false
0 dm-master alive
0 dm-worker alive
0 dm-syncer alive
process dm-master.test already exit
process dm-worker.test already exit
process dm-syncer.test already exit
[Sat May 4 14:59:02 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
got=1 expected=1
got=1 expected=1
got=2 expected=2
got=1 expected=1
got=1 expected=1
got=4 expected=4
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:16787"
got=2 expected=2
got=2 expected=2
use sync_diff_inspector to check full dump loader
check diff successfully
flush logs to force rotate binlog file
apply increment data before restart dm-worker to ensure entering increment phase
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:17824"
dmctl test cmd: "query-status test"
[Sat May 4 14:59:03 CST 2024] <<<<<< start DM-073 pessimistic >>>>>>
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:16787"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:17824"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/double-source-pessimistic.yaml --remove-meta"
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:03 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
check master alive
dmctl test cmd: "list-member"
wait process dm-worker.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
start task and check stage
dmctl test cmd: "start-task /tmp/dm_test/tls/dm-task.yaml --remove-meta=true"
dmctl test cmd: "start-relay -s mysql-replica-02 worker1"
dmctl test cmd: "query-status test"
dmctl test cmd: "stop-task test"
run tidb sql failed 1-th time, retry later
dmctl test cmd: "start-task /tmp/dm_test/start_task/dm-task.yaml --start-time '1995-03-07 01:02:03'"
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
[Sat May 4 14:59:04 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
dmctl test cmd: "validation show-error test"
got=0 expected=1
command: validation show-error test "id": "1" count: 0 != expected: 1, failed the 0-th time, will retry again
got=2 expected=2
dmctl test cmd: "start-relay -s mysql-replica-01 worker2"
dmctl test cmd: "query-status test"
got=2 expected=2
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/sharding2/source1.yaml"
[Sat May 4 14:59:04 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
check diff successfully
dmctl test cmd: "stop-task test"
[Sat May 4 14:59:05 CST 2024] <<<<<< finish DM-073 pessimistic >>>>>>
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:05 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "stop-task test"
dmctl test cmd: "start-task /tmp/dm_test/start_task/dm-task.yaml --start-time '2037-12-12 01:02:03'"
dmctl test cmd: "query-status test"
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:16787"
test http and api interface
Release Version: v8.2.0-alpha-68-g913f7afee
Git Commit Hash: 913f7afee1bacf1d9de4f5f9a1281caed6806640
Git Branch: HEAD
UTC Build Time: 2024-05-04 06:43:36
Go Version: go version go1.21.6 linux/amd64
Failpoint Build: true
rpc addr 127.0.0.1:8261 is alive
got=2 expected=2
use sync_diff_inspector to check increment data
check diff successfully
[Sat May 4 14:59:06 CST 2024] <<<<<< finish test_running >>>>>>
kill dm-master1
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:17824"
dmctl test cmd: "binlog revert test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:17824"
{
"result": true,
"msg": "",
"members": [
{
"leader": {
"msg": "",
"name": "master2",
"addr": "127.0.0.1:8361"
}
},
{
"master": {
"msg": "",
"masters": [
{
"name": "master1",
"memberID": "10068700568285020701",
"alive": true,
"peerURLs": [
"https://127.0.0.1:8291"
],
"clientURLs": [
"https://127.0.0.1:8261"
]
},
{
"name": "master2",
"memberID": "518295880577331676",
"alive": true,
"peerURLs": [
"https://127.0.0.1:8292"
],
"clientURLs": [
"https://127.0.0.1:8361"
]
},
{
"name": "master3",
"memberID": "6800138986734450252",
"alive": true,
"peerURLs": [
"https://127.0.0.1:8293"
],
"clientURLs": [
"https://127.0.0.1:8461"
]
}
]
}
},
{
"worker": {
"msg": "",
"workers": [
{
"name": "worker1",
"addr": "127.0.0.1:8262",
"stage": "bound",
"source": "mysql-replica-01"
}
]
}
}
]
}
rpc addr 127.0.0.1:8261 is alive
use common name not in 'cert-allowed-cn' should not request success
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/sharding2/source2.yaml"
dmctl test cmd: "binlog revert test"
dmctl test cmd: "stop-task test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
dmctl test cmd: "validation ignore-error test --all"
[Sat May 4 14:59:06 CST 2024] <<<<<< start DM-076 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-pessimistic.yaml --remove-meta"
[Sat May 4 14:59:06 CST 2024] <<<<<< finish DM-4220 pessimistic >>>>>>
[Sat May 4 14:59:06 CST 2024] <<<<<< start DM-4220 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/source1.yaml"
[Sat May 4 14:59:06 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
dmctl test cmd: "validation show-error test"
got=0 expected=0
dmctl test cmd: "validation status test"
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "stop-task test"
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
got=1 expected=1
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
dmctl test cmd: "start-relay -s mysql-replica-02 worker2"
check diff failed 1-th time, retry later
wait process dm-master1 exit...
dmctl test cmd: "query-status test"
wait for rpc addr 127.0.0.1:8261 alive the 2-th time
got=1 expected=1
check diff successfully
dmctl test cmd: "stop-task test"
wait process dm-master.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-task-standalone.yaml --remove-meta"
wait process dm-master.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
[Sat May 4 14:59:08 CST 2024] <<<<<< finish DM-076 pessimistic >>>>>>
[Sat May 4 14:59:08 CST 2024] <<<<<< start DM-076 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-optimistic.yaml --remove-meta"
wait process dm-master1 exit...
process dm-master1 already exit
waiting 5 seconds
wait for rpc addr 127.0.0.1:8261 alive the 3-th time
got=2 expected=2
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-task.yaml "
start task after restarted dm-worker
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/relay_interrupt/conf/dm-task.yaml"
dmctl test cmd: "query-status test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:19218"
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:20496"
got=1 expected=1
check diff successfully
read binlog from relay log failed, and will use remote binlog
dmctl test cmd: "config source mysql-replica-01"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test"
dmctl test cmd: "binlog skip test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:19411"
wait process dm-master.test exit...
process dm-master.test already exit
check diff failed 2-th time, retry later
got=1 expected=1
got=1 expected=1
dmctl test cmd: "validation status test"
got=1 expected=1
got=0 expected=1
command: validation status test pendingRowsStatus": "insert\/update\/delete: 0\/0\/0 count: 0 != expected: 1, failed the 0-th time, will retry again
wait process dm-master.test exit...
process dm-master.test already exit
dmctl test cmd: "binlog skip test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:20715"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:19411"
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:20715"
dmctl test cmd: "query-status test"
got=1 expected=1
check diff failed 1-th time, retry later
check sync diff for the increment replication
check diff successfully
dmctl test cmd: "stop-task test"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-task.yaml "
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
wait process dm-worker.test exit...
wait process dm-worker.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test"
wait process dm-worker.test exit...
run tidb sql failed 1-th time, retry later
wait for rpc addr 127.0.0.1:8261 alive the 4-th time
wait process dm-worker.test exit...
wait process dm-worker.test exit...
check sync diff for the increment replication
check diff failed 1-th time, retry later
wait process dm-worker.test exit...
process dm-worker.test already exit
[Sat May 4 14:59:11 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/relay_interrupt/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "list-member"
[Sat May 4 14:59:11 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
check diff failed 3-th time, retry later
got=1 expected=1
got=1 expected=1
got=1 expected=1
--> full mode, check we validate different data types(gtid=true, relay=false)
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
[Sat May 4 14:59:12 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
check diff successfully
dmctl test cmd: "stop-task test"
[Sat May 4 14:59:12 CST 2024] <<<<<< finish DM-076 optimistic >>>>>>
rpc addr 127.0.0.1:8262 is alive
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:12 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
dmctl test cmd: "binlog revert test -s mysql-replica-01 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:19411"
wait for rpc addr 127.0.0.1:8261 alive the 5-th time
dmctl test cmd: "binlog revert test -s mysql-replica-02 -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:20715"
dmctl test cmd: "binlog revert test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:20715"
wait process dm-master.test exit...
dmctl test cmd: "binlog revert test"
dmctl test cmd: "stop-task test"
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:13 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
[Sat May 4 14:59:13 CST 2024] <<<<<< finish DM-4220 optimistic >>>>>>
[Sat May 4 14:59:13 CST 2024] <<<<<< start DM-077 >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-pessimistic.yaml --remove-meta"
rpc addr 127.0.0.1:8263 is alive
wait for rpc addr 127.0.0.1:8261 alive the 6-th time
check task is running
HTTP 127.0.0.1:8361/apis/v1alpha1/status/test is alive
check master2,3 are running
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
process dm-master.test already exit
check diff failed 4-th time, retry later
check diff successfully
kill dm-worker1
[Sat May 4 14:59:13 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
got=2 expected=2
wait process dm-worker.test exit...
wait for rpc addr 127.0.0.1:8261 alive the 7-th time
wait process dm-worker1 exit...
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:14 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
[Sat May 4 14:59:14 CST 2024] <<<<<< start DM-4185 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
wait process dm-worker.test exit...
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "stop-task test"
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:14 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
[Sat May 4 14:59:15 CST 2024] <<<<<< finish DM-077 >>>>>>
[Sat May 4 14:59:15 CST 2024] <<<<<< start DM-077 >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 8-th time
finish prepare_data
[Sat May 4 14:59:13 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/many_tables/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:14 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/many_tables/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-optimistic.yaml --remove-meta"
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "query-status test"
dmctl test cmd: "operate-source create /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/source1.yaml"
wait process dm-worker.test exit...
process dm-worker.test already exit
dmctl test cmd: "operate-source update /tmp/dm_test/incremental_mode/source1.yaml"
dmctl test cmd: "operate-source update /tmp/dm_test/incremental_mode/source2.yaml"
dmctl test cmd: "stop-relay -s mysql-replica-02 worker1"
wait process dm-worker.test exit...
check diff failed 5-th time, retry later
dmctl test cmd: "operate-source create /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/source2.yaml"
got=2 expected=2
dmctl test cmd: "query-status test"
wait process dm-worker1 exit...
process dm-worker1 already exit
kill dm-worker2
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/start_task/source1.yaml"
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /tmp/dm_test/many_tables/source1.yaml"
got=2 expected=2
dmctl test cmd: "stop-relay -s mysql-replica-01 worker2"
got=2 expected=2
dmctl test cmd: "operate-source stop /tmp/dm_test/incremental_mode/source1.yaml"
use sync_diff_inspector to check increment2 data now!
check diff successfully
[Sat May 4 14:59:15 CST 2024] <<<<<< finish test_kill_master >>>>>>
[Sat May 4 14:59:15 CST 2024] <<<<<< start test_kill_and_isolate_worker >>>>>>
[Sat May 4 14:59:15 CST 2024] <<<<<< start test_running >>>>>>
got=2 expected=2
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/many_tables/conf/dm-task.yaml "
dmctl test cmd: "operate-source stop /tmp/dm_test/incremental_mode/source2.yaml"
dmctl test cmd: "operate-source create /tmp/dm_test/incremental_mode/source1.yaml"
2 dm-master alive
2 dm-worker alive
0 dm-syncer alive
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:21842"
dmctl test cmd: "query-status test"
dmctl test cmd: "operate-source create /tmp/dm_test/incremental_mode/source2.yaml"
dmctl test cmd: "start-relay -s mysql-replica-02 worker1"
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:23387"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "start-relay -s mysql-replica-01 worker2"
got=2 expected=2
start task in incremental mode with zero gtid/pos
[Sat May 4 14:59:16 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
wait for rpc addr 127.0.0.1:8261 alive the 9-th time
wait process dm-worker2 exit...
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-task-no-validator.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "stop-task test"
[Sat May 4 14:59:16 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
got=3 expected=3
dmctl test cmd: "stop-task test"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/dm-task.yaml --remove-meta"
[Sat May 4 14:59:17 CST 2024] <<<<<< finish DM-077 >>>>>>
[Sat May 4 14:59:17 CST 2024] <<<<<< finish DM-4185 pessimistic >>>>>>
[Sat May 4 14:59:17 CST 2024] <<<<<< start DM-4185 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
wait for rpc addr 127.0.0.1:8261 alive the 10-th time
check diff failed 6-th time, retry later
wait process dm-worker2 exit...
process dm-worker2 already exit
[Sat May 4 14:59:17 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:17 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
dmctl test cmd: "query-status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:17 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
wait process dm-master.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
--> (fail) validation start: missing mode value
dmctl test cmd: "validation start --mode"
[Sat May 4 14:59:18 CST 2024] <<<<<< start DM-078 >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-pessimistic.yaml --remove-meta"
--> (fail) validation start: invalid mode
dmctl test cmd: "validation start --mode xxx"
--> (fail) validation start: missing start-time value
dmctl test cmd: "validation start --start-time"
wait for rpc addr 127.0.0.1:8261 alive the 11-th time
check diff failed 1-th time, retry later
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:18 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker2.toml >>>>>>
--> (fail) validation start: invalid start-time
dmctl test cmd: "validation start --start-time xx"
--> (fail) validation start: without all-task and task-name
dmctl test cmd: "validation start --mode full"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
--> (fail) validation start: with both all-task and task-name
dmctl test cmd: "validation start --all-task test"
--> (fail) validation start: too many arguments
dmctl test cmd: "validation start test test2"
--> (fail) validation start: non-existed subtask
dmctl test cmd: "validation start xxxxx"
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "start-task /tmp/dm_test/incremental_mode/dm-task.yaml --remove-meta"
got=2 expected=2
rpc addr 127.0.0.1:8262 is alive
dmctl test cmd: "operate-source create /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/source1.yaml"
--> (fail) validation start: non-exist source
dmctl test cmd: "validation start -s xxxxx --all-task"
--> (fail) validation stop: without all-task and task-name
dmctl test cmd: "validation stop"
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:23811"
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
--> (fail) validation stop: with both all-task and task-name
dmctl test cmd: "validation stop --all-task test"
--> (fail) validation stop: too many arguments
dmctl test cmd: "validation stop test test2"
wait for rpc addr 127.0.0.1:8261 alive the 12-th time
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:25545"
dmctl test cmd: "query-status test"
dmctl test cmd: "query-status test"
got=1 expected=1
--> (fail) validation stop: non-existed subtask
dmctl test cmd: "validation stop xxxxx"
--> (fail) validation stop: non-exist source
dmctl test cmd: "validation stop -s xxxxx --all-task"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
rpc addr 127.0.0.1:8263 is alive
wait for port 8262 offline the 1-th time
check diff failed 1-th time, retry later
--> (fail) validation stop: stop not-enabled validator
dmctl test cmd: "validation stop -s mysql-replica-01 test"
--> (success) validation start: start validation without explicit mode for source 1
dmctl test cmd: "validation start -s mysql-replica-01 test"
got=3 expected=3
dmctl test cmd: "stop-task test"
check diff failed 7-th time, retry later
--> (fail) validation start: start all validator with explicit mode, but 1 subtask already enabled
dmctl test cmd: "validation start --mode full test"
--> (fail) validation start: start validation with explicit mode for source 1 again
dmctl test cmd: "validation start -s mysql-replica-01 --mode full test"
[Sat May 4 14:59:20 CST 2024] <<<<<< finish DM-4185 optimistic >>>>>>
[Sat May 4 14:59:20 CST 2024] <<<<<< start DM-4185 pessimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-pessimistic.yaml --remove-meta"
dmctl test cmd: "query-status test"
got=2 expected=2
wait process dm-master.test exit...
--> (fail) validation start: start all validator without explicit mode
dmctl test cmd: "validation start test"
--> (fail) validation stop: stop all but 1 subtask is not enabled
dmctl test cmd: "validation stop test"
--> (success) validation start: start validation with fast mode and start-time for source 2
dmctl test cmd: "validation start -s mysql-replica-02 --mode fast --start-time '2020-01-01 00:01:00' test"
dmctl test cmd: "query-status test"
check diff successfully
1 dm-master alive
1 dm-worker alive
0 dm-syncer alive
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=2 expected=2
got=0 expected=0
--> (success) validation start: start all validator of the task without explicit param again, i.e. resuming
dmctl test cmd: "validation start test"
wait for rpc addr 127.0.0.1:8261 alive the 13-th time
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=2 expected=2
got=0 expected=0
--> (success) validation stop: stop validation of source 1
dmctl test cmd: "validation stop -s mysql-replica-01 test"
[Sat May 4 14:59:20 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
check diff successfully
wait for port 8262 offline the 2-th time
wait process dm-worker.test exit...
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=1 expected=1
--> (success) validation stop: stop all of test
dmctl test cmd: "validation stop test"
wait process dm-master.test exit...
check diff successfully
dmctl test cmd: "stop-task test"
wait process dm-master.test exit...
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=0 expected=0
got=2 expected=2
--> (success) validation stop: stop all of test again
dmctl test cmd: "validation stop test"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/conf/dm-task.yaml "
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
wait for rpc addr 127.0.0.1:8261 alive the 14-th time
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=0 expected=0
got=2 expected=2
--> (success) validation start: start all of test
dmctl test cmd: "validation start test"
got=2 expected=2
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/conf/dm-task-standalone.yaml --remove-meta"
check diff failed 8-th time, retry later
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=2 expected=2
got=0 expected=0
--> (success) validation stop: multiple source
dmctl test cmd: "validation stop -s mysql-replica-01 -s mysql-replica-02 test"
[Sat May 4 14:59:21 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
wait process dm-worker.test exit...
process dm-worker.test already exit
wait process dm-master.test exit...
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:26038"
dmctl test cmd: "query-status test"
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=0 expected=0
got=2 expected=2
--> (success) validation start: multiple source
dmctl test cmd: "validation start -s mysql-replica-01 -s mysql-replica-02 test"
[Sat May 4 14:59:22 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
check diff successfully
dmctl test cmd: "stop-task test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:27703"
dmctl test cmd: "query-status test"
dmctl test cmd: "validation status test"
got=1 expected=1
got=1 expected=1
got=1 expected=1
got=2 expected=2
got=0 expected=0
dmctl test cmd: "stop-task test"
wait for rpc addr 127.0.0.1:8261 alive the 15-th time
[Sat May 4 14:59:22 CST 2024] <<<<<< finish DM-078 >>>>>>
[Sat May 4 14:59:22 CST 2024] <<<<<< start DM-078 >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/shardddl3/conf/single-source-optimistic.yaml --remove-meta"
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-task-no-validator.yaml --remove-meta"
wait process dm-master.test exit...
process dm-master.test already exit
got=3 expected=3
dmctl test cmd: "stop-task test"
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:22 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/sharding2/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
wait process dm-master.test exit...
[Sat May 4 14:59:23 CST 2024] <<<<<< finish DM-4185 pessimistic >>>>>>
[Sat May 4 14:59:23 CST 2024] <<<<<< start DM-4185 optimistic >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/double-source-optimistic.yaml --remove-meta"
0
in Load unit "progress": "0.00 %",
dmctl test cmd: "config source mysql-replica-01"
got=1 expected=1
got=1 expected=1
dmctl test cmd: "validation status test"
got=1 expected=1
got=0 expected=1
command: validation status test pendingRowsStatus": "insert\/update\/delete: 0\/0\/0 count: 0 != expected: 1, failed the 0-th time, will retry again
rpc addr 127.0.0.1:8262 is alive
[Sat May 4 14:59:23 CST 2024] <<<<<< START DM-WORKER on port 8263, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/incremental_mode/conf/dm-worker2.toml >>>>>>
wait for rpc addr 127.0.0.1:8263 alive the 1-th time
wait for rpc addr 127.0.0.1:8261 alive the 16-th time
wait process dm-worker.test exit...
check diff failed 9-th time, retry later
rpc addr 127.0.0.1:8263 is alive
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/dmctl_command/conf/dm-task2-no-validator.yaml --remove-meta"
dmctl test cmd: "stop-relay -s mysql-replica-02 worker2"
got=1 expected=1
check diff failed 1-th time, retry later
check diff successfully
check diff successfully
dmctl test cmd: "start-relay -s mysql-replica-02 worker2"
Aborted by Jenkins Admin
Sending interrupt signal to process
Killing processes
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "query-status test"
wait process dm-worker.test exit...
rpc addr 127.0.0.1:8263 is alive
got=2 expected=2
wait for rpc addr 127.0.0.1:8261 alive the 17-th time
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:28265"
dmctl test cmd: "query-status test"
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:29861"
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
got=2 expected=2
dmctl test cmd: "query-status test2"
got=1 expected=1
--> (fail) validation update: on non-existed validator
dmctl test cmd: "validation update test -s mysql-replica-02 --cutover-binlog-gtid 2726a10e-09e3-11ef-9a2d-36eef573f0f2:1-999"
wait process dm-worker.test exit...
process dm-worker.test already exit
process dm-syncer.test already exit
failpoint=github.com/pingcap/tiflow/dm/pkg/conn/GetSessionVariableFailed=return("sql_mode,1152")
got=2 expected=2
dmctl test cmd: "binlog skip test"
dmctl test cmd: "query-status test"
--> (success) validation start: start all tasks
dmctl test cmd: "validation start --all-task --mode fast"
--> (fail) validation update: missing task-name
dmctl test cmd: "validation update"
check diff successfully
1 dm-master alive
1 dm-worker alive
0 dm-syncer alive
got=1 expected=1
got=1 expected=1
got=1 expected=1
--> full mode, check we validate different data types(gtid=true, relay=true)
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
dmctl test cmd: "query-status -s mysql-replica-02"
got=2 expected=2
check sync diff for the increment replication
check diff successfully
dmctl test cmd: "query-status test"
wait for rpc addr 127.0.0.1:8261 alive the 18-th time
got=3 expected=3
dmctl test cmd: "stop-task test"
--> (fail) validation update: multi tasks
dmctl test cmd: "validation update test test2"
--> (fail) validation update: non-existed subtask
dmctl test cmd: "validation update xxxxx"
got=3 expected=3
got=2 expected=2
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
[Sat May 4 14:59:25 CST 2024] <<<<<< finish DM-4185 optimistic >>>>>>
--> (fail) validation update: non-exist source
dmctl test cmd: "validation update -s xxxxx test"
--> (fail) validation update: not specify cutover-binlog-gtid
dmctl test cmd: "validation update test -s mysql-replica-02"
--> (fail) validation update: not specify cutover-binlog-pos
dmctl test cmd: "validation update test2 -s mysql-replica-01"
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
wait process dm-master.test exit...
wait process dm-master.test exit...
check diff failed 10-th time, retry later
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
[Sat May 4 14:59:26 CST 2024] <<<<<< START DM-MASTER on port 8261, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/relay_interrupt/conf/dm-master.toml >>>>>>
wait for rpc addr 127.0.0.1:8261 alive the 1-th time
--> (fail) validation update: invalid cutover-binlog-pos
dmctl test cmd: "validation update test -s mysql-replica-01 --cutover-binlog-pos xxx"
--> (fail) validation update: specify both cutover-binlog-pos and cutover-binlog-gtid
dmctl test cmd: "validation update test -s mysql-replica-01 --cutover-binlog-pos '(mysql-bin.000001, 2345)' --cutover-binlog-gtid 2726a10e-09e3-11ef-9a2d-36eef573f0f2:1-999"
wait process dm-master.test exit...
--> (success) validation update: on exist source
dmctl test cmd: "validation update test -s mysql-replica-02 --cutover-binlog-gtid 2726a10e-09e3-11ef-9a2d-36eef573f0f2:1-999"
dmctl test cmd: "validation status test"
wait process dm-master.test exit...
got=1 expected=1
got=0 expected=0
got=2 expected=2
got=2 expected=2
got=0 expected=0
got=2 expected=2
got=1 expected=1
dmctl test cmd: "validation status test2"
got=1 expected=1
got=0 expected=0
got=1 expected=1
got=1 expected=1
got=0 expected=0
got=1 expected=1
got=0 expected=0
--> (success) validation stop: stop all tasks
dmctl test cmd: "validation stop --all-task"
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
[Sat May 4 14:59:26 CST 2024] <<<<<< start DM-4201 >>>>>>
dmctl test cmd: "start-task /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/handle_error_2/conf/single-source-no-sharding.yaml --remove-meta"
wait process dm-master.test exit...
dmctl test cmd: "validation status test"
got=1 expected=1
got=0 expected=0
got=2 expected=2
got=0 expected=0
got=2 expected=2
dmctl test cmd: "validation status test2"
restore config
make: *** [dm_integration_test_in_group] Terminated
kill finished with exit code 0
/home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/validator_basic/run.sh: line 53: 2128 Terminated wait_process_exit dm-master.test
restore time_zone
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
Sending interrupt signal to process
Killing processes
script returned exit code 143
wait for rpc addr 127.0.0.1:8261 alive the 2-th time
use sync_diff_inspector to check increment data
check diff successfully
[Sat May 4 14:59:19 CST 2024] <<<<<< finish test_isolate_master_and_worker >>>>>>
5 dm-master alive
5 dm-worker alive
0 dm-syncer alive
wait process dm-master.test exit...
wait process dm-master.test exit...
wait process dm-master.test exit...
wait process dm-master.test exit...
wait process dm-master.test exit...
wait process dm-master.test exit...
wait process dm-master.test exit...
got=1 expected=1
got=0 expected=0
got=1 expected=1
got=0 expected=0
got=1 expected=1
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
make: *** [dm_integration_test_in_group] Terminated
/home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/start_task/../_utils/test_prepare: line 232: 3738 Terminated wait_process_exit dm-master.test
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
script returned exit code 143
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
[2024/05/04 14:59:19.013 +08:00] [INFO] [relay.go:96] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
[2024/05/04 14:59:19.033 +08:00] [INFO] [relay.go:125] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
[2024/05/04 14:59:22.571 +08:00] [INFO] [relay.go:96] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
[2024/05/04 14:59:22.588 +08:00] [INFO] [relay.go:125] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
[2024/05/04 14:59:22.606 +08:00] [INFO] [relay.go:125] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
[2024/05/04 14:59:22.627 +08:00] [INFO] [relay.go:125] ["current earliest active relay log"] [task=test] [unit="binlog replication"] ["active relay log"=29c81cde-09e3-11ef-be02-369f5918e2c7.000001/dm-it-1f3e1520-ab0d-4c50-8660-b188a4022ad0-hrlk4-kgmt3-bin.000001]
dmctl test cmd: "query-status test"
got=4 expected=4
dmctl test cmd: "pause-task test"
wait process dm-master.test exit...
wait process dm-master.test exit...
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
check diff successfully
dmctl test cmd: "stop-task test"
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "query-status test"
wait process dm-master.test exit...
check diff successfully
dmctl test cmd: "stop-task test"
[Sat May 4 14:59:27 CST 2024] <<<<<< finish DM-078 >>>>>>
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
0
in Load unit "progress": "0.00 %",
[Sat May 4 14:59:28 CST 2024] <<<<<< finish DM-COMPACT >>>>>>
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
got=1 expected=1
dmctl test cmd: "binlog skip test -b dm-it-0dfc3eba-e207-449a-95e8-96c637e2beb5-9srsn-mcjms-bin|000001.000001:30234"
rpc addr 127.0.0.1:8261 is alive
[Sat May 4 14:59:28 CST 2024] <<<<<< START DM-WORKER on port 8262, config: /home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/relay_interrupt/conf/dm-worker1.toml >>>>>>
wait for rpc addr 127.0.0.1:8262 alive the 1-th time
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
wait process dm-master.test exit...
dmctl test cmd: "query-status test"
got=1 expected=1
dmctl test cmd: "binlog skip test"
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
make: *** [dm_integration_test_in_group] Terminated
/home/jenkins/agent/workspace/pingcap/tiflow/pull_dm_integration_test/tiflow/dm/tests/many_tables/../_utils/test_prepare: line 232: 9356 Terminated wait_until_sync $WORK_DIR "127.0.0.1:$MASTER_PORT"
ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
script returned exit code 143
wait process dm-master.test exit...
dmctl test cmd: "query-status test"
got=2 expected=2
dmctl test cmd: "stop-task test"
wait process dm-master.test exit...
process dm-master.test already exit
[Sat May 4 14:59:29 CST 2024] <<<<<< finish DM-4201 >>>>>>
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
wait process dm-master.test exit...
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
make: *** [dm_integration_test_in_group] Terminated
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
script returned exit code 143
wait process dm-master.test exit...
kill finished with exit code 0
Sending interrupt signal to process
Killing processes
1 dm-master alive
2 dm-worker alive
0 dm-syncer alive
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
kill finished with exit code 0
make: *** [dm_integration_test_in_group] Terminated
script returned exit code 143
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] }
Cache not saved (inner-step execution failed)
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] // cache
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] // dir
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
Click here to forcibly terminate running steps
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] // timeout
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] // container
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] // node
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] // podTemplate
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] // stage
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G00'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G01'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G02'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G03'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G04'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G05'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G06'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G07'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G08'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G09'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G10'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'G11'
[Pipeline] }
Failed in branch Matrix - TEST_GROUP = 'TLS_GROUP'
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 0bdd0939-2f20-4806-9a72-2b606dbfca3b
Finished: ABORTED
![[Jenkins]](/jenkins/static/62b7ca6c/images/svgs/logo.svg){kind=logo}
