Changes

feat(pingcap/tidb): enable job `pull-integration-realcluster-test-next-gen` (#3706)

After removing failing test groups, now it can be turned as required if
be triggered.

This pull request updates the integration test pipeline and presubmit
configuration for the TiDB project. The main focus is on cleaning up the
test matrix by removing or commenting out unstable test groups and
updating the job configuration to better manage CI triggers and
reporting.

Pipeline test matrix cleanup:

* Removed several unstable or failing test groups from the
`SCRIPT_AND_ARGS` axis in `pipeline.groovy`, including
`bazel_pessimistictest`, `bazel_txntest`, `bazel_addindextest`,
`bazel_pipelineddmltest`, and `bazel_ddltest`, as well as the
integration test with `-d y` option. These have been commented out for
future reference.
[[1]](diffhunk://#diff-de7893675d755e67054c7fbc4b762de79fb76cf2472fe9110f5fc8f47abb51edL78-L84)
[[2]](diffhunk://#diff-de7893675d755e67054c7fbc4b762de79fb76cf2472fe9110f5fc8f47abb51edL93-R95)

Presubmit job configuration updates:

* Enabled the `skip_if_only_changed` filter and updated the job context
to `pull-integration-realcluster-test-next-gen` in the presubmit YAML,
while removing the `optional: true` flag and the old non-blocking
context.


Test results:

-
https://do.pingcap.net/jenkins/job/pingcap/job/tidb/job/pull_integration_realcluster_test_next_gen/73/
> <img width="1147" height="157" alt="image"
src="https://github.com/user-attachments/assets/f6224d1b-20fa-455d-ad69-5fe7cdfa26a6"
/>
-
https://do.pingcap.net/jenkins/job/pingcap/job/tidb/job/pull_integration_realcluster_test_next_gen/75/
  - >
 (commit: cf80423)