[Nasf-Fan]

When incrementally reintegrate some target, before changing the pool map, the PS leader will set pool_prop_recov_cont (a new pool property in RDB) to indicate that the recovering containers for the pool is inprocess. And then the PS leader will send POOL_RECOV_CONT RPC to related target to be reintegrated with the PS leader known containers list for the pool.

During the POOL_RECOV_CONT RPC process, it is possible that someone wants to create or destroy some container(s), then the PS leader will reset the pool_prop_recov_cont in RDB at the beginning of handling related request.

When POOL_RECOV_CONT RPC is replied, the PS leader will know container(s) changes via checking such property. And then it will ask related sponsor to restart the reintegration.

Scuh mechanism will not block container create/destroy during recovering containers. And even if PS leader was switched, it still can work.

For the target being reintegrated, when receives POOL_RECOV_CONT RPC, it will do two cycles scan:

1. Go through the PS leader offered containers list, for each container, if it does not exist locally, then create it.

2. Iterate containers in the local VOS pool, for the one that is not in the PS leader offered list, destroy it. For efficiency, we build the containers tree with the PS leader offered list.

Steps for the author:

• Commit message follows the guidelines.
• Appropriate Features or Test-tag pragmas were used.
• Appropriate Functional Test Stages were run.
• At least two positive code reviews including at least one code owner from each category referenced in the PR.
• Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

• Gatekeeper requested (daos-gatekeeper added as a reviewer).

[github-actions]

Ticket title is 'Handle container create and destroy for incremental reintegration'
Status is 'In Review'
Labels: '2.8pp'
https://daosio.atlassian.net/browse/DAOS-17857

[daosbuild3]

Test stage Unit Test on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16810/1/display/redirect

[daosbuild3]

Test stage Unit Test bdev with memcheck on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16810/1/display/redirect

[daosbuild3]

Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16810/3/display/redirect

[daosbuild3]

Test stage Functional on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16810/6/display/redirect

[Nasf-Fan]

resolve merge conflict.

[daosbuild3]

Test stage Functional Hardware Medium Verbs Provider MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-16810/10/execution/node/1540/log

[daosbuild3]

Test stage Functional Hardware Medium Verbs Provider MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-16810/12/execution/node/1594/log

[Nasf-Fan]

Rebuild_ec timeout because of DAOS-17657.

[Nasf-Fan]

Ping reviewers, Thanks!

[Nasf-Fan]

Something wrong caused the PR closing unexpectedly.

[Nasf-Fan]

Replaced by #16903