Fix integ test

[emmuhamm]

Description

Fixes #932

See above for a description of the problem.

Also added another quick change in which we now have the --verbosity option in the drunc integ test. by default, it set to whatever the default is so the output in the terminal is truncated. But for debugging, it is useful to set the verbosity level explicitly. I find that 5 works well.

Type of change

• Optimization
• Bug fix

List of required branches from other repositories

Will need to go in with or after this goes in

DUNE-DAQ/integrationtest#158

Suggested manual testing checklist

Check out this branch

Check out the linked PR

Make sure they are both in the pythoncode directory

Run

• ./scripts/drunc_integtest_bundle.sh . See its all green
• ./scripts/drunc_integtest_bundle.sh --verbosity 5. See that extra messages are being printed out in the terminal.

Developer checklist

Prior to marking this as "Ready for Review"

Tests ran on: np04-srv-028 from release NFD_DEV_260526_A9

• Unit tests (pytest --marker) passed
  • With relevant marker
  • Without marker
• Integration tests passed
  • Only daqsystemtest_integtest_bundle.sh -k minimal_system_quick_test.py
  • Full daqsystemtest_integtest_bundle.sh
• Testing skipped as there are no core code changes in this PR, this only relates to documentation/CI workflows
• Drunc integration tests pass (./scripts/drunc_integtest_bundle.sh)

Final checklist prior to marking this as "Ready for Review"

• Code is clearly commented.
• New unit tests have been added, or is documented in # ISSUE NUMBER
• A suitable reviewer has been chosen from this list.

Reviewer checklist

• This branch has been rebased with develop prior to testing.
• Suggested manual tests show changes.
• CI workflows fails documented (if present)
• Integration tests passed (on either np0x or IC HEP clusters)
  • Use the following guidelines to determine which of the integration tests you need to run
    • You do not need to run any integration tests if
      • Code changes are not associated with src/
      • PR changes only affect docstrings
      • In this case, be sure to validate any suggested manual testing.
    • Run only the minimum integration test as daqsystemtest_integtest_bundle.sh -k minimal_system_quick_test.py if
      • PR changes only affect a few log entries
      • PR changes are small, and do not have a large impact on the workflow (use carefully)
    • Otherwise run the full integration test bundle as daqsystemtest_integtest_bundle.sh
  • What to do if the integration tests fail?
    • Only concern yourself if failures related to drunc are in the log files
    • If non-drunc failure appears:
      • Validate failure in fresh working area
      • Contact Pawel if unsure
• If you have ran the full integration test bundle, leave a comment on the PR stating
  • Which host the integration tests have ran on
  • [Optional] A copy of the test summary
• Drunc integration tests pass (scripts/drunc_integtest_bundle.sh)

Once the above boxes are checked, the PR(s) can be merged following the steps below.

Prior to merging

Choose one of the following an complete all substeps

• Changes only affect the Run Control, are in a single repository, and do not affect the end user.
  • Changes are documented in docstrings and code comments
  • Wiki has been updated if architectural or endpoint changes
• Otherwise
  • Workflow changes demonstrated in the Change Log (if necessary)
  • Wiki has been updated (if necessary)
  • #dunedaq-integration Slack channel notified (see below)

Once completed, the reviewer can merge the PR.

Notification message for a Slack channel

Note - this should be to #dunedaq-integration for general workflow that isn't during a release candidate period, and to #daq-release-prep otherwise.

For an single merge that changes the user workflow

The CCM WG has an isolated PR ready to merge that affects user workflows. The PR is:

_URL_

I will leave time for any comments, otherwise will merge these at the end of the work day _Insert your time zone_.

For co-ordinated merge

The CCM WG has a set of co-ordinated merges ready to merge. The PRs are:

_URL_

_URL_


I will leave time for any comments, otherwise will merge these at the end of the day.

[emmuhamm]

Hi @Aurashk, @PawelPlesniak, this is a small PR that fixes the PM integtest we have in drunc. See details in the issue.

I think the changes in the output messages are from #896, so @Aurashk could you also take a look at this and see if you're happy with it?

Its a small PR so I'm happy if either @Aurashk or @PawelPlesniak can look into it and approve it. It has to go in after the listed dependency tho so we can wait until that gets merged before this can go in.

[emmuhamm]

Flagging this up for the review of DUNE-DAQ/integrationtest#158

[Aurashk]

lgtm, only one thing to mention below

[Aurashk]

My only concern is this line, I'm not sure we've consistently got things so that the remote pid is always retrieved successfully for a clean exit. Although this is strictly better as it's more up to date so would support merging this

[emmuhamm]

Thanks Aurash, thats useful information to know!

My two cents:

• Was unaware that we might not have a consistent thing for the remote pid
• For this integtest I think its fine
  • integtest is not a blocker for most PRs to merge
  • this (transiently?) failing would be good to keep track of the issue until the remote pid issue gets properly fixed

[emmuhamm]

I'll consider this approved and ready to go in now. Since this is a coordinated merge, I'll announce in the relevant slack channels for ppl to know about this. Fingers crossed I'll go and get this merged by tomorrow 10 am UK time or so

[PawelPlesniak]

Confirmed the expected behaviour, so this can be announced today