Skip to content

Commit 86aa26e

Browse files
GH-3894: Prepare RC: stage through nexus-staging-maven-plugin (#3895)
* Prepare RC: stage through nexus-staging-maven-plugin A plain `mvn deploy` to repository.apache.org relies on Nexus implicitly creating a staging repository, and never tells the client which one it used. prepare-rc.sh then had to guess: nexus_find_open_staging_repo listed every open repository in the org.apache.parquet profile and refused to continue unless there was exactly one. That guess fails in two ways. On 2026-09-10 a single 1.19.0 run ended with two open repositories (orgapacheparquet-1085 and -1086) and aborted before closing either. And parquet-format publishes under the same groupId, so it shares the staging profile: any open repository from a parquet-format release would block a parquet-java release, or be mistaken for one. GitHub concurrency groups are per repository, so the workflows cannot serialize across the two. Add nexus-staging-maven-plugin 1.7.0 to the apache-release profile with extensions enabled, so it replaces maven-deploy-plugin for `mvn deploy` under that profile only. It stages every module locally, opens one staging repository, uploads into it, closes it, and records the repository ID in target/nexus-staging/staging/<profileId>.properties. prepare-rc.sh now reads the ID from that file rather than searching for it, which removes nexus_find_open_staging_repo and its multiple-repo guard. The plugin also takes over the close from our bulk-close call. Its close waits for completion and prints rule failures (such as the missing public key on 2026-09-14), where the bulk call returned while the close was still in progress. keepStagingRepositoryOnCloseRuleFailure keeps a repository that fails a rule so it can be inspected, and the properties file is written before the close, so a failed close still reports which repository was left behind. stagingProgressTimeoutMinutes is raised from 5 to 15 because a 16-module close may run past the default. -DstagingDescription sets "Apache Parquet <version> RC<n>" on the repository at creation, which is the description cancel-rc.sh and publish-release.sh verify. After the deploy, prepare-rc.sh runs the same nexus_verify_staging_repo check against the new repository. stagingProfileId is left unset, so the plugin resolves the profile from the groupId through the profile_evaluate endpoint. If the deployer account cannot call that endpoint, the next run fails before uploading anything and the profile ID has to be configured explicitly. nexus_close_staging_repo had no remaining callers and is removed. cancel-rc.sh and publish-release.sh already take the repository ID as an argument and are unchanged. Generated-by: Claude Opus 5.5 (1M context) * Prepare RC: find the staging repo from the deploy log on failure On a close rule failure, nexus-staging-maven-plugin 1.7.0 deletes its staging properties file in afterUploadFailure before deciding whether to keep the repository. With keepStagingRepositoryOnCloseRuleFailure set, that is exactly the case where a repository is left behind, so reading the properties file on the failure path could never report it, and printed a misleading "Expected one staging properties file" error instead. maven_deploy now tees Maven output to target/nexus-deploy.log, and the failure path reads every ` * Created staging repository with ID "..."` line from it. The plugin logs that line for each repository it opens, before any upload, so it survives every failure mode. The message says to check whether the repository still exists, because some failures (an upload IOException) drop it automatically. PIPESTATUS keeps mvnw's exit status through the tee regardless of pipefail. Set autoReleaseAfterClose=false explicitly. It is the default, but the plugin also binds it to -DautoReleaseAfterClose, and a release after close would publish an RC before the vote; configuration in the pom takes precedence over the user property. The plugin version moves to a nexus-staging-maven-plugin.version property like the other build plugins. The step summary now reports success only after nexus_verify_staging_repo passes, and records a line for each failure exit. Tests cover the stale staging directory being cleared, the description reaching mvnw as one argument, mvnw's exit status propagating through the tee, more than one properties file, the default staging directory, and parsing repository IDs from the deploy log. Generated-by: Claude Opus 5.5 (1M context) * Prepare RC: name the staging repo when its ID cannot be read When the deploy succeeds but maven_read_staging_repo_id fails, the step summary only said to look in the Maven log. The deploy log already records every repository the plugin created, so report those IDs the same way the deploy failure path does. If the log shows no created repository either, nexus-staging-maven-plugin most likely did not run and maven-deploy-plugin deployed into a repository Nexus created implicitly, so say that and point at Nexus. Generated-by: Claude Opus 5.5 (1M context)
1 parent 02af344 commit 86aa26e

7 files changed

Lines changed: 231 additions & 231 deletions

File tree

‎pom.xml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@
103103
<net.openhft.version>0.27ea1</net.openhft.version>
104104
<build-helper-maven-plugin.version>3.6.2</build-helper-maven-plugin.version>
105105
<exec-maven-plugin.version>3.6.4</exec-maven-plugin.version>
106+
<nexus-staging-maven-plugin.version>1.7.0</nexus-staging-maven-plugin.version>
106107
<protoc-jar-maven-plugin.version>3.11.4</protoc-jar-maven-plugin.version>
107108
<jts.version>1.20.0</jts.version>
108109

@@ -726,6 +727,27 @@
726727
</properties>
727728
</profile>
728729

730+
<profile>
731+
<id>apache-release</id>
732+
<build>
733+
<plugins>
734+
<plugin>
735+
<groupId>org.sonatype.plugins</groupId>
736+
<artifactId>nexus-staging-maven-plugin</artifactId>
737+
<version>${nexus-staging-maven-plugin.version}</version>
738+
<extensions>true</extensions>
739+
<configuration>
740+
<nexusUrl>https://repository.apache.org/</nexusUrl>
741+
<serverId>apache.releases.https</serverId>
742+
<autoReleaseAfterClose>false</autoReleaseAfterClose>
743+
<keepStagingRepositoryOnCloseRuleFailure>true</keepStagingRepositoryOnCloseRuleFailure>
744+
<stagingProgressTimeoutMinutes>15</stagingProgressTimeoutMinutes>
745+
</configuration>
746+
</plugin>
747+
</plugins>
748+
</build>
749+
</profile>
750+
729751
<profile>
730752
<id>vector-plugins</id>
731753
<modules>

‎release/bin/prepare-rc.sh‎

Lines changed: 24 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -263,27 +263,35 @@ step_summary "### Nexus Deployment"
263263
settings_file=$(mktemp "${RUNNER_TEMP:-${TMPDIR:-/tmp}}/parquet-release-settings.XXXXXX")
264264
generate_maven_settings "${settings_file}"
265265

266-
maven_deploy "${settings_file}"
267-
268-
step_summary "Deployed artifacts to Apache Nexus staging"
269-
270-
# Find and close the staging repo. Verify it actually contains the
271-
# artifacts we just deployed before closing -- another concurrent run
272-
# could have created the only open repo we see, and closing the wrong
273-
# one would corrupt that release.
274-
nexus_find_open_staging_repo "${NEXUS_PROFILE_NAME}"
266+
staging_description="Apache Parquet ${version} RC${rc_number}"
267+
268+
if ! maven_deploy "${settings_file}" "${staging_description}"; then
269+
if maven_find_created_staging_repos; then
270+
print_error "Deploy failed after creating staging repository ${created_staging_repo_ids[*]}; check whether it is still in Nexus and drop it before retrying."
271+
step_summary "Deploy **FAILED** after creating staging repository \`${created_staging_repo_ids[*]}\`; check Nexus for leftovers"
272+
else
273+
step_summary "Deploy **FAILED** before a staging repository was created"
274+
fi
275+
exit 1
276+
fi
275277

276-
if [[ ${DRY_RUN:-1} -ne 1 ]]; then
277-
if ! nexus_check_staging_artifact "${staging_repo_id}" "${version}"; then
278-
print_error "Open staging repo ${staging_repo_id} does not contain ${version} artifacts."
279-
print_error "Refusing to close it -- another release may be in flight. Investigate Nexus before retrying."
280-
exit 1
278+
if ! maven_read_staging_repo_id; then
279+
if maven_find_created_staging_repos; then
280+
print_error "Deploy succeeded but the staging repository ID could not be read; the deploy log reports creating ${created_staging_repo_ids[*]}."
281+
step_summary "Deploy succeeded but the staging repository ID could not be read; deploy log reports \`${created_staging_repo_ids[*]}\`"
282+
else
283+
print_error "Deploy succeeded but no staging repository was found; nexus-staging-maven-plugin may not have run, so check Nexus for an implicitly created repository."
284+
step_summary "Deploy succeeded but no staging repository was found; check Nexus for an implicitly created repository"
281285
fi
286+
exit 1
282287
fi
283288

284-
nexus_close_staging_repo "${staging_repo_id}" "Apache Parquet ${version} RC${rc_number}"
289+
if ! nexus_verify_staging_repo "${staging_repo_id}" "${version}" "${rc_number}"; then
290+
step_summary "Verification **FAILED** for staging repository \`${staging_repo_id}\`"
291+
exit 1
292+
fi
285293

286-
step_summary "Closed staging repository: \`${staging_repo_id}\`"
294+
step_summary "Deployed, closed and verified staging repository: \`${staging_repo_id}\`"
287295

288296
maven_cleanup_settings "${settings_file}"
289297

‎release/libs/_constants.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ NEXUS_CONTENT_BASE_URL=${NEXUS_CONTENT_BASE_URL:-"${NEXUS_BASE_URL}/repositories
3333
NEXUS_STAGING_GROUP_URL="https://repository.apache.org/content/groups/staging/org/apache/parquet/"
3434

3535
NEXUS_PROFILE_NAME="org.apache.parquet"
36+
NEXUS_LOCAL_STAGING_DIR="target/nexus-staging"
37+
MAVEN_DEPLOY_LOG="target/nexus-deploy.log"
3638
NEXUS_VERIFY_GROUP_PATH="org/apache/parquet"
3739
NEXUS_VERIFY_ARTIFACT_ID="parquet-common"
3840

‎release/libs/_maven.sh‎

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,20 +62,76 @@ SETTINGS_EOF
6262
print_info "Generated Maven settings at ${settings_file} (credentials resolved from env vars at build time)"
6363
}
6464

65+
# maven_deploy <settings-file> <staging-description>
66+
# Deploys through nexus-staging-maven-plugin, which opens one staging
67+
# repository, uploads every module into it, and closes it. Output is
68+
# also written to MAVEN_DEPLOY_LOG.
6569
function maven_deploy {
6670
local settings_file="${1:-.release-settings.xml}"
71+
local staging_description="$2"
6772

6873
if [[ ! -f "${settings_file}" ]]; then
6974
print_info "Generating Maven settings..."
7075
generate_maven_settings "${settings_file}"
7176
fi
7277

78+
exec_process rm -rf "${NEXUS_LOCAL_STAGING_DIR}"
79+
mkdir -p "$(dirname "${MAVEN_DEPLOY_LOG}")"
80+
7381
exec_process ./mvnw deploy \
7482
-Papache-release \
7583
-DskipTests \
7684
-Darguments=-DskipTests \
85+
"-DstagingDescription=${staging_description}" \
7786
--settings "${settings_file}" \
78-
--batch-mode
87+
--batch-mode | tee "${MAVEN_DEPLOY_LOG}"
88+
return "${PIPESTATUS[0]}"
89+
}
90+
91+
# maven_read_staging_repo_id [local-staging-dir]
92+
# Sets staging_repo_id from the properties file nexus-staging-maven-plugin
93+
# writes after uploading, before it attempts the close.
94+
function maven_read_staging_repo_id {
95+
local staging_dir="${1:-${NEXUS_LOCAL_STAGING_DIR}}/staging"
96+
97+
if [[ ${DRY_RUN:-1} -eq 1 ]]; then
98+
print_command "Dry-run, WOULD read staging repository ID from ${staging_dir}/*.properties"
99+
staging_repo_id="DRY-RUN-REPO-ID"
100+
return 0
101+
fi
102+
103+
local properties_files=("${staging_dir}"/*.properties)
104+
if [[ ${#properties_files[@]} -ne 1 || ! -f "${properties_files[0]}" ]]; then
105+
print_error "Expected one staging properties file in ${staging_dir}, found: ${properties_files[*]}"
106+
return 1
107+
fi
108+
109+
staging_repo_id=$(sed -n 's/^stagingRepository\.id=//p' "${properties_files[0]}")
110+
if [[ -z "${staging_repo_id}" ]]; then
111+
print_error "No stagingRepository.id in ${properties_files[0]}"
112+
return 1
113+
fi
114+
115+
print_info "Staging repository: ${staging_repo_id}"
116+
return 0
117+
}
118+
119+
# maven_find_created_staging_repos [deploy-log]
120+
# Sets created_staging_repo_ids to every staging repository the plugin
121+
# reported creating. Used after a failed deploy, when the plugin may
122+
# already have deleted its properties file.
123+
function maven_find_created_staging_repos {
124+
local deploy_log="${1:-${MAVEN_DEPLOY_LOG}}"
125+
126+
created_staging_repo_ids=()
127+
if [[ -f "${deploy_log}" ]]; then
128+
local repo_id
129+
while IFS= read -r repo_id; do
130+
created_staging_repo_ids+=("${repo_id}")
131+
done < <(sed -n 's/.*Created staging repository with ID "\([^"]*\)".*/\1/p' "${deploy_log}")
132+
fi
133+
134+
[[ ${#created_staging_repo_ids[@]} -gt 0 ]]
79135
}
80136

81137
function maven_cleanup_settings {

‎release/libs/_nexus.sh‎

Lines changed: 0 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -50,12 +50,6 @@ function _nexus_bulk_action {
5050
fi
5151
}
5252

53-
function nexus_close_staging_repo {
54-
local repo_id="$1"
55-
local description="${2:-Closing staging repository}"
56-
_nexus_bulk_action "close" "${repo_id}" "${description}"
57-
}
58-
5953
function nexus_release_staging_repo {
6054
local repo_id="$1"
6155
local description="${2:-Releasing staging repository}"
@@ -170,71 +164,3 @@ function nexus_verify_staging_repo {
170164
print_info "Staging repository ${repo_id} verified."
171165
return 0
172166
}
173-
174-
function nexus_find_open_staging_repo {
175-
local profile_name="${1:-${NEXUS_PROFILE_NAME}}"
176-
177-
print_info "Searching for open staging repository for ${profile_name}..."
178-
179-
if [[ ${DRY_RUN:-1} -eq 1 ]]; then
180-
print_command "Dry-run, WOULD search Nexus for open staging repo"
181-
staging_repo_id="DRY-RUN-REPO-ID"
182-
return 0
183-
fi
184-
185-
local response
186-
if ! response=$(curl --fail --silent --show-error \
187-
-K <(printf 'user = "%s:%s"\n' "${NEXUS_USERNAME}" "${NEXUS_PASSWORD}") \
188-
"${NEXUS_BASE_URL}/staging/profile_repositories"); then
189-
print_error "Failed to query Nexus staging repositories"
190-
return 1
191-
fi
192-
193-
# Filter on <profileName> exactly (not on a substring of <repositoryId>),
194-
# and reject the result if multiple open repos exist for the profile —
195-
# Nexus permits this but it indicates a leftover from a previous failed
196-
# run that the operator must clean up before we can pick the right one.
197-
local matches
198-
matches=$(echo "${response}" | \
199-
NEXUS_PROFILE_NAME_VAR="${profile_name}" python3 -c "
200-
import sys, os, xml.etree.ElementTree as ET
201-
profile = os.environ['NEXUS_PROFILE_NAME_VAR']
202-
try:
203-
tree = ET.parse(sys.stdin)
204-
except ET.ParseError as e:
205-
print('PARSE_ERROR:' + str(e), file=sys.stderr)
206-
sys.exit(2)
207-
for repo in tree.findall('.//stagingProfileRepository'):
208-
repo_type = repo.find('type')
209-
repo_profile = repo.find('profileName')
210-
repo_id = repo.find('repositoryId')
211-
if repo_type is None or repo_id is None or repo_profile is None:
212-
continue
213-
if repo_type.text == 'open' and repo_profile.text == profile:
214-
print(repo_id.text)
215-
" 2>&1)
216-
local py_status=$?
217-
218-
if [[ ${py_status} -ne 0 ]]; then
219-
print_error "Failed to parse Nexus staging response: ${matches}"
220-
return 1
221-
fi
222-
223-
local match_count
224-
match_count=$(echo -n "${matches}" | grep -c . || true)
225-
226-
if [[ ${match_count} -eq 0 ]]; then
227-
print_error "No open staging repository found for profile '${profile_name}'"
228-
return 1
229-
fi
230-
if [[ ${match_count} -gt 1 ]]; then
231-
print_error "Multiple open staging repositories found for profile '${profile_name}':"
232-
echo "${matches}" | while read -r line; do print_error " - ${line}"; done
233-
print_error "Please drop or close the stale repos in Nexus and re-run."
234-
return 1
235-
fi
236-
237-
staging_repo_id=$(echo "${matches}" | head -n 1)
238-
print_info "Found staging repository: ${staging_repo_id}"
239-
return 0
240-
}

0 commit comments

Comments
 (0)