Auto-generate JSON schema from Rust types

[mre]

• Builds on feat: generate a JSON schema for the config file #2286.
• Fixes the generated schema so it matches supported config values.
• Adds coverage for valid and invalid configs.
• @ChrisJr404, could you take a look? The diff to schema PR is here. Note that it also contains some changes that landed in master. You can rebase your branch to see the clean diff if you like. 😃

[mre]

While working on this feature, I noticed that lychee trims surrounding whitespace from basic-auth selectors. For example:

basic_auth = ["example.com user:password "]

The trailing space is silently removed, even though it could be part of the password. That's what lychee did before this PR as well, so it's not something introduced by this PR.

Other tools generally preserve spaces in the supplied credentials rather than trim or reject them: curl, Python Requests, and HTTPie. RFC 7617 also permits ordinary spaces in Basic-auth credentials, though it forbids control characters.

For this PR, I've made the schema match our existing parser. That's because rejecting whitespace only in the schema would mean that editor validation rejects configurations that lychee accepts at the moment. And I've added a comment explaining that choice.

I think we should revisit the trimming in the near future. Ideally, we'd preserve credential content rather than silently change it, but that requires a decision on how we will handle our combined <uri-regex> <username>:<password> syntax in the future. Not sure and open for suggestions.

[mre]

The failing basic auth test is unrelated. Flaky endpoint.

[mre]

This has been open for a while. I made a few more tweaks and switched to JSON Schema Draft 7, matching the draft used by the published SchemaStore schema.

I compared both schemas against lychee's config loader. They aren't identical: the generated schema covers newer options and follows the parser more closely in several places:

Configuration	Published schema	Generated schema	lychee
cache_location = "cache.json"	Rejects	Accepts	Accepts
Per-host accept = [200, 429]	Rejects	Accepts	Accepts
verbose = "Warning"	Rejects	Accepts	Accepts
generate = "config-schema"	Rejects	Accepts	Accepts
accept = "banana"	Accepts	Rejects	Rejects
method = "GET POST"	Accepts	Rejects	Rejects
basic_auth = ["user:password"]	Accepts	Rejects	Rejects
accept = "200..1000"	Accepts	Accepts	Accepts

The comparison also caught a bug in our handling of exclusive range endpoints, which is now fixed. Some checks still belong to the parser: for example, both schemas accept "500..400", but lychee rejects the reversed range.

The generated file will be at docs/schema.json, with a test to keep it in sync with the Rust configuration types. It will also be included in release archives. This finally resolves the request for a versioned schema in #1382.

@Nick2bad4u, thanks for adding the existing SchemaStore schema. Would you mind updating it to use this generated version?

I'll go ahead and merge this now as the change should be rather uncontroversial. :)

[Nick2bad4u]

This has been open for a while. I made a few more tweaks and switched to JSON Schema Draft 7, matching the draft used by the published SchemaStore schema.

I compared both schemas against lychee's config loader. They aren't identical: the generated schema covers newer options and follows the parser more closely in several places:

Configuration	Published schema	Generated schema	lychee
cache_location = "cache.json"	Rejects	Accepts	Accepts
Per-host accept = [200, 429]	Rejects	Accepts	Accepts
verbose = "Warning"	Rejects	Accepts	Accepts
generate = "config-schema"	Rejects	Accepts	Accepts
accept = "banana"	Accepts	Rejects	Rejects
method = "GET POST"	Accepts	Rejects	Rejects
basic_auth = ["user:password"]	Accepts	Rejects	Rejects
accept = "200..1000"	Accepts	Accepts	Accepts

The comparison also caught a bug in our handling of exclusive range endpoints, which is now fixed. Some checks still belong to the parser: for example, both schemas accept "500..400", but lychee rejects the reversed range.

The generated file will be at docs/schema.json, with a test to keep it in sync with the Rust configuration types. It will also be included in release archives. This finally resolves the request for a versioned schema in #1382.

@Nick2bad4u, thanks for adding the existing SchemaStore schema. Would you mind updating it to use this generated version?

I'll go ahead and merge this now as the change should be rather uncontroversial. :)

Which url do you want to use? The raw.github @ main branch since it'll pull directly from the url instead and update automatically?

[nobkd]

Maybe github has something like https://raw.githubusercontent.com/lycheeverse/lychee/tags/nightly/docs/schema.json, but for the latest full release (instead of nightly tag)?