diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..29a0c893 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,8 @@ +* text=auto eol=lf +*.java text eol=lf +*.xml text eol=lf +*.properties text eol=lf +*.md text eol=lf +*.yaml text eol=lf +*.yml text eol=lf +*.sh text eol=lf diff --git a/.github/workflows/ci-jmeter-compatibility.yaml b/.github/workflows/ci-jmeter-compatibility.yaml index 93a7f4fa..2354aac0 100644 --- a/.github/workflows/ci-jmeter-compatibility.yaml +++ b/.github/workflows/ci-jmeter-compatibility.yaml @@ -43,13 +43,23 @@ jobs: uses: actions/cache@v5 with: path: ${{ github.workspace }}/.jmeter - key: jmeter-${{ runner.os }}-${{ hashFiles('pom.xml', 'src/test/resources/jmeter/**') }} + key: jmeter-${{ runner.os }}-${{ hashFiles('pom.xml', 'src/test/resources/jmeter/**', 'src/test/resources/jmeter-regression/**') }} restore-keys: | jmeter-${{ runner.os }}- - name: Build project and jmeter-test bundle run: xvfb-run -a mvn -U --batch-mode clean install + - name: Run JMeter HTTP parity regression (all tiers) + run: >- + xvfb-run -a mvn -Pjmeter-regression + -Dcheckstyle.skip=true + -DskipTests + -Djmeter.regression.tier=all + -Djmeter.regression.tolerateExternalServiceDrift=true + --batch-mode + verify + - name: Run compatibility matrix on Java 17 working-directory: target/jmeter-test env: diff --git a/.gitignore b/.gitignore index 44469d5c..66b640ab 100644 --- a/.gitignore +++ b/.gitignore @@ -211,6 +211,7 @@ Temporary Items ### Maven ### target/ +cp.txt pom.xml.tag pom.xml.releaseBackup pom.xml.versionsBackup diff --git a/README.md b/README.md index f833e901..bc268fdf 100644 --- a/README.md +++ b/README.md @@ -427,6 +427,15 @@ Uses **Maven 3** from the repository root: Artifacts land under **`target/`**. Compilation uses the **`jmeter.version`** declared in **`pom.xml`**; at runtime install the packaged JAR against the JMeter build you intend to run and validate with a short smoke plan. +### HTTP parity regression (JMeter 5.6.3 test plans) + +To compare **HttpClient4** with the migrated **BlazeMeter HTTP** sampler against Apache’s official `bin/testfiles` JMX plans, see **[docs/jmeter-regression.md](docs/jmeter-regression.md)**. + +```bash +mvn -Dcheckstyle.skip=true package +mvn -Pjmeter-regression verify +``` + ## License diff --git a/checkstyle.xml b/checkstyle.xml index 13460b7a..5d6c5a86 100644 --- a/checkstyle.xml +++ b/checkstyle.xml @@ -10,6 +10,11 @@ + + + + @@ -104,12 +109,6 @@ - - - - - diff --git a/docs/jmeter-regression.md b/docs/jmeter-regression.md new file mode 100644 index 00000000..3a5550c5 --- /dev/null +++ b/docs/jmeter-regression.md @@ -0,0 +1,126 @@ +# JMeter HTTP regression parity tests + +This suite compares Apache JMeter **HttpClient4** (reference) with the migrated **BlazeMeter HTTP** sampler (`HTTP2Sampler`) using official JMeter `bin/testfiles` plans from release **5.6.3**. + +## What it does + +1. Runs the stock JMX with `-Jjmeter.httpsampler=HttpClient4` (no plugin). +2. Migrates HTTP Request samplers headlessly via `JmxBlazeMeterHttpMigrator`. +3. Runs the migrated plan with the plugin installed under `lib/ext/`. +4. Compares sample trees semantically (`HttpsampleResultComparator`): labels, success, response codes/messages, bodies, and normalized response headers. Timings and volatile headers are ignored. + +## Regression groups + +These groups are this project's own way of organizing which JMeter `bin/testfiles` plans run in which CI stage — not an Apache JMeter classification. All vendored plans (including the ones below) come from the same upstream source (see [Resources](#resources)). + +| Group | Maven | Plans | Notes | +|-------|-------|-------|-------| +| **Core** (`tier=core`) | `-Pjmeter-regression` | `TEST_HTTP`, `ResponseDecompression`, `TestHeaderManager`, `TestCookieManager` | Default CI; mirror + managers | +| **Extended — HTTPS/Auth/HTML** (`tier=extended`) | `-Pjmeter-regression-extended` | `HTMLParserTestFile_2`, `TEST_HTTPS`, `Http4ImplDigestAuth`, `Http4ImplPreemptiveBasicAuth` | HTTPS + embedded HTML + auth | +| **External services** (`tier=external`) | `-Pjmeter-regression-external` | `TestKeepAlive`, `TestRedirectionPolicies` | Calls real third-party hosts; opt-in only, not in default CI | +| **JUnit parity** | `mvn test -Dtest=com.blazemeter.jmeter.http2.parity.*` | JUnit ports of Apache HTTP module tests | HttpClient4 vs plugin in-process; HTTP/1.1 — see [below](#junit-parity-apache-srcprotocolhttp-tests) | + +`BUG_62847` / `Bug54685` are vendored under `5.6.3/` but exercise JMeter controllers/Java samplers only — not HTTP plugin parity. + +`SlowCharsFeature` (optional): run with `-Djmeter.regression.tests=SlowCharsFeature -Djmeter.regression.enableSlowChars=true` once Jetty supports HttpClient4-style **CPS** throttling (`httpclient.socket.*.cps`). See [SlowCharsFeature](#slowcharsfeature) below. + +### SlowCharsFeature + +Apache batch plan `SlowCharsFeature.jmx`: + +1. A setup thread sets `httpclient.socket.http.cps` and `httpclient.socket.https.cps` to **1500** (bytes per second). +2. The main sampler requests `https://jmeter.apache.org/...` with `Range: bytes=0-7000` (partial content). +3. Assertions expect **HTTP 206**, ~7001 bytes, and **elapsed time > 5s** (download throttled to ~1.5 KB/s). + +HttpClient4 implements CPS by wrapping socket streams with a rate limiter (`org.apache.http.impl.conn.SocketFactory` / connection socket config). **Jetty has no equivalent property**: you would need a custom `ClientConnector` / `EndPoint` wrapper that throttles `fill()`/`flush()` per connection, and decide how CPS applies across HTTP/2 multiplexing and HTTP/3 QUIC streams (HC4's model is per-TCP-socket). Until that exists, the plan stays behind `-Djmeter.regression.enableSlowChars=true`. + +`TestKeepAlive` is compared on **http1-only** only (HttpClient4 never speaks HTTP/2; keep-alive / `Connection: close` are HTTP/1.1 semantics). + +Use `-Djmeter.regression.tier=extended|external|all` instead of a profile, or override with `-Djmeter.regression.tests=PlanA,PlanB`. + +## Prerequisites + +- **Java 17+** +- **Maven 3.6+** +- Network access (first run downloads JMeter 5.6.3; Core/Extended plans may call external hosts). + +## Build the plugin JAR + +```bash +mvn -Dcheckstyle.skip=true package +``` + +## Run regression tests + +```bash +mvn -Pjmeter-regression verify +``` + +### Options + +| Property | Default | Description | +|----------|---------|-------------| +| `jmeter.regression` | `false` (`true` with `-Pjmeter-regression*`) | Must be `true` to execute regression ITs | +| `jmeter.regression.tests` | Core group list (see `pom.xml`) | Comma-separated JMX base names (no `.jmx`) | +| `jmeter.regression.tier` | _(unset)_ | `core`, `extended`, `external`, or `all` when `jmeter.regression.tests` is unset | +| `jmeter.regression.version` | `5.6.3` | JMeter distribution version to download | +| `jmeter.home` | _(auto)_ | Use an existing JMeter install instead of downloading | +| `it.http3` | `false` | Also run each plan with HTTP/3 profile | +| `jmeter.regression.protocol` | _(all profiles)_ | Filter: `http1-only`, `http2`, or `http3` (with `it.http3`) | +| `jmeter.regression.timeoutMinutes` | `10` | Per-run timeout | +| `jmeter.regression.tolerateExternalServiceDrift` | `false` (auto for Extended auth + External-services plans) | Skip strict compare when ref/plugin saw 5xx vs 2xx on the same sample (sequential runs vs flaky hosts) | + +### Examples + +Core group only, HTTP/1.1 profile: + +```bash +mvn -Pjmeter-regression -Djmeter.regression.protocol=http1-only verify +``` + +Extended group — start with local HTML parser (no network): + +```bash +mvn -Pjmeter-regression-extended -Djmeter.regression.tests=HTMLParserTestFile_2 verify +``` + +Full Extended group: + +```bash +mvn -Pjmeter-regression-extended verify +``` + +External-services / optional batch plans: + +```bash +mvn -Pjmeter-regression-external verify +``` + +### JUnit parity (Apache `src/protocol/http` tests) + +Runs in the normal unit-test phase (`mvn test`), not Failsafe. Each test executes the same sampler configuration through **HttpClient4** and **HTTP2JettyClient** (HTTP/1.1 forced). + +| Class | Apache source | +|-------|----------------| +| `HttpMirrorParityTest` | `TestHTTPSamplersAgainstHttpMirrorServer` (GET/PUT, POST urlencoded/multipart/raw, file upload) | +| `HttpMirrorItemisedParityTest` | same source — `itemised_testPostRequest_UrlEncoded` (0–7), `itemised_testGetRequest_Parameters` (0–5) | +| `HttpMirrorMultipartParityTest` | same source — `testPostRequest_FormMultipart` (0–6) | +| `HttpMirrorRawBodyParityTest` | same source — `testPostRequest_BodyFromParameterValues` (0–9) | +| `HttpMirrorFileUploadParityTest` | same source — `testPostRequest_FileUpload` (0–2) | +| `HttpRedirectsParityTest` | `TestRedirects` (301–308, no follow; incl. HEAD/PUT/DELETE) | +| `HttpRedirectsFollowParityTest` | redirect follow (`followRedirects` + `autoRedirects`) | +| `HttpCookieManagerParityTest` | `TestHC4CookieManager` (set/echo cookies) | +| `HttpCacheManagerParityTest` | `TestCacheManagerHC4` (embedded cache hit) | +| `HttpDisableArgumentsParityTest` | skippable args (5.6.3); `enabled` checkbox → JMeter 5.7+ | + +```bash +mvn test -Dcheckstyle.skip=true -Dtest=com.blazemeter.jmeter.http2.parity.* +``` + +## CI + +`ci-jmeter-compatibility.yaml` runs **all regression groups** (`tier=all`: Core + Extended + External-services) via `-Pjmeter-regression` after `mvn clean install` (which also runs JUnit parity via Surefire). External-service drift tolerance is enabled in CI for auth/keep-alive plans. + +## Resources + +Vendored under `src/test/resources/jmeter-regression/5.6.3/` from [apache/jmeter rel/v5.6.3 bin/testfiles](https://github.com/apache/jmeter/tree/rel/v5.6.3/bin/testfiles). diff --git a/pom.xml b/pom.xml index 7735dcb4..31c22d20 100644 --- a/pom.xml +++ b/pom.xml @@ -31,6 +31,10 @@ UTF-8 UTF-8 5.5 + 5.6.3 + TEST_HTTP,ResponseDecompression,TestHeaderManager,TestCookieManager + HTMLParserTestFile_2,TEST_HTTPS,Http4ImplDigestAuth,Http4ImplPreemptiveBasicAuth + TestKeepAlive,TestRedirectionPolicies 12.1.11 1.23.0 @@ -49,6 +53,18 @@ ${jmeter.version} provided + + org.apache.jmeter + ApacheJMeter_components + ${jmeter.version} + provided + + + org.apache.jmeter + ApacheJMeter_java + ${jmeter.version} + test + @@ -63,6 +79,12 @@ 4.5.14 compile + + commons-io + commons-io + 2.15.1 + test + org.apache.commons commons-lang3 @@ -470,7 +492,14 @@ org.apache.maven.plugins maven-checkstyle-plugin - 3.1.0 + 3.3.1 + + + com.puppycrawl.tools + checkstyle + 10.17.0 + + validate @@ -492,9 +521,13 @@ maven-failsafe-plugin 2.22.2 + false **/*IntegrationTest.java + + **/regression/** + @@ -649,5 +682,117 @@ + + jmeter-regression + + true + true + true + + + + + org.apache.maven.plugins + maven-surefire-plugin + + true + + + + org.apache.maven.plugins + maven-failsafe-plugin + + false + + **/regression/**/*IntegrationTest.java + + + + ${jmeter.regression} + ${jmeter.regression.tests} + ${jmeter.regression.tier} + ${jmeter.regression.version} + + + + + + + + jmeter-regression-extended + + true + true + true + extended + ${jmeter.regression.extended.tests} + + + + + org.apache.maven.plugins + maven-surefire-plugin + + true + + + + org.apache.maven.plugins + maven-failsafe-plugin + + false + + **/regression/**/*IntegrationTest.java + + + + ${jmeter.regression} + ${jmeter.regression.tests} + ${jmeter.regression.tier} + ${jmeter.regression.version} + + + + + + + + jmeter-regression-external + + true + true + true + external + ${jmeter.regression.external.tests} + + + + + org.apache.maven.plugins + maven-surefire-plugin + + true + + + + org.apache.maven.plugins + maven-failsafe-plugin + + false + + **/regression/**/*IntegrationTest.java + + + + ${jmeter.regression} + ${jmeter.regression.tests} + ${jmeter.regression.tier} + ${jmeter.regression.version} + + + + + + diff --git a/src/main/java/com/blazemeter/jmeter/http2/core/JmeterRequestHeadersSupport.java b/src/main/java/com/blazemeter/jmeter/http2/core/JmeterRequestHeadersSupport.java index fd3b4f7c..051e4aec 100644 --- a/src/main/java/com/blazemeter/jmeter/http2/core/JmeterRequestHeadersSupport.java +++ b/src/main/java/com/blazemeter/jmeter/http2/core/JmeterRequestHeadersSupport.java @@ -78,7 +78,6 @@ private static void applyConnectionHeader(Request request, boolean useKeepAlive) } } - /** * Matches {@code HTTPHC4Impl}: when no {@code User-Agent} is configured, send a plugin default * unless {@code httpclient4.default_user_agent_disabled=true}. diff --git a/src/main/java/com/blazemeter/jmeter/http2/sampler/AsyncCompletionSamplePipeline.java b/src/main/java/com/blazemeter/jmeter/http2/sampler/AsyncCompletionSamplePipeline.java index a0c79505..0c593194 100644 --- a/src/main/java/com/blazemeter/jmeter/http2/sampler/AsyncCompletionSamplePipeline.java +++ b/src/main/java/com/blazemeter/jmeter/http2/sampler/AsyncCompletionSamplePipeline.java @@ -28,7 +28,8 @@ final class AsyncCompletionSamplePipeline { private static final Logger LOG = LoggerFactory.getLogger(AsyncCompletionSamplePipeline.class); - private AsyncCompletionSamplePipeline() {} + private AsyncCompletionSamplePipeline() { + } /** * Mirrors {@code JMeterThread.executeSamplePackage} for the fragment after a successful sample, diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCacheManagerParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCacheManagerParityTest.java new file mode 100644 index 00000000..7584fa65 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCacheManagerParityTest.java @@ -0,0 +1,115 @@ +package com.blazemeter.jmeter.http2.parity; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.JmeterCachedResourceModeSupport; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.core.ServerBuilder; +import com.blazemeter.jmeter.http2.core.ServerBuilder.TeardownableServer; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.net.URL; +import org.apache.jmeter.protocol.http.control.CacheManager; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.apache.jmeter.util.JMeterUtils; +import org.eclipse.jetty.server.ServerConnector; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +/** Port of cache behaviour from Apache {@code TestCacheManagerHC4} (embedded resource caching). */ +public class HttpCacheManagerParityTest extends HTTP2TestBase { + + private TeardownableServer server; + private HTTP2JettyClient client; + private int serverPort; + private String previousCacheMode; + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + JMeterUtils.setProperty("cache_manager.cached_resource_mode", "RETURN_200_CACHE"); + JMeterUtils.setProperty("RETURN_200_CACHE.message", "cached"); + try { + JmeterCachedResourceModeSupport.refreshSnapshotFromProperties(); + } catch (ReflectiveOperationException e) { + throw new IllegalStateException("Failed to refresh JMeter cache mode snapshot", e); + } + } + + @Before + public void setUp() throws Exception { + previousCacheMode = JMeterUtils.getProperty("cache_manager.cached_resource_mode"); + JMeterUtils.setProperty("cache_manager.cached_resource_mode", "RETURN_200_CACHE"); + JMeterUtils.setProperty("RETURN_200_CACHE.message", "cached"); + try { + JmeterCachedResourceModeSupport.refreshSnapshotFromProperties(); + } catch (ReflectiveOperationException e) { + throw new IllegalStateException("Failed to refresh JMeter cache mode snapshot", e); + } + + server = new ServerBuilder().withHTTP1().withSSL().buildServer(); + server.start(); + serverPort = ((ServerConnector) server.getConnectors()[0]).getLocalPort(); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("cache-parity"); + } + + @After + public void tearDown() throws Exception { + if (previousCacheMode != null) { + JMeterUtils.setProperty("cache_manager.cached_resource_mode", previousCacheMode); + } + if (client != null) { + client.stop(); + } + if (server != null) { + server.stop(); + } + } + + @Test + public void secondEmbeddedFetchUsesCacheLikeHttpClient4() throws Exception { + CacheManager cacheManager = new CacheManager(); + cacheManager.setUseExpires(true); + cacheManager.setClearEachIteration(false); + cacheManager.testStarted(ServerBuilder.HOST_NAME); + cacheManager.testIterationStart(null); + + HTTP2Sampler sampler = new HTTP2Sampler(); + sampler.setMethod(HTTPConstants.GET); + sampler.setProtocol("https"); + sampler.setDomain(ServerBuilder.HOST_NAME); + sampler.setPort(serverPort); + sampler.setPath(ServerBuilder.SERVER_PATH_200_EMBEDDED); + sampler.setImageParser(true); // embedded resource download + sampler.setUseKeepAlive(true); + sampler.setCacheManager(cacheManager); + + URL url = new URL("https", ServerBuilder.HOST_NAME, serverPort, + ServerBuilder.SERVER_PATH_200_EMBEDDED); + + HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + HTTPSampleResult refSecond = HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + + HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + HTTPSampleResult pluginSecond = HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + + assertThat(pluginSecond == null).isEqualTo(refSecond == null); + if (refSecond == null || pluginSecond == null) { + return; + } + HttpClient4PluginParitySupport.assertCoreParity(refSecond, pluginSecond, "cached embedded"); + org.apache.jmeter.samplers.SampleResult[] refSubs = refSecond.getSubResults(); + org.apache.jmeter.samplers.SampleResult[] pluginSubs = pluginSecond.getSubResults(); + int refSubCount = refSubs != null ? refSubs.length : 0; + int pluginSubCount = pluginSubs != null ? pluginSubs.length : 0; + assertThat(pluginSubCount).isEqualTo(refSubCount); + if (refSubCount > 0) { + assertThat(pluginSubs[0].getResponseMessage()) + .isEqualTo(refSubs[0].getResponseMessage()); + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpClient4PluginParitySupport.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpClient4PluginParitySupport.java new file mode 100644 index 00000000..6117b0b7 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpClient4PluginParitySupport.java @@ -0,0 +1,107 @@ +package com.blazemeter.jmeter.http2.parity; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.blazemeter.jmeter.http2.core.HTTP2ClientProfileConfig; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import java.lang.reflect.Method; +import java.net.URL; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase; +import org.apache.jmeter.protocol.http.sampler.HTTPSamplerFactory; +import org.apache.jmeter.util.JMeterUtils; + +/** Runs the same sampler configuration through HttpClient4 and the BlazeMeter HTTP Jetty client. */ +public final class HttpClient4PluginParitySupport { + + private static final String HTTP_CLIENT4 = "HttpClient4"; + + private HttpClient4PluginParitySupport() { + } + + public static void forceHttp1ClientProfile() { + JMeterUtils.setProperty("httpJettyClient.enableHttp1", "true"); + JMeterUtils.setProperty("httpJettyClient.enableHttp2", "false"); + JMeterUtils.setProperty("httpJettyClient.enableHttp3", "false"); + JMeterUtils.setProperty("httpJettyClient.alpnEnabled", "false"); + } + + public static HTTP2JettyClient newHttp1PluginClient(String name) throws Exception { + forceHttp1ClientProfile(); + HTTP2JettyClient client = new HTTP2JettyClient(false, name, + HTTP2ClientProfileConfig.builder().enableHttp1(true).enableHttp2(false).enableHttp3(false) + .build()); + client.start(); + return client; + } + + public static void copyHttpSamplerConfig(HTTPSamplerBase from, HTTPSamplerBase to) { + to.setDomain(from.getDomain()); + to.setPort(from.getPort()); + to.setProtocol(from.getProtocol()); + to.setPath(from.getPath()); + to.setMethod(from.getMethod()); + to.setFollowRedirects(from.getFollowRedirects()); + to.setAutoRedirects(from.getAutoRedirects()); + to.setUseKeepAlive(from.getUseKeepAlive()); + to.setDoMultipart(from.getDoMultipart()); + to.setPostBodyRaw(from.getPostBodyRaw()); + to.setContentEncoding(from.getContentEncoding()); + to.setArguments(from.getArguments()); + to.setHTTPFiles(from.getHTTPFiles()); + to.setCookieManager(from.getCookieManager()); + to.setCacheManager(from.getCacheManager()); + to.setHeaderManager(from.getHeaderManager()); + to.setAuthManager(from.getAuthManager()); + to.setImageParser(from.isImageParser()); + } + + public static HTTPSampleResult sampleHttpClient4(HTTP2Sampler sampler, URL url) + throws Exception { + HTTPSamplerBase hc4 = HTTPSamplerFactory.newInstance(HTTP_CLIENT4); + copyHttpSamplerConfig(sampler, hc4); + Method sample = HTTPSamplerBase.class.getDeclaredMethod( + "sample", URL.class, String.class, boolean.class, int.class); + sample.setAccessible(true); + return (HTTPSampleResult) sample.invoke( + hc4, url, sampler.getMethod(), sampler.getFollowRedirects(), 1); + } + + public static HTTPSampleResult samplePlugin(HTTP2JettyClient client, HTTP2Sampler sampler, + URL url) throws Exception { + HTTPSampleResult shell = new HTTPSampleResult(); + shell.setURL(url); + shell.setHTTPMethod(sampler.getMethod()); + shell.setSampleLabel(sampler.getName()); + client.loadProperties(); + return client.sample(sampler, shell, false, 0); + } + + public static void assertCoreParity(HTTPSampleResult reference, HTTPSampleResult plugin, + String context) { + assertThat(plugin.isSuccessful()) + .as("%s success", context) + .isEqualTo(reference.isSuccessful()); + assertThat(plugin.getResponseCode()) + .as("%s response code", context) + .isEqualTo(reference.getResponseCode()); + assertThat(normalizeMessage(plugin.getResponseMessage())) + .as("%s response message", context) + .isEqualTo(normalizeMessage(reference.getResponseMessage())); + assertThat(plugin.getRedirectLocation()) + .as("%s redirect location", context) + .isEqualTo(reference.getRedirectLocation()); + } + + public static void assertResponseBodyParity(HTTPSampleResult reference, HTTPSampleResult plugin, + String context) { + assertThat(plugin.getResponseDataAsString()) + .as("%s response body", context) + .isEqualTo(reference.getResponseDataAsString()); + } + + private static String normalizeMessage(String message) { + return message == null ? "" : message.trim(); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCookieManagerParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCookieManagerParityTest.java new file mode 100644 index 00000000..0d4fe7a8 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpCookieManagerParityTest.java @@ -0,0 +1,85 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.core.ServerBuilder; +import com.blazemeter.jmeter.http2.core.ServerBuilder.TeardownableServer; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.net.URL; +import org.apache.jmeter.protocol.http.control.CookieManager; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.eclipse.jetty.server.ServerConnector; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +/** Port of cookie jar behaviour covered by Apache {@code TestHC4CookieManager} scenarios. */ +public class HttpCookieManagerParityTest extends HTTP2TestBase { + + private TeardownableServer server; + private HTTP2JettyClient client; + private int serverPort; + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + server = new ServerBuilder().withHTTP1().withSSL().buildServer(); + server.start(); + serverPort = ((ServerConnector) server.getConnectors()[0]).getLocalPort(); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("cookie-parity"); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (server != null) { + server.stop(); + } + } + + @Test + public void setCookieAndEchoMatchesHttpClient4() throws Exception { + CookieManager cookieManager = new CookieManager(); + cookieManager.testStarted(ServerBuilder.HOST_NAME); + + HTTP2Sampler setCookies = baseSampler(ServerBuilder.SERVER_PATH_SET_COOKIES); + setCookies.setCookieManager(cookieManager); + URL setUrl = new URL("https", ServerBuilder.HOST_NAME, serverPort, + ServerBuilder.SERVER_PATH_SET_COOKIES); + HttpClient4PluginParitySupport.assertCoreParity( + HttpClient4PluginParitySupport.sampleHttpClient4(setCookies, setUrl), + HttpClient4PluginParitySupport.samplePlugin(client, setCookies, setUrl), + "set cookies"); + + HTTP2Sampler useCookies = baseSampler(ServerBuilder.SERVER_PATH_USE_COOKIES); + useCookies.setCookieManager(cookieManager); + URL useUrl = new URL("https", ServerBuilder.HOST_NAME, serverPort, + ServerBuilder.SERVER_PATH_USE_COOKIES); + HTTPSampleResult reference = HttpClient4PluginParitySupport.sampleHttpClient4(useCookies, useUrl); + HTTPSampleResult plugin = HttpClient4PluginParitySupport.samplePlugin(client, useCookies, useUrl); + + HttpClient4PluginParitySupport.assertCoreParity(reference, plugin, "echo cookies"); + HttpClient4PluginParitySupport.assertResponseBodyParity(reference, plugin, "cookie body"); + } + + private HTTP2Sampler baseSampler(String path) { + HTTP2Sampler sampler = new HTTP2Sampler(); + sampler.setMethod(HTTPConstants.GET); + sampler.setProtocol("https"); + sampler.setDomain(ServerBuilder.HOST_NAME); + sampler.setPort(serverPort); + sampler.setPath(path); + sampler.setUseKeepAlive(true); + sampler.setFollowRedirects(true); + return sampler; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpDisableArgumentsParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpDisableArgumentsParityTest.java new file mode 100644 index 00000000..e40b5c2a --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpDisableArgumentsParityTest.java @@ -0,0 +1,80 @@ +package com.blazemeter.jmeter.http2.parity; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.net.URL; +import org.apache.jmeter.config.Arguments; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.util.HTTPArgument; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +/** + * Parity for skippable HTTP arguments (blank name / unresolved variable) as in JMeter 5.6.3. + * Per-argument {@code enabled} checkbox parity targets JMeter 5.7+ ({@code HttpSamplerDisableArgumentsTest}). + */ +public class HttpDisableArgumentsParityTest extends HTTP2TestBase { + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("disable-args-parity"); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void skippableArgumentsAreOmittedLikeHttpClient4() throws Exception { + HTTP2Sampler sampler = new HTTP2Sampler(); + sampler.setMethod(HTTPConstants.POST); + sampler.setProtocol("http"); + sampler.setDomain("localhost"); + sampler.setPort(mirrorPort); + sampler.setPath("/mirror-args"); + sampler.setUseKeepAlive(true); + + Arguments args = new Arguments(); + args.addArgument(new HTTPArgument("keep", "yes", false)); + args.addArgument(new HTTPArgument("", "blank-name", false)); + args.addArgument(new HTTPArgument("${optionalVar}", "unresolved", false)); + sampler.setArguments(args); + + URL url = new URL("http", "localhost", mirrorPort, "/mirror-args"); + HTTPSampleResult reference = HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + HTTPSampleResult plugin = HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + + HttpClient4PluginParitySupport.assertCoreParity(reference, plugin, "skippable args"); + assertThat(reference.getResponseDataAsString()).contains("keep=yes"); + assertThat(plugin.getResponseDataAsString()).contains("keep=yes"); + assertThat(reference.getResponseDataAsString()).doesNotContain("blank-name"); + assertThat(plugin.getResponseDataAsString()).doesNotContain("blank-name"); + assertThat(reference.getResponseDataAsString()).doesNotContain("unresolved"); + assertThat(plugin.getResponseDataAsString()).doesNotContain("unresolved"); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorFileUploadParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorFileUploadParityTest.java new file mode 100644 index 00000000..e4d837e9 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorFileUploadParityTest.java @@ -0,0 +1,120 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MirrorParityResult; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.io.File; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.Arrays; +import java.util.Collection; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.AfterClass; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** Port of Apache {@code testPostRequest_FileUpload} scenarios (items 0–2). */ +@RunWith(Parameterized.class) +public class HttpMirrorFileUploadParityTest extends HTTP2TestBase { + + private static final String TITLE = "title"; + private static final String DESCRIPTION = "description"; + private static final String ISO_8859_1 = "ISO-8859-1"; + private static final byte[] UPLOAD_BYTES = + "some foo content &?=01234+56789-|œ♪".getBytes(StandardCharsets.UTF_8); + + private static File uploadFile; + + @Parameterized.Parameter + public int item; + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @Parameterized.Parameters(name = "file-upload-item-{0}") + public static Collection data() { + return Arrays.asList(0, 1, 2); + } + + @BeforeClass + public static void setupClass() throws Exception { + JMeterTestUtils.setupJmeterEnv(); + uploadFile = Files.createTempFile("HttpMirrorFileUploadParityTest-", ".tmp").toFile(); + Files.write(uploadFile.toPath(), UPLOAD_BYTES); + uploadFile.deleteOnExit(); + } + + @AfterClass + public static void tearDownClass() { + if (uploadFile != null) { + uploadFile.delete(); + } + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("mirror-file-upload"); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void fileUploadEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = buildFileUploadSampler(item); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "file upload item " + item); + result.assertEchoContains("name=\"file1\"", "filename="); + result.assertMultipartFieldValuesMatch(TITLE); + result.assertMultipartFieldValuesMatch(DESCRIPTION); + result.assertEchoContains("some foo content"); + } + + private HTTP2Sampler buildFileUploadSampler(int test) { + String titleValue = "mytitle"; + String descriptionValue = "mydescription"; + String contentEncoding = ""; + + switch (test) { + case 0: + break; + case 1: + contentEncoding = ISO_8859_1; + break; + case 2: + contentEncoding = "UTF-8"; + titleValue = "mytitleœ₡ĕÅ"; + descriptionValue = "mydescriptionœ₡ĕÅ"; + break; + default: + throw new IllegalArgumentException("Unsupported file upload item: " + test); + } + + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setDoMultipart(true); + if (!contentEncoding.isEmpty()) { + sampler.setContentEncoding(contentEncoding); + } + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, false); + HttpMirrorParitySupport.addFileUpload(sampler, "file1", uploadFile, "text/plain"); + return sampler; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorItemisedParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorItemisedParityTest.java new file mode 100644 index 00000000..e24ce820 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorItemisedParityTest.java @@ -0,0 +1,235 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MirrorParityResult; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.util.HTTPArgument; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** + * Port of Apache {@code itemised_testPostRequest_UrlEncoded} and + * {@code itemised_testGetRequest_Parameters} (cases 0–7 / 0–5). + */ +@RunWith(Parameterized.class) +public class HttpMirrorItemisedParityTest extends HTTP2TestBase { + + private static final String TITLE = "title"; + private static final String DESCRIPTION = "description"; + private static final String ISO_8859_1 = "ISO-8859-1"; + + @Parameterized.Parameter(0) + public String mode; + + @Parameterized.Parameter(1) + public int item; + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @Parameterized.Parameters(name = "{0}-item-{1}") + public static Collection data() { + List rows = new ArrayList<>(); + for (int i = 0; i <= 7; i++) { + rows.add(new Object[] {"POST", i}); + } + for (int i = 0; i <= 5; i++) { + rows.add(new Object[] {"GET", i}); + } + return rows; + } + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("mirror-itemised"); + } + + @After + public void tearDown() throws Exception { + HttpMirrorParitySupport.clearMirrorVariables(); + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void mirrorEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = buildSampler(); + String context = mode + " mirror item " + item; + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity(client, sampler, context); + if ("GET".equals(mode)) { + result.assertRequestLineMatches(); + } else { + result.assertPostBodyMatches(); + } + } + + private HTTP2Sampler buildSampler() throws Exception { + if ("GET".equals(mode)) { + return buildGetParametersSampler(item); + } + return buildPostUrlEncodedSampler(item); + } + + private HTTP2Sampler buildGetParametersSampler(int test) throws Exception { + String titleValue = "mytitle"; + String descriptionValue = "mydescription"; + String contentEncoding = ""; + boolean alwaysEncoded = false; + + switch (test) { + case 0: + break; + case 1: + contentEncoding = ISO_8859_1; + titleValue = "mytitle1Œ"; + descriptionValue = "mydescription1Œ"; + break; + case 2: + contentEncoding = "UTF-8"; + titleValue = "mytitle2œ₡ĕÅ"; + descriptionValue = "mydescription2œ₡ĕÅ"; + break; + case 3: + contentEncoding = "UTF-8"; + titleValue = "mytitle3œ+₡ ĕ&yesÅ"; + descriptionValue = "mydescription3 œ ₡ ĕ Å"; + break; + case 4: + contentEncoding = "UTF-8"; + titleValue = "mytitle4%2F%3D"; + descriptionValue = "mydescription4+++%2F%5C"; + alwaysEncoded = true; + break; + case 5: + return buildGetWithVariablesSampler(); + default: + throw new IllegalArgumentException("Unsupported GET item: " + test); + } + + HTTP2Sampler sampler = baseGetSampler(contentEncoding); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, alwaysEncoded); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, alwaysEncoded); + return sampler; + } + + private HTTP2Sampler buildGetWithVariablesSampler() throws Exception { + HttpMirrorParitySupport.setupMirrorVariables(); + HTTP2Sampler sampler = baseGetSampler("UTF-8"); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, + "${title_prefix}mytitle5œ₡ĕÅ", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, + "mydescription5œ₡ĕÅ${description_suffix}", false); + HttpMirrorParitySupport.replaceSamplerVariables(sampler); + return sampler; + } + + private HTTP2Sampler buildPostUrlEncodedSampler(int test) throws Exception { + String titleValue = "mytitle"; + String descriptionValue = "mydescription"; + String contentEncoding = ""; + boolean alwaysEncoded = false; + + switch (test) { + case 0: + break; + case 1: + contentEncoding = ISO_8859_1; + break; + case 2: + contentEncoding = "UTF-8"; + titleValue = "mytitle2œ₡ĕÅ"; + descriptionValue = "mydescription2œ₡ĕÅ"; + break; + case 3: + contentEncoding = "UTF-8"; + titleValue = "mytitle3/="; + descriptionValue = "mydescription3 /\\"; + break; + case 4: + contentEncoding = "UTF-8"; + titleValue = "mytitle4%2F%3D"; + descriptionValue = "mydescription4+++%2F%5C"; + alwaysEncoded = true; + break; + case 5: + contentEncoding = "UTF-8"; + titleValue = "/wEPDwULLTE2MzM2OTA0NTYPZBYCAgMPZ/rA+8DZ2dnZ2dnZ2d/GNDar6OshPwdJc="; + descriptionValue = "mydescription5"; + break; + case 6: + contentEncoding = "UTF-8"; + titleValue = "%2FwEPDwULLTE2MzM2OTA0NTYPZBYCAgMPZ%2FrA%2B8DZ2dnZ2dnZ2d%2FGNDar6OshPwdJc%3D"; + descriptionValue = "mydescription6"; + return buildPostProxyStyleSampler(contentEncoding, titleValue, descriptionValue); + case 7: + return buildPostWithVariablesSampler(); + default: + throw new IllegalArgumentException("Unsupported POST item: " + test); + } + + HTTP2Sampler sampler = basePostSampler(contentEncoding); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, alwaysEncoded); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, alwaysEncoded); + return sampler; + } + + private HTTP2Sampler buildPostProxyStyleSampler(String contentEncoding, String titleValue, + String descriptionValue) { + HTTP2Sampler sampler = basePostSampler(contentEncoding); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, false); + ((HTTPArgument) sampler.getArguments().getArgument(0)).setAlwaysEncoded(false); + ((HTTPArgument) sampler.getArguments().getArgument(1)).setAlwaysEncoded(false); + return sampler; + } + + private HTTP2Sampler buildPostWithVariablesSampler() throws Exception { + HttpMirrorParitySupport.setupMirrorVariables(); + HTTP2Sampler sampler = basePostSampler("UTF-8"); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, + "${title_prefix}mytitle7œ₡ĕÅ", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, + "mydescription7œ₡ĕÅ${description_suffix}", false); + HttpMirrorParitySupport.replaceSamplerVariables(sampler); + return sampler; + } + + private HTTP2Sampler baseGetSampler(String contentEncoding) { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + if (!contentEncoding.isEmpty()) { + sampler.setContentEncoding(contentEncoding); + } + return sampler; + } + + private HTTP2Sampler basePostSampler(String contentEncoding) { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + if (!contentEncoding.isEmpty()) { + sampler.setContentEncoding(contentEncoding); + } + return sampler; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorMultipartParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorMultipartParityTest.java new file mode 100644 index 00000000..ef61f32a --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorMultipartParityTest.java @@ -0,0 +1,137 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MirrorParityResult; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.util.Arrays; +import java.util.Collection; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.apache.jmeter.threads.JMeterContextService; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** Port of Apache {@code testPostRequest_FormMultipart} scenarios (items 0–6). */ +@RunWith(Parameterized.class) +public class HttpMirrorMultipartParityTest extends HTTP2TestBase { + + private static final String TITLE = "title"; + private static final String DESCRIPTION = "description"; + private static final String ISO_8859_1 = "ISO-8859-1"; + + @Parameterized.Parameter + public int item; + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @Parameterized.Parameters(name = "multipart-item-{0}") + public static Collection data() { + return Arrays.asList(0, 1, 2, 3, 4, 5, 6); + } + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("mirror-multipart"); + } + + @After + public void tearDown() throws Exception { + HttpMirrorParitySupport.clearMirrorVariables(); + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void multipartEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = buildMultipartSampler(item); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "multipart item " + item); + result.assertEchoContains("Content-Transfer-Encoding: 8bit"); + result.assertMultipartFieldValuesMatch(TITLE); + result.assertMultipartFieldValuesMatch(DESCRIPTION); + } + + private HTTP2Sampler buildMultipartSampler(int test) throws Exception { + String titleValue = "mytitle"; + String descriptionValue = "mydescription"; + String contentEncoding = ""; + boolean alwaysEncoded = false; + + switch (test) { + case 0: + break; + case 1: + contentEncoding = ISO_8859_1; + break; + case 2: + contentEncoding = "UTF-8"; + titleValue = "mytitleœ₡ĕÅ"; + descriptionValue = "mydescriptionœ₡ĕÅ"; + break; + case 3: + contentEncoding = "UTF-8"; + titleValue = "mytitle/="; + descriptionValue = "mydescription /\\"; + break; + case 4: + contentEncoding = "UTF-8"; + titleValue = "mytitle%2F%3D"; + descriptionValue = "mydescription+++%2F%5C"; + alwaysEncoded = true; + break; + case 5: + contentEncoding = "UTF-8"; + titleValue = "/wEPDwULLTE2MzM2OTA0NTYPZBYCAgMPZ/rA+8DZ2dnZ2dnZ2d/GNDar6OshPwdJc="; + descriptionValue = "mydescription"; + break; + case 6: + return buildMultipartWithVariablesSampler(); + default: + throw new IllegalArgumentException("Unsupported multipart item: " + test); + } + + HTTP2Sampler sampler = baseMultipartSampler(contentEncoding); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, alwaysEncoded); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, alwaysEncoded); + return sampler; + } + + private HTTP2Sampler buildMultipartWithVariablesSampler() throws Exception { + HttpMirrorParitySupport.setupMirrorVariables(); + HTTP2Sampler sampler = baseMultipartSampler("UTF-8"); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, + "${title_prefix}mytitleœ₡ĕÅ", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, + "mydescriptionœ₡ĕÅ${description_suffix}", false); + HttpMirrorParitySupport.replaceSamplerVariables(sampler); + return sampler; + } + + private HTTP2Sampler baseMultipartSampler(String contentEncoding) { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setDoMultipart(true); + if (!contentEncoding.isEmpty()) { + sampler.setContentEncoding(contentEncoding); + } + return sampler; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParitySupport.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParitySupport.java new file mode 100644 index 00000000..995306c7 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParitySupport.java @@ -0,0 +1,280 @@ +package com.blazemeter.jmeter.http2.parity; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.ServerSocket; +import java.net.Socket; +import java.net.URL; +import java.net.URLDecoder; +import java.nio.charset.StandardCharsets; +import java.util.LinkedHashMap; +import java.util.Locale; +import java.util.Map; +import java.util.concurrent.TimeUnit; +import org.apache.jmeter.config.Arguments; +import org.apache.jmeter.engine.util.ValueReplacer; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.testelement.TestPlan; +import org.apache.jmeter.threads.JMeterContextService; +import org.apache.jmeter.threads.JMeterVariables; +import org.apache.jmeter.util.JMeterUtils; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.util.HTTPArgument; +import org.apache.jmeter.protocol.http.util.HTTPFileArg; + +/** Helpers for {@link HttpMirrorParityTest} (Apache mirror-server parity). */ +public final class HttpMirrorParitySupport { + + /** Default path used by {@code TestHTTPSamplersAgainstHttpMirrorServer}. */ + public static final String MIRROR_PATH = "/test/somescript.jsp"; + + private HttpMirrorParitySupport() { + } + + public static int findFreePort() throws IOException { + try (ServerSocket socket = new ServerSocket(0)) { + return socket.getLocalPort(); + } + } + + /** + * {@link HttpMirrorServer} extends {@link Thread} and binds its listening socket inside + * {@code run()}, so {@code start()} returns before the socket is actually bound - a race that + * only shows up under load (e.g. running the full suite), not when a test runs alone. Poll the + * port until it accepts a real connection before handing the server back. + */ + public static HttpMirrorServer startMirrorServer(int port) throws Exception { + HttpMirrorServer server = new HttpMirrorServer(port, 10, 10); + server.start(); + awaitListening(port); + return server; + } + + private static void awaitListening(int port) throws InterruptedException { + long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(5); + IOException lastFailure = null; + while (System.nanoTime() < deadline) { + try (Socket probe = new Socket()) { + probe.connect(new InetSocketAddress("localhost", port), 200); + return; + } catch (IOException e) { + lastFailure = e; + Thread.sleep(20); + } + } + throw new IllegalStateException( + "HttpMirrorServer did not start listening on port " + port + " within 5s", lastFailure); + } + + public static HTTP2Sampler baseMirrorSampler(int mirrorPort, String method) { + HTTP2Sampler sampler = new HTTP2Sampler(); + sampler.setMethod(method); + sampler.setProtocol("http"); + sampler.setDomain("localhost"); + sampler.setPort(mirrorPort); + sampler.setPath(MIRROR_PATH); + sampler.setUseKeepAlive(true); + sampler.setFollowRedirects(true); + return sampler; + } + + public static void addFormPair(HTTP2Sampler sampler, String name, String value, + boolean alwaysEncoded) { + if (alwaysEncoded) { + sampler.addEncodedArgument(name, value); + return; + } + Arguments args = sampler.getArguments(); + if (args == null) { + args = new Arguments(); + sampler.setArguments(args); + } + args.addArgument(new HTTPArgument(name, value, false)); + } + + public static void addRawBodyValues(HTTP2Sampler sampler, String... values) { + Arguments args = new Arguments(); + for (String value : values) { + HTTPArgument arg = new HTTPArgument("", value, false); + arg.setAlwaysEncoded(false); + args.addArgument(arg); + } + sampler.setArguments(args); + sampler.setPostBodyRaw(true); + } + + public static void addFileUpload(HTTP2Sampler sampler, String fieldName, java.io.File file, + String mimeType) { + sampler.setDoMultipart(true); + sampler.setHTTPFiles(new HTTPFileArg[] { + new HTTPFileArg(file.getAbsolutePath(), fieldName, mimeType) + }); + } + + public static MirrorParityResult runMirrorParity(HTTP2JettyClient client, HTTP2Sampler sampler, + String context) throws Exception { + URL url = sampler.getUrl(); + HTTPSampleResult reference = HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + HTTPSampleResult plugin = HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + HttpClient4PluginParitySupport.assertCoreParity(reference, plugin, context); + return new MirrorParityResult(reference, plugin); + } + + public static String requestLine(HTTPSampleResult mirrorEcho) { + String echo = mirrorEcho.getResponseDataAsString().replace("\r\n", "\n"); + int newline = echo.indexOf('\n'); + return newline >= 0 ? echo.substring(0, newline) : echo; + } + + public static void assertSameRequestLine(HTTPSampleResult reference, HTTPSampleResult plugin) { + assertThat(requestLine(plugin)).isEqualTo(requestLine(reference)); + } + + public static void assertEchoContainsBoth(HTTPSampleResult reference, HTTPSampleResult plugin, + String... fragments) { + for (String fragment : fragments) { + assertThat(reference.getResponseDataAsString()).as("reference echo").contains(fragment); + assertThat(plugin.getResponseDataAsString()).as("plugin echo").contains(fragment); + } + } + + public static void assertPostBodyContainsBoth(HTTPSampleResult reference, HTTPSampleResult plugin, + String bodyFragment) { + String refBody = bodyAfterHeaders(reference.getResponseDataAsString()); + String pluginBody = bodyAfterHeaders(plugin.getResponseDataAsString()); + assertThat(refBody).contains(bodyFragment); + assertThat(pluginBody).contains(bodyFragment); + } + + static String bodyAfterHeaders(String echo) { + String normalized = echo.replace("\r\n", "\n"); + int divider = normalized.indexOf("\n\n"); + return divider >= 0 ? normalized.substring(divider + 2) : normalized; + } + + static String multipartFieldValue(String multipartBody, String fieldName) { + String normalized = multipartBody.replace("\r\n", "\n"); + String marker = "name=\"" + fieldName + "\""; + int fieldStart = normalized.indexOf(marker); + if (fieldStart < 0) { + return ""; + } + int transferEncoding = normalized.indexOf("Content-Transfer-Encoding: 8bit", fieldStart); + int searchFrom = transferEncoding >= 0 ? transferEncoding : fieldStart; + int lineEnd = normalized.indexOf('\n', searchFrom); + if (lineEnd < 0) { + return ""; + } + int valueStart = lineEnd + 1; + if (valueStart < normalized.length() && normalized.charAt(valueStart) == '\n') { + valueStart++; + } + int valueEnd = normalized.indexOf("\n--", valueStart); + String value = valueEnd >= 0 ? normalized.substring(valueStart, valueEnd) + : normalized.substring(valueStart); + return value.replace("\n", "").trim(); + } + + static Map parseUrlEncodedBody(String body) { + Map fields = new LinkedHashMap<>(); + if (body == null || body.isEmpty()) { + return fields; + } + for (String pair : body.split("&")) { + int eq = pair.indexOf('='); + if (eq <= 0) { + continue; + } + String name = urlDecodeComponent(pair.substring(0, eq)); + String value = urlDecodeComponent(pair.substring(eq + 1)); + fields.put(name, value); + } + return fields; + } + + private static String urlDecodeComponent(String component) { + return URLDecoder.decode(component, StandardCharsets.UTF_8); + } + + public static void setupMirrorVariables() { + JMeterUtils.setLocale(Locale.ENGLISH); + JMeterVariables vars = new JMeterVariables(); + vars.put("title_prefix", "a testÅ"); + vars.put("description_suffix", "the_end"); + JMeterContextService.getContext().setVariables(vars); + JMeterContextService.getContext().setSamplingStarted(true); + } + + public static void clearMirrorVariables() { + JMeterContextService.getContext().setVariables(null); + JMeterContextService.getContext().setSamplingStarted(false); + } + + public static void replaceSamplerVariables(HTTP2Sampler sampler) throws Exception { + ValueReplacer replacer = new ValueReplacer(); + replacer.setUserDefinedVariables(new TestPlan().getUserDefinedVariables()); + replacer.replaceValues(sampler); + } + + public static void assertRawPostBodiesMatch(HTTPSampleResult reference, HTTPSampleResult plugin) { + String refBody = bodyAfterHeaders(reference.getResponseDataAsString()); + String pluginBody = bodyAfterHeaders(plugin.getResponseDataAsString()); + assertThat(pluginBody).as("raw POST body echo").isEqualTo(refBody); + } + + public static void assertUrlEncodedBodiesMatch(HTTPSampleResult reference, + HTTPSampleResult plugin) { + String refBody = bodyAfterHeaders(reference.getResponseDataAsString()); + String pluginBody = bodyAfterHeaders(plugin.getResponseDataAsString()); + assertThat(parseUrlEncodedBody(pluginBody)) + .as("urlencoded POST body") + .isEqualTo(parseUrlEncodedBody(refBody)); + } + + public static void assertMultipartFieldValuesMatch(HTTPSampleResult reference, + HTTPSampleResult plugin, String fieldName) { + String refBody = bodyAfterHeaders(reference.getResponseDataAsString()); + String pluginBody = bodyAfterHeaders(plugin.getResponseDataAsString()); + assertThat(multipartFieldValue(pluginBody, fieldName)) + .as("multipart field %s", fieldName) + .isEqualTo(multipartFieldValue(refBody, fieldName)); + } + + public static URL mirrorUrl(int port) throws Exception { + return new URL("http", "localhost", port, MIRROR_PATH); + } + + public static URL mirrorUrl(int port, String path) throws Exception { + return new URL("http", "localhost", port, path); + } + + public record MirrorParityResult(HTTPSampleResult reference, HTTPSampleResult plugin) { + public void assertRequestLineMatches() { + assertSameRequestLine(reference, plugin); + } + + public void assertEchoContains(String... fragments) { + assertEchoContainsBoth(reference, plugin, fragments); + } + + public void assertPostBodyContains(String fragment) { + assertPostBodyContainsBoth(reference, plugin, fragment); + } + + public void assertMultipartFieldValuesMatch(String fieldName) { + HttpMirrorParitySupport.assertMultipartFieldValuesMatch(reference, plugin, fieldName); + } + + public void assertPostBodyMatches() { + HttpMirrorParitySupport.assertUrlEncodedBodiesMatch(reference, plugin); + } + + public void assertRawPostBodyMatches() { + HttpMirrorParitySupport.assertRawPostBodiesMatch(reference, plugin); + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParityTest.java new file mode 100644 index 00000000..98e6b922 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorParityTest.java @@ -0,0 +1,219 @@ +package com.blazemeter.jmeter.http2.parity; + +import static com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MIRROR_PATH; + +import com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MirrorParityResult; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.io.File; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.AfterClass; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; + +/** + * Port of scenarios from Apache JMeter {@code TestHTTPSamplersAgainstHttpMirrorServer}. + * Compares HttpClient4 vs BlazeMeter HTTP on {@link HttpMirrorServer} echo (HTTP/1.1). + */ +public class HttpMirrorParityTest extends HTTP2TestBase { + + private static final String TITLE = "title"; + private static final String DESCRIPTION = "description"; + private static final byte[] UPLOAD_BYTES = + "some foo content &?=01234+56789-|œ♪".getBytes(StandardCharsets.UTF_8); + + private static File uploadFile; + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @BeforeClass + public static void setupClass() throws Exception { + JMeterTestUtils.setupJmeterEnv(); + uploadFile = Files.createTempFile("HttpMirrorParityTest-", ".tmp").toFile(); + Files.write(uploadFile.toPath(), UPLOAD_BYTES); + uploadFile.deleteOnExit(); + } + + @AfterClass + public static void tearDownClass() { + if (uploadFile != null) { + uploadFile.delete(); + } + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("mirror-parity"); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void getWithIso88591EncodingMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + sampler.setContentEncoding("ISO-8859-1"); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "GET ISO-8859-1"); + result.assertRequestLineMatches(); + } + + @Test + public void getRequestEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "GET mirror"); + result.assertRequestLineMatches(); + result.assertEchoContains("GET " + MIRROR_PATH + " HTTP/1.1"); + } + + @Test + public void getWithQueryParametersMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription", false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "GET query params"); + result.assertRequestLineMatches(); + result.assertEchoContains("title=mytitle", "description=mydescription"); + } + + @Test + public void getWithUtf8SpecialCharactersMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + sampler.setContentEncoding("UTF-8"); + String titleValue = "mytitle3œ+♪ ĕ&yesÅ"; + String descriptionValue = "mydescription3 œ ♪ ĕ Å"; + HttpMirrorParitySupport.addFormPair(sampler, TITLE, titleValue, false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, descriptionValue, false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "GET UTF-8 params"); + result.assertRequestLineMatches(); + } + + @Test + public void getWithAlwaysEncodedParametersMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.GET); + sampler.setContentEncoding("UTF-8"); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle4%2F%3D", true); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription4+++%2F%5C", true); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "GET encoded params"); + result.assertRequestLineMatches(); + result.assertEchoContains("title=mytitle4%2F%3D", "description=mydescription4+++%2F%5C"); + } + + @Test + public void postUrlEncodedEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription", false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST urlencoded"); + result.assertPostBodyContains("title=mytitle"); + result.assertPostBodyContains("description=mydescription"); + } + + @Test + public void postUrlEncodedUtf8MatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setContentEncoding("UTF-8"); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitleœ♪ĕÅ", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescriptionœ♪ĕÅ", false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST UTF-8"); + result.assertPostBodyContains("title="); + result.assertPostBodyContains("description="); + } + + @Test + public void postMultipartEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setDoMultipart(true); + HttpMirrorParitySupport.addFormPair(sampler, "name1", "value1", false); + + sampler.setPath(MIRROR_PATH + "?name0=value0"); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST multipart"); + result.assertEchoContains("Content-Transfer-Encoding: 8bit", "name=\"name1\"", "value1"); + } + + @Test + public void postMultipartUtf8SpecialCharsMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setContentEncoding("UTF-8"); + sampler.setDoMultipart(true); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle/=", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription /\\", false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST multipart UTF-8"); + result.assertEchoContains("name=\"" + TITLE + "\"", "name=\"" + DESCRIPTION + "\""); + result.assertMultipartFieldValuesMatch(TITLE); + result.assertMultipartFieldValuesMatch(DESCRIPTION); + } + + @Test + public void postFileUploadMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setDoMultipart(true); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription", false); + HttpMirrorParitySupport.addFileUpload(sampler, "file1", uploadFile, "text/plain"); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST file upload"); + result.assertEchoContains("name=\"file1\"", "filename=", "some foo content"); + } + + @Test + public void postBodyRawFromParameterValuesMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + HttpMirrorParitySupport.addRawBodyValues(sampler, "mytitle", "mydescription"); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "POST body raw"); + result.assertPostBodyContains("mytitlemydescription"); + } + + @Test + public void putWithFormBodyMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.PUT); + HttpMirrorParitySupport.addFormPair(sampler, TITLE, "mytitle", false); + HttpMirrorParitySupport.addFormPair(sampler, DESCRIPTION, "mydescription", false); + ((org.apache.jmeter.protocol.http.util.HTTPArgument) sampler.getArguments().getArgument(0)) + .setAlwaysEncoded(false); + ((org.apache.jmeter.protocol.http.util.HTTPArgument) sampler.getArguments().getArgument(1)) + .setAlwaysEncoded(false); + + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "PUT form"); + result.assertRequestLineMatches(); + result.assertPostBodyContains("title=mytitle"); + result.assertPostBodyContains("description=mydescription"); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorRawBodyParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorRawBodyParityTest.java new file mode 100644 index 00000000..3198f4f0 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpMirrorRawBodyParityTest.java @@ -0,0 +1,158 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.parity.HttpMirrorParitySupport.MirrorParityResult; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.util.Arrays; +import java.util.Collection; +import org.apache.jmeter.protocol.http.control.HttpMirrorServer; +import org.apache.jmeter.protocol.http.util.HTTPArgument; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** Port of Apache {@code testPostRequest_BodyFromParameterValues} scenarios (items 0–9). */ +@RunWith(Parameterized.class) +public class HttpMirrorRawBodyParityTest extends HTTP2TestBase { + + private static final String ISO_8859_1 = "ISO-8859-1"; + + @Parameterized.Parameter + public int item; + + private HttpMirrorServer mirrorServer; + private HTTP2JettyClient client; + private int mirrorPort; + + @Parameterized.Parameters(name = "raw-body-item-{0}") + public static Collection data() { + return Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7, 8, 9); + } + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + mirrorPort = HttpMirrorParitySupport.findFreePort(); + mirrorServer = HttpMirrorParitySupport.startMirrorServer(mirrorPort); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("mirror-raw-body"); + } + + @After + public void tearDown() throws Exception { + HttpMirrorParitySupport.clearMirrorVariables(); + if (client != null) { + client.stop(); + } + if (mirrorServer != null) { + mirrorServer.stopServer(); + } + } + + @Test + public void rawBodyEchoMatchesHttpClient4() throws Exception { + HTTP2Sampler sampler = buildRawBodySampler(item); + MirrorParityResult result = HttpMirrorParitySupport.runMirrorParity( + client, sampler, "raw body item " + item); + result.assertRawPostBodyMatches(); + } + + private HTTP2Sampler buildRawBodySampler(int test) throws Exception { + String titleValue = "mytitle"; + String descriptionValue = "mydescription"; + String contentEncoding = ""; + boolean alwaysEncoded = false; + + switch (test) { + case 0: + break; + case 1: + contentEncoding = ISO_8859_1; + break; + case 2: + contentEncoding = "UTF-8"; + titleValue = "mytitleœ₡ĕÅ"; + descriptionValue = "mydescriptionœ₡ĕÅ"; + break; + case 3: + contentEncoding = "UTF-8"; + titleValue = "mytitle/="; + descriptionValue = "mydescription /\\"; + break; + case 4: + contentEncoding = "UTF-8"; + titleValue = "mytitle/="; + descriptionValue = "mydescription /\\"; + alwaysEncoded = true; + break; + case 5: + contentEncoding = "UTF-8"; + titleValue = "mytitle%2F%3D"; + descriptionValue = "mydescription+++%2F%5C"; + break; + case 6: + contentEncoding = "UTF-8"; + titleValue = "mytitle%2F%3D"; + descriptionValue = "mydescription+++%2F%5C"; + alwaysEncoded = true; + break; + case 7: + contentEncoding = "UTF-8"; + titleValue = "/wEPDwULLTE2MzM2OTA0NTYPZBYCAgMPZ/rA+8DZ2dnZ2dnZ2d/GNDar6OshPwdJc="; + descriptionValue = "mydescription"; + break; + case 8: + contentEncoding = "UTF-8"; + titleValue = "mytitle++"; + descriptionValue = "mydescription+"; + break; + case 9: + return buildRawBodyWithVariablesSampler(); + default: + throw new IllegalArgumentException("Unsupported raw body item: " + test); + } + + HTTP2Sampler sampler = baseRawBodySampler(contentEncoding); + addRawValue(sampler, titleValue, alwaysEncoded); + addRawValue(sampler, descriptionValue, alwaysEncoded); + return sampler; + } + + private HTTP2Sampler buildRawBodyWithVariablesSampler() throws Exception { + HttpMirrorParitySupport.setupMirrorVariables(); + HTTP2Sampler sampler = baseRawBodySampler("UTF-8"); + addRawValue(sampler, "${title_prefix}mytitleœ₡ĕÅ", false); + addRawValue(sampler, "mydescriptionœ₡ĕÅ${description_suffix}", false); + HttpMirrorParitySupport.replaceSamplerVariables(sampler); + return sampler; + } + + private static void addRawValue(HTTP2Sampler sampler, String value, boolean alwaysEncoded) { + org.apache.jmeter.config.Arguments args = sampler.getArguments(); + if (args == null) { + args = new org.apache.jmeter.config.Arguments(); + sampler.setArguments(args); + } + HTTPArgument arg = new HTTPArgument("", value, alwaysEncoded); + arg.setAlwaysEncoded(alwaysEncoded); + args.addArgument(arg); + } + + private HTTP2Sampler baseRawBodySampler(String contentEncoding) { + HTTP2Sampler sampler = HttpMirrorParitySupport.baseMirrorSampler(mirrorPort, HTTPConstants.POST); + sampler.setPostBodyRaw(true); + if (!contentEncoding.isEmpty()) { + sampler.setContentEncoding(contentEncoding); + } + return sampler; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsFollowParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsFollowParityTest.java new file mode 100644 index 00000000..61d063db --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsFollowParityTest.java @@ -0,0 +1,113 @@ +package com.blazemeter.jmeter.http2.parity; + +import static org.assertj.core.api.Assertions.assertThat; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.net.URL; +import java.util.Arrays; +import java.util.Collection; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** + * Redirect parity with automatic follow enabled ({@code followRedirects} + {@code autoRedirects}). + * Jetty and HttpClient4 both follow at the client layer when {@code autoRedirects} is true. + */ +@RunWith(Parameterized.class) +public class HttpRedirectsFollowParityTest extends HTTP2TestBase { + + @Parameterized.Parameter(0) + public int redirectCode; + + @Parameterized.Parameter(1) + public String method; + + @Parameterized.Parameter(2) + public boolean shouldFollowToTarget; + + private ParityRedirectServer redirectServer; + private HTTP2JettyClient client; + private HTTP2Sampler sampler; + + @Parameterized.Parameters(name = "follow-{0}-{1}-toTarget={2}") + public static Collection data() { + return Arrays.asList( + row(301, "GET", true), + row(301, "HEAD", true), + row(302, "GET", true), + row(302, "HEAD", true), + row(303, "GET", true), + row(303, "HEAD", true), + row(307, "GET", true), + row(307, "HEAD", true), + row(307, "POST", true), + row(308, "GET", true), + row(308, "HEAD", true)); + } + + private static Object[] row(int code, String method, boolean shouldFollow) { + return new Object[] {code, method, shouldFollow}; + } + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + redirectServer = new ParityRedirectServer(); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("redirect-follow-parity"); + sampler = new HTTP2Sampler(); + sampler.setFollowRedirects(true); + sampler.setAutoRedirects(true); + sampler.setUseKeepAlive(true); + sampler.setMethod(method); + sampler.setProtocol("http"); + sampler.setDomain("localhost"); + sampler.setPort(redirectServer.getPort()); + sampler.setPath("/some-location?status=" + redirectCode); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (redirectServer != null) { + redirectServer.close(); + } + } + + @Test + public void pluginMatchesHttpClient4WhenFollowingRedirects() throws Exception { + URL url = new URL(redirectServer.url("/some-location?status=" + redirectCode)); + HTTPSampleResult reference = HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + HTTPSampleResult plugin = HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + + HttpClient4PluginParitySupport.assertCoreParity(reference, plugin, + "follow " + redirectCode + " " + method); + + if (shouldFollowToTarget) { + assertThat(reference.getResponseCode()).isEqualTo("200"); + assertThat(plugin.getResponseCode()).isEqualTo("200"); + assertThat(reference.getURL().toString()).endsWith("/target"); + assertThat(plugin.getURL().toString()).endsWith("/target"); + if ("GET".equals(method)) { + HttpClient4PluginParitySupport.assertResponseBodyParity(reference, plugin, + "follow GET body"); + } + } else { + assertThat(reference.getResponseCode()).isEqualTo(String.valueOf(redirectCode)); + assertThat(plugin.getResponseCode()).isEqualTo(String.valueOf(redirectCode)); + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsParityTest.java b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsParityTest.java new file mode 100644 index 00000000..1a51c970 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/HttpRedirectsParityTest.java @@ -0,0 +1,144 @@ +package com.blazemeter.jmeter.http2.parity; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.core.HTTP2JettyClient; +import com.blazemeter.jmeter.http2.sampler.HTTP2Sampler; +import com.blazemeter.jmeter.http2.sampler.JMeterTestUtils; +import java.net.URL; +import java.util.Arrays; +import java.util.Collection; +import org.apache.jmeter.protocol.http.sampler.HTTPSampleResult; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.junit.After; +import org.junit.Before; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** + * Port of Apache JMeter {@code TestRedirects}: redirect location and status with follow disabled. + */ +@RunWith(Parameterized.class) +public class HttpRedirectsParityTest extends HTTP2TestBase { + + @Parameterized.Parameter(0) + public int redirectCode; + + @Parameterized.Parameter(1) + public String method; + + @Parameterized.Parameter(2) + public boolean shouldExposeRedirectLocation; + + /** + * Whether THIS plugin exposes the redirect location - differs from + * {@link #shouldExposeRedirectLocation} (which reference HttpClient4/JMeter 5.6.3 does) only + * for 307 + a non-GET/HEAD method: JMeter 5.6.3's {@code HTTPSampleResult.isRedirect()} treats + * that combination as "not a redirect" and never exposes the location, a bug fixed upstream in + * apache/jmeter PR #6658 (see {@code Rfc9110Redirects}). This plugin ports that fix by default, + * so it deliberately diverges from the (buggy) reference here; set + * {@code -Dblazemeter.http.legacyRedirectMethodHandling=true} to match the reference exactly. + */ + @Parameterized.Parameter(3) + public boolean pluginShouldExposeRedirectLocation; + + private ParityRedirectServer redirectServer; + private HTTP2JettyClient client; + private HTTP2Sampler sampler; + + @Parameterized.Parameters(name = "{0}-{1}-redirect={2}") + public static Collection data() { + return Arrays.asList( + row(301, "GET", true), + row(301, "HEAD", true), + row(301, "POST", true), + row(301, "PUT", true), + row(301, "DELETE", true), + row(302, "GET", true), + row(302, "HEAD", true), + row(302, "POST", true), + row(303, "GET", true), + row(303, "POST", true), + row(307, "GET", true), + row(307, "HEAD", true), + row(307, "POST", false, true), + row(307, "PUT", false, true), + row(308, "GET", true), + row(308, "HEAD", true), + row(308, "POST", true), + row(300, "GET", false), + row(304, "GET", false), + row(305, "GET", false), + row(306, "GET", false)); + } + + private static Object[] row(int code, String method, boolean shouldRedirect) { + return row(code, method, shouldRedirect, shouldRedirect); + } + + private static Object[] row(int code, String method, boolean shouldRedirect, + boolean pluginShouldRedirect) { + return new Object[] {code, method, shouldRedirect, pluginShouldRedirect}; + } + + @BeforeClass + public static void setupClass() { + JMeterTestUtils.setupJmeterEnv(); + } + + @Before + public void setUp() throws Exception { + redirectServer = new ParityRedirectServer(); + client = HttpClient4PluginParitySupport.newHttp1PluginClient("redirect-parity"); + sampler = new HTTP2Sampler(); + sampler.setFollowRedirects(false); + sampler.setAutoRedirects(false); + sampler.setUseKeepAlive(true); + sampler.setMethod(method); + sampler.setProtocol("http"); + sampler.setDomain("localhost"); + sampler.setPort(redirectServer.getPort()); + sampler.setPath("/some-location?status=" + redirectCode); + } + + @After + public void tearDown() throws Exception { + if (client != null) { + client.stop(); + } + if (redirectServer != null) { + redirectServer.close(); + } + } + + @Test + public void pluginMatchesHttpClient4RedirectSemantics() throws Exception { + URL url = new URL(redirectServer.url("/some-location?status=" + redirectCode)); + HTTPSampleResult reference = HttpClient4PluginParitySupport.sampleHttpClient4(sampler, url); + HTTPSampleResult plugin = HttpClient4PluginParitySupport.samplePlugin(client, sampler, url); + + // Not assertCoreParity(): its blanket redirectLocation check doesn't allow the deliberate + // 307+non-GET/HEAD divergence below, so success/response code/message are asserted directly. + String context = redirectCode + " " + method; + org.assertj.core.api.Assertions.assertThat(plugin.isSuccessful()) + .as("%s success", context).isEqualTo(reference.isSuccessful()); + org.assertj.core.api.Assertions.assertThat(plugin.getResponseCode()) + .as("%s response code", context).isEqualTo(reference.getResponseCode()); + + String expectedReferenceLocation = shouldExposeRedirectLocation + ? redirectServer.url("/target") + : null; + String expectedPluginLocation = pluginShouldExposeRedirectLocation + ? redirectServer.url("/target") + : null; + org.assertj.core.api.Assertions.assertThat(reference.getRedirectLocation()) + .isEqualTo(expectedReferenceLocation); + org.assertj.core.api.Assertions.assertThat(plugin.getRedirectLocation()) + .isEqualTo(expectedPluginLocation); + org.assertj.core.api.Assertions.assertThat(reference.getResponseCode()) + .isEqualTo(String.valueOf(redirectCode)); + org.assertj.core.api.Assertions.assertThat(plugin.getResponseCode()) + .isEqualTo(String.valueOf(redirectCode)); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/parity/ParityRedirectServer.java b/src/test/java/com/blazemeter/jmeter/http2/parity/ParityRedirectServer.java new file mode 100644 index 00000000..7bcadc9c --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/parity/ParityRedirectServer.java @@ -0,0 +1,75 @@ +package com.blazemeter.jmeter.http2.parity; + +import jakarta.servlet.http.HttpServlet; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import java.io.IOException; +import org.apache.jmeter.protocol.http.util.HTTPConstants; +import org.eclipse.jetty.ee10.servlet.ServletContextHandler; +import org.eclipse.jetty.ee10.servlet.ServletHolder; +import org.eclipse.jetty.server.Server; +import org.eclipse.jetty.server.ServerConnector; + +/** Minimal HTTP/1 server that returns configurable redirect responses for parity tests. */ +public final class ParityRedirectServer implements AutoCloseable { + + private final Server server; + private final int port; + + public ParityRedirectServer() throws Exception { + server = new Server(); + ServerConnector connector = new ServerConnector(server); + connector.setPort(0); + server.addConnector(connector); + ServletContextHandler context = new ServletContextHandler(); + context.addServlet(new ServletHolder(new RedirectServlet()), "/*"); + server.setHandler(context); + server.start(); + port = connector.getLocalPort(); + } + + public int getPort() { + return port; + } + + public String url(String path) { + String normalized = path.startsWith("/") ? path : "/" + path; + return "http://localhost:" + port + normalized; + } + + @Override + public void close() throws Exception { + server.stop(); + } + + private final class RedirectServlet extends HttpServlet { + + @Override + protected void service(HttpServletRequest req, HttpServletResponse resp) throws IOException { + String path = req.getRequestURI(); + if ("/target".equals(path)) { + resp.setStatus(HttpServletResponse.SC_OK); + resp.getWriter().write("ok"); + return; + } + if ("/some-location".equals(path)) { + int status = parseStatus(req.getParameter("status"), HttpServletResponse.SC_FOUND); + resp.setStatus(status); + resp.setHeader(HTTPConstants.HEADER_LOCATION, url("/target")); + return; + } + resp.setStatus(HttpServletResponse.SC_NOT_FOUND); + } + + private int parseStatus(String raw, int defaultStatus) { + if (raw == null || raw.isBlank()) { + return defaultStatus; + } + try { + return Integer.parseInt(raw.trim()); + } catch (NumberFormatException e) { + return defaultStatus; + } + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/HttpsampleResultComparator.java b/src/test/java/com/blazemeter/jmeter/http2/regression/HttpsampleResultComparator.java new file mode 100644 index 00000000..71f14270 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/HttpsampleResultComparator.java @@ -0,0 +1,237 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +/** Semantic diff of two JMeter sample trees (ignores timings and volatile headers). */ +public final class HttpsampleResultComparator { + + private static final Set IGNORED_RESPONSE_HEADER_NAMES = new HashSet<>(Arrays.asList( + "date", "server", "connection", "transfer-encoding", "keep-alive", + "content-length", "alt-svc", "via", "x-firefox-spdy", "x-powered-by", + "cf-ray", "age", "cf-cache-status", "accept-ranges", "set-cookie")); + + /** Headers compared between ref/plugin runs; others vary by timing or CDN edge. */ + private static final Set COMPARED_RESPONSE_HEADER_NAMES = new HashSet<>(Arrays.asList( + "content-encoding", "content-type", "location")); + + private static final Pattern EMBEDDED_FILE_LABEL_INDEX = + Pattern.compile("^(?:file:.+|.+)-(\\d+)$"); + + private HttpsampleResultComparator() { + } + + public static ComparisonResult compare(List reference, + List actual) { + return compare(reference, actual, false); + } + + public static ComparisonResult compare(List reference, + List actual, boolean tolerateExternalServiceDrift) { + List differences = new ArrayList<>(); + compareLists(reference, actual, "", differences, tolerateExternalServiceDrift); + return new ComparisonResult(differences.isEmpty(), differences); + } + + private static void compareLists(List reference, List actual, + String path, List differences, boolean tolerateExternalServiceDrift) { + if (reference.size() != actual.size()) { + differences.add(path + "sample count: expected " + reference.size() + + " but was " + actual.size()); + int limit = Math.min(reference.size(), actual.size()); + for (int i = 0; i < limit; i++) { + compareSample(reference.get(i), actual.get(i), path + "[" + i + "].", differences, + tolerateExternalServiceDrift); + } + return; + } + for (int i = 0; i < reference.size(); i++) { + compareSample(reference.get(i), actual.get(i), path + "[" + i + "].", differences, + tolerateExternalServiceDrift); + } + } + + private static void compareSample(SampleRecord expected, SampleRecord actual, String path, + List differences, boolean tolerateExternalServiceDrift) { + if (!labelsEquivalent(expected.getLabel(), actual.getLabel())) { + differences.add(path + "label: expected '" + expected.getLabel() + + "' but was '" + actual.getLabel() + "'"); + } + if (tolerateExternalServiceDrift && isExternalServiceDrift(expected, actual)) { + return; + } + if (expected.isSuccessful() != actual.isSuccessful()) { + differences.add(path + "success: expected " + expected.isSuccessful() + + " but was " + actual.isSuccessful() + + " (code=" + actual.getResponseCode() + " msg=" + actual.getResponseMessage() + ")"); + } + if (!normalizeResponseCode(expected.getResponseCode()) + .equals(normalizeResponseCode(actual.getResponseCode()))) { + differences.add(path + "responseCode: expected " + expected.getResponseCode() + + " but was " + actual.getResponseCode()); + } + if (!normalizeMessage(expected.getResponseMessage()) + .equals(normalizeMessage(actual.getResponseMessage()))) { + differences.add(path + "responseMessage: expected '" + expected.getResponseMessage() + + "' but was '" + actual.getResponseMessage() + "'"); + } + if (!normalizeBody(expected.getResponseData()).equals(normalizeBody(actual.getResponseData()))) { + differences.add(path + "responseData differs for label '" + expected.getLabel() + "'"); + } + Map expectedHeaders = normalizeHeaders(expected.getResponseHeaders()); + Map actualHeaders = normalizeHeaders(actual.getResponseHeaders()); + for (Map.Entry entry : expectedHeaders.entrySet()) { + String key = entry.getKey(); + if (!actualHeaders.containsKey(key)) { + differences.add(path + "missing response header '" + key + "'"); + } else if (!entry.getValue().equals(actualHeaders.get(key))) { + differences.add(path + "response header '" + key + "': expected '" + entry.getValue() + + "' but was '" + actualHeaders.get(key) + "'"); + } + } + compareLists(expected.getChildren(), actual.getChildren(), path + "children.", + differences, tolerateExternalServiceDrift); + } + + /** One run saw a transient upstream 5xx while the other succeeded (or vice versa). */ + private static boolean isExternalServiceDrift(SampleRecord expected, SampleRecord actual) { + int expectedCode = parseHttpStatus(expected.getResponseCode()); + int actualCode = parseHttpStatus(actual.getResponseCode()); + if (expectedCode < 0 || actualCode < 0) { + return false; + } + boolean expectedTransient = expectedCode >= 502 && expectedCode <= 504; + boolean actualTransient = actualCode >= 502 && actualCode <= 504; + boolean expectedOk = expectedCode >= 200 && expectedCode < 300; + boolean actualOk = actualCode >= 200 && actualCode < 300; + boolean expectedRedirect = expectedCode >= 301 && expectedCode < 400; + boolean actualRedirect = actualCode >= 301 && actualCode < 400; + return (expectedTransient && (actualOk || actualRedirect)) + || (actualTransient && (expectedOk || expectedRedirect)); + } + + private static int parseHttpStatus(String code) { + if (code == null || code.isBlank()) { + return -1; + } + try { + return Integer.parseInt(code.trim()); + } catch (NumberFormatException e) { + return -1; + } + } + + private static String normalizeResponseCode(String code) { + if (code == null) { + return ""; + } + if (code.contains("ConnectException")) { + return "connect-failure"; + } + return code; + } + + private static String normalizeMessage(String message) { + if (message == null) { + return ""; + } + String normalized = message.trim(); + if (normalized.contains("Connection refused")) { + return "connection-refused"; + } + // Jetty often invents the default reason phrase when the status line has none; + // HttpClient4 may keep an empty response message for the same redirect. + if (isDefaultHttpReasonPhrase(normalized)) { + return ""; + } + return normalized; + } + + private static boolean isDefaultHttpReasonPhrase(String message) { + return "Found".equals(message) + || "Moved Permanently".equals(message) + || "Moved Temporarily".equals(message) + || "See Other".equals(message) + || "Temporary Redirect".equals(message) + || "Permanent Redirect".equals(message) + || "OK".equals(message); + } + + private static String normalizeBody(String body) { + if (body == null) { + return ""; + } + // HC4 may echo explicit Connection: keep-alive; Jetty omits it on HTTP/1.1 wire. + // Keep-alive parity is asserted on sample request headers, not mirror body. + String normalized = body.replace("\r\n", "\n"); + normalized = normalized.replaceAll("(?im)^Connection:\\s*keep-alive\\s*\\n?", ""); + return normalized.trim(); + } + + private static Map normalizeHeaders(String raw) { + Map headers = new HashMap<>(); + if (raw == null || raw.trim().isEmpty()) { + return headers; + } + String[] lines = raw.replace("\r\n", "\n").split("\n"); + for (String line : lines) { + int colon = line.indexOf(':'); + if (colon <= 0) { + continue; + } + String name = line.substring(0, colon).trim().toLowerCase(Locale.ROOT); + if (IGNORED_RESPONSE_HEADER_NAMES.contains(name) + || !COMPARED_RESPONSE_HEADER_NAMES.contains(name)) { + continue; + } + String value = line.substring(colon + 1).trim(); + headers.put(name, value); + } + return headers; + } + + /** + * HttpClient4 labels file embedded resources as {@code file:-N}; the plugin may use the + * parent sampler name. The numeric suffix is the stable identity for parity checks. + */ + private static boolean labelsEquivalent(String expected, String actual) { + if (expected.equals(actual)) { + return true; + } + Matcher expectedIndex = EMBEDDED_FILE_LABEL_INDEX.matcher(expected); + Matcher actualIndex = EMBEDDED_FILE_LABEL_INDEX.matcher(actual); + return expectedIndex.matches() && actualIndex.matches() + && expectedIndex.group(1).equals(actualIndex.group(1)); + } + + public static final class ComparisonResult { + private final boolean equal; + private final List differences; + + ComparisonResult(boolean equal, List differences) { + this.equal = equal; + this.differences = differences; + } + + public boolean isEqual() { + return equal; + } + + public List getDifferences() { + return differences; + } + + public String formattedDiff() { + return differences.stream().collect(Collectors.joining(System.lineSeparator())); + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterDistribution.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterDistribution.java new file mode 100644 index 00000000..b446c2ce --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterDistribution.java @@ -0,0 +1,183 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.net.URI; +import java.net.URL; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.nio.file.StandardCopyOption; +import java.util.Comparator; +import java.util.stream.Stream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; +import org.apache.commons.io.FileUtils; + +/** + * Locates or downloads an Apache JMeter binary distribution for forked regression runs. + */ +public final class JmeterDistribution { + + private static final String VERSION = System.getProperty( + "jmeter.regression.version", "5.6.3"); + private static final String DOWNLOAD_URL = "https://archive.apache.org/dist/jmeter/binaries/" + + "apache-jmeter-" + VERSION + ".zip"; + + private final Path homeDir; + private final Path binDir; + private final File jmeterJar; + + private JmeterDistribution(Path homeDir) { + this.homeDir = homeDir; + this.binDir = homeDir.resolve("bin"); + this.jmeterJar = binDir.resolve("ApacheJMeter.jar").toFile(); + } + + public static JmeterDistribution resolve() throws IOException { + String explicit = System.getProperty("jmeter.home"); + if (explicit != null && !explicit.trim().isEmpty()) { + Path home = Paths.get(explicit.trim()); + JmeterDistribution dist = new JmeterDistribution(home); + if (!dist.jmeterJar.isFile()) { + throw new IOException("jmeter.home does not contain bin/ApacheJMeter.jar: " + home); + } + return dist; + } + + Path baseDir = Paths.get("target", "jmeter-dist-" + VERSION).toAbsolutePath(); + Path home = provisionHome(baseDir); + JmeterDistribution dist = new JmeterDistribution(home); + if (!dist.jmeterJar.isFile()) { + throw new IOException("Failed to provision JMeter " + VERSION + " under " + baseDir); + } + return dist; + } + + public Path getHomeDir() { + return homeDir; + } + + public Path getBinDir() { + return binDir; + } + + public File getJmeterJar() { + return jmeterJar; + } + + public void installPluginJar(File pluginJar) throws IOException { + Path extDir = homeDir.resolve("lib").resolve("ext"); + Files.createDirectories(extDir); + Files.copy(pluginJar.toPath(), extDir.resolve(pluginJar.getName()), + StandardCopyOption.REPLACE_EXISTING); + } + + public void removePluginJars() throws IOException { + Path extDir = homeDir.resolve("lib").resolve("ext"); + if (!Files.isDirectory(extDir)) { + return; + } + try (Stream stream = Files.list(extDir)) { + stream.filter(path -> path.getFileName().toString().startsWith("jmeter-bzm-http")) + .forEach(path -> { + try { + Files.deleteIfExists(path); + } catch (IOException e) { + throw new IllegalStateException("Could not delete " + path, e); + } + }); + } + } + + private static Path provisionHome(Path baseDir) throws IOException { + Path existing = findJmeterHomeIfPresent(baseDir); + if (existing != null) { + return existing; + } + Files.createDirectories(baseDir); + Path zipPath = baseDir.resolve("apache-jmeter-" + VERSION + ".zip"); + if (!Files.isRegularFile(zipPath)) { + download(zipPath); + } + Path extractRoot = baseDir.resolve("extract"); + if (Files.exists(extractRoot)) { + FileUtils.deleteDirectory(extractRoot.toFile()); + } + unzip(zipPath, extractRoot); + Path extractedHome = findJmeterHome(extractRoot); + return extractedHome; + } + + private static Path findJmeterHomeIfPresent(Path baseDir) throws IOException { + if (!Files.isDirectory(baseDir)) { + return null; + } + Path fromExtract = baseDir.resolve("extract"); + if (Files.isDirectory(fromExtract)) { + try { + return findJmeterHome(fromExtract); + } catch (IOException ignored) { + return null; + } + } + try { + return findJmeterHome(baseDir); + } catch (IOException ignored) { + return null; + } + } + + private static void download(Path destination) throws IOException { + URL url = URI.create(DOWNLOAD_URL).toURL(); + try (InputStream in = url.openStream()) { + Files.copy(in, destination, StandardCopyOption.REPLACE_EXISTING); + } + } + + private static void unzip(Path zipFile, Path destination) throws IOException { + Files.createDirectories(destination); + try (ZipInputStream zis = new ZipInputStream(Files.newInputStream(zipFile))) { + ZipEntry entry; + while ((entry = zis.getNextEntry()) != null) { + Path outPath = destination.resolve(entry.getName()).normalize(); + if (!outPath.startsWith(destination)) { + throw new IOException("Zip entry outside target dir: " + entry.getName()); + } + if (entry.isDirectory()) { + Files.createDirectories(outPath); + } else { + Files.createDirectories(outPath.getParent()); + Files.copy(zis, outPath, StandardCopyOption.REPLACE_EXISTING); + } + zis.closeEntry(); + } + } + } + + private static Path findJmeterHome(Path extractRoot) throws IOException { + try (Stream stream = Files.walk(extractRoot, 3)) { + return stream + .filter(path -> path.getFileName().toString().equals("ApacheJMeter.jar")) + .map(path -> path.getParent().getParent()) + .findFirst() + .orElseThrow(() -> new IOException("ApacheJMeter.jar not found under " + extractRoot)); + } + } + + static void deleteRecursively(Path path) throws IOException { + if (!Files.exists(path)) { + return; + } + try (Stream walk = Files.walk(path)) { + walk.sorted(Comparator.reverseOrder()).forEach(p -> { + try { + Files.deleteIfExists(p); + } catch (IOException e) { + throw new IllegalStateException(e); + } + }); + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterHttpRegressionIntegrationTest.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterHttpRegressionIntegrationTest.java new file mode 100644 index 00000000..1ab36a20 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterHttpRegressionIntegrationTest.java @@ -0,0 +1,176 @@ +package com.blazemeter.jmeter.http2.regression; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assume.assumeTrue; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.sampler.JmxBlazeMeterHttpMigrator; +import java.io.File; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import org.apache.jorphan.collections.HashTree; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; + +/** + * Runs Apache JMeter {@code bin/testfiles} plans against HttpClient4 (reference) and migrated + * {@code HTTP2Sampler} (plugin), comparing sample results semantically. + */ +@RunWith(Parameterized.class) +public class JmeterHttpRegressionIntegrationTest extends HTTP2TestBase { + + private static final String REF_IMPL = "HttpClient4"; + private static final String PLUGIN_IMPL = "BzmHttp"; + + private static JmeterDistribution distribution; + private static JmeterRegressionRunner runner; + private static Path workRoot; + private static File pluginJar; + private static boolean http3Enabled; + + private final String testName; + private final RegressionProtocolProfile protocolProfile; + + public JmeterHttpRegressionIntegrationTest(String testName, + RegressionProtocolProfile protocolProfile) { + this.testName = testName; + this.protocolProfile = protocolProfile; + } + + @Parameterized.Parameters(name = "{0}-{1}") + public static List data() { + List rows = new ArrayList<>(); + RegressionProtocolProfile protocolFilter = resolveProtocolFilter(); + for (String test : JmeterRegressionSupport.configuredTestNames()) { + String baseName = JmeterRegressionSupport.testBaseName(test); + for (RegressionProtocolProfile profile : RegressionProtocolProfile.values()) { + if (profile == RegressionProtocolProfile.HTTP3 && !Boolean.getBoolean("it.http3")) { + continue; + } + if (profile != RegressionProtocolProfile.HTTP1_ONLY + && JmeterRegressionSupport.isHttp1OnlyPlan(baseName)) { + continue; + } + if (protocolFilter != null && profile != protocolFilter) { + continue; + } + rows.add(new Object[] {test, profile}); + } + } + return rows; + } + + private static RegressionProtocolProfile resolveProtocolFilter() { + String value = System.getProperty("jmeter.regression.protocol"); + if (value == null || value.isBlank()) { + return null; + } + return RegressionProtocolProfile.fromSystemProperty(); + } + + @BeforeClass + public static void setUpClass() throws Exception { + assumeTrue("Set -Djmeter.regression=true to run JMeter regression tests", + Boolean.getBoolean("jmeter.regression")); + http3Enabled = Boolean.getBoolean("it.http3"); + distribution = JmeterDistribution.resolve(); + workRoot = Path.of("target", "jmeter-regression-work"); + Files.createDirectories(workRoot); + runner = new JmeterRegressionRunner(distribution, workRoot); + pluginJar = JmeterRegressionSupport.resolvePluginJar(); + } + + @AfterClass + public static void tearDownClass() throws Exception { + if (distribution != null) { + distribution.removePluginJars(); + } + } + + @Test + public void referenceAndPluginProduceEquivalentSamples() throws Exception { + if (protocolProfile == RegressionProtocolProfile.HTTP3 && !http3Enabled) { + return; + } + + String baseName = JmeterRegressionSupport.testBaseName(testName); + assumeTrue("Plan requires -Djmeter.regression.enableSlowChars=true (CPS throttling not in Jetty yet)", + !JmeterRegressionSupport.isOptionalUnsupportedPlan(baseName)); + Path caseDir = workRoot.resolve(baseName + "-" + protocolProfile.getId()); + Files.createDirectories(caseDir); + + File sourceJmx = JmeterRegressionSupport.copyResourceToWorkDir(testName + ".jmx", caseDir); + File batchProps = JmeterRegressionSupport.copyResourceToWorkDir( + "jmeter-batch.properties", caseDir); + File log4jXml = JmeterRegressionSupport.copyResourceToWorkDir("log4j2-batch.xml", caseDir); + + String migratedBaseName = baseName + "_bzm"; + JmeterRegressionSupport.cleanupResultArtifacts(distribution, baseName, REF_IMPL, PLUGIN_IMPL); + JmeterRegressionSupport.cleanupPlanCollectorArtifacts(distribution, baseName, migratedBaseName); + JmeterRegressionSupport.stageFixtures(baseName, distribution, caseDir); + + Map refArgs = JmeterRegressionRunner.referenceHttpClient4Args(); + distribution.removePluginJars(); + JmeterRegressionRunner.RunResult refRun = runner.run( + baseName + "-ref-" + protocolProfile.getId(), + sourceJmx, + batchProps, + log4jXml, + refArgs); + + assertEquals("Reference JMeter run failed, see " + refRun.getLogFile(), + 0, refRun.getExitCode()); + assertTrue("Reference log has errors: " + refRun.getLogFile(), + !JmeterRegressionRunner.logHasErrors(refRun.getLogFile())); + + File refSamples = refRun.resolveSampleFile(); + assertNotNull("Reference sample file not found for " + baseName, refSamples); + assertTrue("Reference sample file missing: " + refSamples, refSamples.isFile()); + File archivedRef = JmeterRegressionSupport.archiveResultXml( + refSamples, caseDir, "reference-samples.xml"); + List referenceSamples = JtlSampleLoader.load(archivedRef); + + JmeterRegressionSupport.cleanupPlanCollectorArtifacts(distribution, baseName, migratedBaseName); + + File migratedJmx = caseDir.resolve(migratedBaseName + ".jmx").toFile(); + HashTree tree = JmxBlazeMeterHttpMigrator.loadTree(sourceJmx); + int replaced = JmxBlazeMeterHttpMigrator.migrateTree(tree); + assertTrue("Expected HTTP samplers to migrate in " + testName, replaced > 0); + JmxBlazeMeterHttpMigrator.saveTree(tree, migratedJmx); + + Map pluginArgs = JmeterRegressionRunner.pluginArgs(protocolProfile); + pluginArgs.put("jmeter.httpsampler", PLUGIN_IMPL); + + distribution.installPluginJar(pluginJar); + JmeterRegressionRunner.RunResult pluginRun = runner.run( + baseName + "-plugin-" + protocolProfile.getId(), + migratedJmx, + batchProps, + log4jXml, + pluginArgs); + + assertEquals("Plugin JMeter run failed, see " + pluginRun.getLogFile(), + 0, pluginRun.getExitCode()); + assertTrue("Plugin log has errors: " + pluginRun.getLogFile(), + !JmeterRegressionRunner.logHasErrors(pluginRun.getLogFile())); + + File pluginSamplesFile = pluginRun.resolveSampleFile(); + assertNotNull("Plugin sample file not found for " + baseName, pluginSamplesFile); + assertTrue("Plugin sample file missing: " + pluginSamplesFile, pluginSamplesFile.isFile()); + List pluginSamples = JtlSampleLoader.load(pluginSamplesFile); + + HttpsampleResultComparator.ComparisonResult comparison = + HttpsampleResultComparator.compare(referenceSamples, pluginSamples, + JmeterRegressionSupport.toleratesExternalServiceDrift(baseName)); + assertTrue("Sample mismatch for " + testName + " [" + protocolProfile.getId() + "]: " + + comparison.formattedDiff(), comparison.isEqual()); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionRunner.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionRunner.java new file mode 100644 index 00000000..121695e5 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionRunner.java @@ -0,0 +1,193 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.TimeUnit; + +/** Forks JMeter in non-GUI mode using the same flags as Apache's BatchTest task. */ +public final class JmeterRegressionRunner { + + private static final long DEFAULT_TIMEOUT_MINUTES = Long.parseLong( + System.getProperty("jmeter.regression.timeoutMinutes", "10")); + + private final JmeterDistribution distribution; + private final Path workDir; + + public JmeterRegressionRunner(JmeterDistribution distribution, Path workDir) throws IOException { + this.distribution = distribution; + this.workDir = workDir; + Files.createDirectories(workDir); + } + + public RunResult run(String runId, File jmxFile, File batchProperties, File log4jXml, + Map jmeterArgs) throws IOException, InterruptedException { + Path runDir = workDir.resolve(runId); + Files.createDirectories(runDir); + + File logFile = runDir.resolve("jmeter.log").toFile(); + File jtlFile = runDir.resolve("results.jtl").toFile(); + File errFile = runDir.resolve("jmeter.err").toFile(); + Files.deleteIfExists(jtlFile.toPath()); + Files.deleteIfExists(logFile.toPath()); + Files.deleteIfExists(errFile.toPath()); + + List command = new ArrayList<>(); + command.add(resolveJavaExecutable()); + command.add("-Xms128m"); + command.add("-Xmx512m"); + command.add("-Djava.awt.headless=true"); + command.add("-Duser.language=en"); + command.add("-Duser.region=en"); + command.add("-Duser.country=US"); + command.add("-cp"); + command.add(distribution.getJmeterJar().getAbsolutePath()); + command.add("org.apache.jmeter.NewDriver"); + command.add("-p"); + command.add(distribution.getBinDir().resolve("jmeter.properties").toString()); + command.add("-q"); + command.add(batchProperties.getAbsolutePath()); + command.add("-n"); + command.add("-t"); + command.add(jmxFile.getAbsolutePath()); + command.add("-i"); + command.add(log4jXml.getAbsolutePath()); + command.add("-j"); + command.add(logFile.getAbsolutePath()); + command.add("-l"); + command.add(jtlFile.getAbsolutePath()); + command.add("-Jmodule=Module"); + command.add("-Gmodule=Module"); + + if (jmeterArgs != null) { + for (Map.Entry entry : jmeterArgs.entrySet()) { + command.add("-J" + entry.getKey() + "=" + entry.getValue()); + } + } + + ProcessBuilder builder = new ProcessBuilder(command); + builder.directory(distribution.getBinDir().toFile()); + builder.redirectError(errFile); + Process process = builder.start(); + boolean finished = process.waitFor(DEFAULT_TIMEOUT_MINUTES, TimeUnit.MINUTES); + if (!finished) { + process.destroyForcibly(); + throw new IOException("JMeter timed out after " + DEFAULT_TIMEOUT_MINUTES + " minutes"); + } + + int exitCode = process.exitValue(); + File xmlResult = findXmlResult(runDir, jmxFile); + return new RunResult(runId, exitCode, logFile, jtlFile, xmlResult, errFile); + } + + private File findXmlResult(Path runDir, File jmxFile) { + String baseName = jmxFile.getName(); + if (baseName.endsWith(".jmx")) { + baseName = baseName.substring(0, baseName.length() - 4); + } + File inRunDir = runDir.resolve(baseName + ".xml").toFile(); + if (inRunDir.isFile()) { + return inRunDir; + } + File inBin = distribution.getBinDir().resolve(baseName + ".xml").toFile(); + if (inBin.isFile()) { + return inBin; + } + File jtl = runDir.resolve("results.jtl").toFile(); + return jtl.isFile() ? jtl : null; + } + + static String resolveJavaExecutable() { + String javaHome = System.getProperty("java.home"); + Path javaBin = Path.of(javaHome, "bin", isWindows() ? "java.exe" : "java"); + return javaBin.toString(); + } + + static boolean isWindows() { + return System.getProperty("os.name", "").toLowerCase().contains("win"); + } + + public static Map referenceHttpClient4Args() { + Map args = new LinkedHashMap<>(); + args.put("jmeter.httpsampler", "HttpClient4"); + return args; + } + + public static Map pluginArgs(RegressionProtocolProfile profile) { + Map args = new LinkedHashMap<>(); + args.putAll(profile.jmeterProperties()); + return args; + } + + public static boolean logHasErrors(File logFile) throws IOException { + if (logFile == null || !logFile.isFile() || logFile.length() == 0L) { + return false; + } + String content = new String(Files.readAllBytes(logFile.toPath()), StandardCharsets.UTF_8); + return content.contains("ERROR o.a.j.JMeter:") + || content.contains("AssertionFailedError"); + } + + public static final class RunResult { + private final String runId; + private final int exitCode; + private final File logFile; + private final File jtlFile; + private final File xmlResultFile; + private final File errFile; + + RunResult(String runId, int exitCode, File logFile, File jtlFile, File xmlResultFile, + File errFile) { + this.runId = runId; + this.exitCode = exitCode; + this.logFile = logFile; + this.jtlFile = jtlFile; + this.xmlResultFile = xmlResultFile; + this.errFile = errFile; + } + + public String getRunId() { + return runId; + } + + public int getExitCode() { + return exitCode; + } + + public File getLogFile() { + return logFile; + } + + public File getJtlFile() { + return jtlFile; + } + + public File getXmlResultFile() { + return xmlResultFile; + } + + public File getErrFile() { + return errFile; + } + + /** + * Per-run {@code -l results.jtl} is preferred over plan {@code ResultCollector} XML in + * {@code bin/}, which is shared and append-only across runs. + */ + public File resolveSampleFile() { + if (jtlFile != null && jtlFile.isFile() && jtlFile.length() > 0L) { + return jtlFile; + } + if (xmlResultFile != null && xmlResultFile.isFile()) { + return xmlResultFile; + } + return jtlFile; + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionSupport.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionSupport.java new file mode 100644 index 00000000..4aea285f --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JmeterRegressionSupport.java @@ -0,0 +1,258 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.StandardCopyOption; +import java.util.Arrays; +import java.util.List; +import java.util.Locale; +import java.util.Set; +import java.util.stream.Collectors; + +/** Shared helpers for JMeter HTTP regression suites. */ +public final class JmeterRegressionSupport { + + public static final String RESOURCE_ROOT = "jmeter-regression/5.6.3"; + + private JmeterRegressionSupport() { + } + + public static final String CORE_TESTS = + "TEST_HTTP,ResponseDecompression,TestHeaderManager,TestCookieManager"; + + /** Extended: HTTPS, HTML embedded parser, digest/basic auth (see docs/jmeter-regression.md). */ + public static final String EXTENDED_TESTS = + "HTMLParserTestFile_2,TEST_HTTPS,Http4ImplDigestAuth,Http4ImplPreemptiveBasicAuth"; + + /** + * HTTP-focused plans that call real third-party hosts (not in default CI; opt-in only). + * BUG_62847/Bug54685 are JMeter core-only plans, not HTTP sampler tests, so they're excluded. + */ + public static final String EXTERNAL_TESTS = "TestKeepAlive,TestRedirectionPolicies"; + + /** Optional; requires HttpClient4 CPS throttling not yet implemented in the Jetty client. */ + public static final String EXTERNAL_OPTIONAL_TESTS = "SlowCharsFeature"; + + /** + * Plans that call third-party hosts; ref/plugin runs are sequential so 5xx vs 2xx drift is + * environmental, not a sampler parity gap. + */ + private static final Set EXTERNAL_SERVICE_DRIFT_TESTS = Set.of( + "Http4ImplDigestAuth", + "Http4ImplPreemptiveBasicAuth", + "TestKeepAlive", + "TestRedirectionPolicies", + "SlowCharsFeature"); + + public static boolean isOptionalUnsupportedPlan(String testBaseName) { + return "SlowCharsFeature".equals(testBaseName) + && !Boolean.getBoolean("jmeter.regression.enableSlowChars"); + } + + private static final String DEFAULT_REGRESSION_TESTS = CORE_TESTS; + + /** Plans that only make sense against HttpClient4 / HTTP/1.1 (keep-alive, Connection: close). */ + private static final Set HTTP1_ONLY_PLANS = Set.of("TestKeepAlive"); + + public static boolean isHttp1OnlyPlan(String testBaseName) { + return HTTP1_ONLY_PLANS.contains(testBaseName); + } + + public static boolean toleratesExternalServiceDrift(String testBaseName) { + if (Boolean.getBoolean("jmeter.regression.tolerateExternalServiceDrift")) { + return true; + } + return EXTERNAL_SERVICE_DRIFT_TESTS.contains(testBaseName); + } + + public static List configuredTestNames() { + String raw = System.getProperty("jmeter.regression.tests"); + if (raw == null || raw.isBlank()) { + raw = resolveTestsForTier(System.getProperty("jmeter.regression.tier")); + } + if (raw == null || raw.isBlank()) { + raw = DEFAULT_REGRESSION_TESTS; + } + return Arrays.stream(raw.split(",")) + .map(String::trim) + .filter(s -> !s.isEmpty()) + .collect(Collectors.toList()); + } + + public static Path regressionResourceRoot() { + return Path.of("src", "test", "resources", RESOURCE_ROOT); + } + + public static File copyResourceToWorkDir(String resourceName, Path workDir) throws IOException { + String resourcePath = RESOURCE_ROOT + "/" + resourceName; + try (InputStream in = JmeterRegressionSupport.class.getClassLoader() + .getResourceAsStream(resourcePath)) { + if (in == null) { + Path fallback = regressionResourceRoot().resolve(resourceName); + if (Files.isRegularFile(fallback)) { + Files.createDirectories(workDir); + Path target = workDir.resolve(resourceName); + Files.copy(fallback, target, StandardCopyOption.REPLACE_EXISTING); + return target.toFile(); + } + throw new IOException("Regression resource not found: " + resourcePath); + } + Files.createDirectories(workDir); + Path target = workDir.resolve(resourceName); + Files.copy(in, target, StandardCopyOption.REPLACE_EXISTING); + return target.toFile(); + } + } + + public static File resolveResultXml(JmeterDistribution distribution, Path workDir, + String testBaseName, String implementationSuffix) { + String suffixed = testBaseName + "_" + implementationSuffix + ".xml"; + File inBin = distribution.getBinDir().resolve(suffixed).toFile(); + if (inBin.isFile()) { + return inBin; + } + File inWork = workDir.resolve(suffixed).toFile(); + if (inWork.isFile()) { + return inWork; + } + String fixed = testBaseName + ".xml"; + File fixedBin = distribution.getBinDir().resolve(fixed).toFile(); + if (fixedBin.isFile()) { + return fixedBin; + } + File fixedWork = workDir.resolve(fixed).toFile(); + if (fixedWork.isFile()) { + return fixedWork; + } + return null; + } + + public static File archiveResultXml(File source, Path workDir, String archiveName) + throws IOException { + if (source == null || !source.isFile()) { + return null; + } + Files.createDirectories(workDir); + Path target = workDir.resolve(archiveName); + Files.copy(source.toPath(), target, StandardCopyOption.REPLACE_EXISTING); + return target.toFile(); + } + + public static void cleanupResultArtifacts(JmeterDistribution distribution, String testBaseName, + String... suffixes) throws IOException { + cleanupPlanCollectorArtifacts(distribution, testBaseName); + for (String suffix : suffixes) { + deleteIfExists(distribution.getBinDir().resolve(testBaseName + "_" + suffix + ".xml")); + deleteIfExists(distribution.getBinDir().resolve(testBaseName + "_" + suffix + ".csv")); + deleteIfExists(distribution.getBinDir().resolve(testBaseName + ".jtl")); + deleteIfExists(distribution.getBinDir().resolve(testBaseName + ".log")); + } + } + + /** + * Deletes {@code ResultCollector} outputs under JMeter {@code bin/}. Those files are opened in + * append mode and must be removed before each forked run to avoid duplicated samples. + */ + public static void cleanupPlanCollectorArtifacts(JmeterDistribution distribution, + String... planBaseNames) throws IOException { + Path bin = distribution.getBinDir(); + for (String baseName : planBaseNames) { + if (baseName == null || baseName.isEmpty()) { + continue; + } + deleteIfExists(bin.resolve(baseName + ".xml")); + deleteIfExists(bin.resolve(baseName + ".csv")); + } + } + + private static void deleteIfExists(Path path) throws IOException { + Files.deleteIfExists(path); + } + + /** Locates the plugin jar built by {@code mvn package} under {@code target/}. */ + public static File resolvePluginJar() throws IOException { + try (var stream = Files.list(Path.of("target"))) { + return stream + .filter(p -> p.getFileName().toString().startsWith("jmeter-bzm-http2") + && p.getFileName().toString().endsWith(".jar") + && !p.getFileName().toString().contains("original")) + .map(Path::toFile) + .findFirst() + .orElseThrow(() -> new IOException( + "Plugin jar not found under target/. Run mvn package first.")); + } + } + + public static String testBaseName(String testName) { + if (testName.endsWith(".jmx")) { + return testName.substring(0, testName.length() - 4); + } + return testName; + } + + private static String resolveTestsForTier(String tier) { + if (tier == null || tier.isBlank()) { + return null; + } + return switch (tier.trim().toLowerCase(Locale.ROOT)) { + case "core" -> CORE_TESTS; + case "extended" -> EXTENDED_TESTS; + case "external", "optional" -> EXTERNAL_TESTS; + case "all" -> CORE_TESTS + "," + EXTENDED_TESTS + "," + EXTERNAL_TESTS; + default -> null; + }; + } + + /** Copies plan-specific fixtures into JMeter {@code bin/} (and optionally the case work dir). */ + public static void stageFixtures(String baseName, JmeterDistribution distribution, Path caseDir) + throws IOException { + if ("TEST_HTTP".equals(baseName)) { + stageTestHttpFixtures(distribution, caseDir); + } else if ("HTMLParserTestFile_2".equals(baseName)) { + stageHtmlParserFixtures(distribution, caseDir); + } + } + + /** Copies fixture files required by {@code TEST_HTTP.jmx} into JMeter {@code bin/} and {@code caseDir}. */ + public static void stageTestHttpFixtures(JmeterDistribution distribution, Path caseDir) + throws IOException { + for (String fixture : new String[] {"user.properties", "TEST_GET.jmx"}) { + copyResourceToWorkDir(fixture, distribution.getBinDir()); + if (caseDir != null) { + copyResourceToWorkDir(fixture, caseDir); + } + } + } + + /** Copies {@code testfiles/} tree required by {@code HTMLParserTestFile_2.jmx}. */ + public static void stageHtmlParserFixtures(JmeterDistribution distribution, Path caseDir) + throws IOException { + Path sourceRoot = regressionResourceRoot().resolve("testfiles"); + if (!Files.isDirectory(sourceRoot)) { + throw new IOException("HTML parser fixtures missing: " + sourceRoot); + } + copyResourceTree(sourceRoot, distribution.getBinDir().resolve("testfiles")); + if (caseDir != null) { + copyResourceTree(sourceRoot, caseDir.resolve("testfiles")); + } + } + + private static void copyResourceTree(Path sourceRoot, Path targetRoot) throws IOException { + Files.createDirectories(targetRoot); + try (var stream = Files.walk(sourceRoot)) { + for (Path source : stream.toList()) { + Path relative = sourceRoot.relativize(source); + Path target = targetRoot.resolve(relative); + if (Files.isDirectory(source)) { + Files.createDirectories(target); + } else { + Files.createDirectories(target.getParent()); + Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING); + } + } + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JmxBlazeMeterHttpMigratorCliIntegrationTest.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JmxBlazeMeterHttpMigratorCliIntegrationTest.java new file mode 100644 index 00000000..5a1a3948 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JmxBlazeMeterHttpMigratorCliIntegrationTest.java @@ -0,0 +1,108 @@ +package com.blazemeter.jmeter.http2.regression; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertTrue; +import static org.junit.Assume.assumeTrue; + +import com.blazemeter.jmeter.http2.HTTP2TestBase; +import com.blazemeter.jmeter.http2.sampler.JmxBlazeMeterHttpMigrator; +import com.blazemeter.jmeter.http2.sampler.JmxBlazeMeterHttpMigratorCli; +import java.io.File; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.TimeUnit; +import org.apache.jorphan.collections.HashTree; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +/** + * Forks {@link JmxBlazeMeterHttpMigratorCli} as a real OS process against a downloaded JMeter + * distribution, with this plugin's jar dropped into {@code lib/ext/} exactly as end users would + * run it: {@code JMeterCliEnvironment} must self-locate {@code JMETER_HOME} from that layout + * alone, with no {@code JMETER_HOME} env var or {@code -Djmeter.home} set. + */ +public class JmxBlazeMeterHttpMigratorCliIntegrationTest extends HTTP2TestBase { + + private static final long TIMEOUT_MINUTES = 2; + + private static JmeterDistribution distribution; + + @BeforeClass + public static void setUpClass() throws Exception { + assumeTrue("Set -Djmeter.regression=true to run JMeter regression tests", + Boolean.getBoolean("jmeter.regression")); + distribution = JmeterDistribution.resolve(); + distribution.installPluginJar(JmeterRegressionSupport.resolvePluginJar()); + } + + @AfterClass + public static void tearDownClass() throws Exception { + if (distribution != null) { + distribution.removePluginJars(); + } + } + + @Test + public void migratesJmxWhenRunFromRealJmeterLibExt() throws Exception { + Path workDir = Path.of("target", "jmx-migrator-cli-it"); + Files.createDirectories(workDir); + File sourceJmx = JmeterRegressionSupport.copyResourceToWorkDir("TEST_HTTP.jmx", workDir); + File targetJmx = workDir.resolve("TEST_HTTP_migrated.jmx").toFile(); + Files.deleteIfExists(targetJmx.toPath()); + + ProcessOutput processOutput = runCli(sourceJmx, targetJmx); + assertEquals("CLI process failed. Output:\n" + processOutput.output, + 0, processOutput.exitCode); + assertTrue("CLI process did not write " + targetJmx, targetJmx.isFile()); + + HashTree migrated = JmxBlazeMeterHttpMigrator.loadTree(targetJmx); + assertEquals(0, JmxBlazeMeterHttpMigrator.countMigratableSamplers(migrated)); + assertTrue(JmxBlazeMeterHttpMigrator.countHttp2Samplers(migrated) > 0); + } + + private ProcessOutput runCli(File sourceJmx, File targetJmx) + throws Exception { + Path libDir = distribution.getHomeDir().resolve("lib"); + Path extDir = libDir.resolve("ext"); + // "*" is a JVM classpath-wildcard convention, not a real path segment: Path.resolve("*") + // throws InvalidPathException on Windows, where "*" is a reserved filename character. + String classpath = libDir + File.separator + "*" + File.pathSeparator + + extDir + File.separator + "*"; + + List command = new ArrayList<>(); + command.add(JmeterRegressionRunner.resolveJavaExecutable()); + command.add("-Djava.awt.headless=true"); + command.add("-cp"); + command.add(classpath); + command.add(JmxBlazeMeterHttpMigratorCli.class.getName()); + command.add(sourceJmx.getAbsolutePath()); + command.add(targetJmx.getAbsolutePath()); + + ProcessBuilder builder = new ProcessBuilder(command); + builder.directory(distribution.getBinDir().toFile()); + builder.redirectErrorStream(true); + Process process = builder.start(); + String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + boolean finished = process.waitFor(TIMEOUT_MINUTES, TimeUnit.MINUTES); + if (!finished) { + process.destroyForcibly(); + throw new IllegalStateException( + "CLI process timed out after " + TIMEOUT_MINUTES + " minutes. Output:\n" + output); + } + return new ProcessOutput(process.exitValue(), output); + } + + private static final class ProcessOutput { + private final int exitCode; + private final String output; + + ProcessOutput(int exitCode, String output) { + this.exitCode = exitCode; + this.output = output; + } + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/JtlSampleLoader.java b/src/test/java/com/blazemeter/jmeter/http2/regression/JtlSampleLoader.java new file mode 100644 index 00000000..49a835de --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/JtlSampleLoader.java @@ -0,0 +1,99 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.io.File; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.util.ArrayList; +import java.util.List; +import javax.xml.parsers.DocumentBuilder; +import javax.xml.parsers.DocumentBuilderFactory; +import org.w3c.dom.Document; +import org.w3c.dom.Element; +import org.w3c.dom.Node; +import org.w3c.dom.NodeList; + +/** Loads JMeter XML sample logs (JTL or Result Collector output). */ +public final class JtlSampleLoader { + + private JtlSampleLoader() { + } + + public static List load(File xmlFile) throws Exception { + if (!xmlFile.isFile()) { + throw new IOException("Sample file not found: " + xmlFile); + } + DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); + factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true); + factory.setFeature("http://xml.org/sax/features/external-general-entities", false); + factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false); + factory.setExpandEntityReferences(false); + DocumentBuilder builder = factory.newDocumentBuilder(); + Document doc = builder.parse(xmlFile); + Element root = doc.getDocumentElement(); + List samples = new ArrayList<>(); + if ("testResults".equals(root.getTagName())) { + collectTopLevelSamples(root, samples); + } else if (isSampleElement(root)) { + samples.add(parseSample(root)); + } else { + collectTopLevelSamples(root, samples); + } + return samples; + } + + private static void collectTopLevelSamples(Element parent, List out) { + NodeList children = parent.getChildNodes(); + for (int i = 0; i < children.getLength(); i++) { + Node node = children.item(i); + if (node instanceof Element && isSampleElement((Element) node)) { + out.add(parseSample((Element) node)); + } + } + } + + private static boolean isSampleElement(Element element) { + String tag = element.getTagName(); + return "httpSample".equals(tag) + || "sample".equals(tag) + || "sampleResult".equals(tag); + } + + private static SampleRecord parseSample(Element element) { + String label = element.getAttribute("lb"); + if (label.isEmpty()) { + label = element.getAttribute("label"); + } + boolean success = Boolean.parseBoolean(element.getAttribute("s")); + String rc = element.getAttribute("rc"); + String rm = element.getAttribute("rm"); + String responseData = childText(element, "responseData"); + String responseHeaders = childText(element, "responseHeader"); + if (responseHeaders.isEmpty()) { + responseHeaders = childText(element, "responseHeaders"); + } + + List sub = new ArrayList<>(); + NodeList children = element.getChildNodes(); + for (int i = 0; i < children.getLength(); i++) { + Node node = children.item(i); + if (node instanceof Element && isSampleElement((Element) node)) { + sub.add(parseSample((Element) node)); + } + } + return new SampleRecord(label, success, rc, rm, responseData, responseHeaders, sub); + } + + private static String childText(Element parent, String tagName) { + NodeList nodes = parent.getElementsByTagName(tagName); + if (nodes.getLength() == 0) { + return ""; + } + Node node = nodes.item(0); + return node.getTextContent() == null ? "" : node.getTextContent(); + } + + static String readTextFile(File file) throws IOException { + return new String(Files.readAllBytes(file.toPath()), StandardCharsets.UTF_8); + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/RegressionProtocolProfile.java b/src/test/java/com/blazemeter/jmeter/http2/regression/RegressionProtocolProfile.java new file mode 100644 index 00000000..bb2a1f7a --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/RegressionProtocolProfile.java @@ -0,0 +1,75 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** JVM properties passed to JMeter when exercising BlazeMeter HTTP under a fixed protocol stack. */ +public enum RegressionProtocolProfile { + + HTTP1_ONLY("http1-only", + "legacy", + true, + false, + false, + false), + + HTTP2("http2", + "browser-compatible", + true, + true, + false, + true), + + HTTP3("http3", + "browser-compatible", + true, + true, + true, + true); + + private final String id; + private final String profileName; + private final boolean enableHttp1; + private final boolean enableHttp2; + private final boolean enableHttp3; + private final boolean alpnEnabled; + + RegressionProtocolProfile(String id, String profileName, boolean enableHttp1, + boolean enableHttp2, boolean enableHttp3, boolean alpnEnabled) { + this.id = id; + this.profileName = profileName; + this.enableHttp1 = enableHttp1; + this.enableHttp2 = enableHttp2; + this.enableHttp3 = enableHttp3; + this.alpnEnabled = alpnEnabled; + } + + public String getId() { + return id; + } + + public Map jmeterProperties() { + Map props = new LinkedHashMap<>(); + props.put("blazemeter.http.profile", profileName); + props.put("blazemeter.http.enableHttp1", Boolean.toString(enableHttp1)); + props.put("blazemeter.http.enableHttp2", Boolean.toString(enableHttp2)); + props.put("blazemeter.http.enableHttp3", Boolean.toString(enableHttp3)); + props.put("blazemeter.http.alpnEnabled", Boolean.toString(alpnEnabled)); + props.put("blazemeter.http.fallbackEnabled", "true"); + if (this == HTTP1_ONLY) { + props.put("blazemeter.http.altSvcCacheEnabled", "false"); + props.put("blazemeter.http.h2cCacheEnabled", "false"); + } + return props; + } + + public static RegressionProtocolProfile fromSystemProperty() { + String value = System.getProperty("jmeter.regression.protocol", "http1-only"); + for (RegressionProtocolProfile profile : values()) { + if (profile.id.equalsIgnoreCase(value)) { + return profile; + } + } + return HTTP1_ONLY; + } +} diff --git a/src/test/java/com/blazemeter/jmeter/http2/regression/SampleRecord.java b/src/test/java/com/blazemeter/jmeter/http2/regression/SampleRecord.java new file mode 100644 index 00000000..5eace756 --- /dev/null +++ b/src/test/java/com/blazemeter/jmeter/http2/regression/SampleRecord.java @@ -0,0 +1,64 @@ +package com.blazemeter.jmeter.http2.regression; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** Normalized view of one JMeter sample (and nested sub-samples). */ +public final class SampleRecord { + + private final String label; + private final boolean successful; + private final String responseCode; + private final String responseMessage; + private final String responseData; + private final String responseHeaders; + private final List children; + + public SampleRecord( + String label, + boolean successful, + String responseCode, + String responseMessage, + String responseData, + String responseHeaders, + List children) { + this.label = label; + this.successful = successful; + this.responseCode = responseCode; + this.responseMessage = responseMessage; + this.responseData = responseData == null ? "" : responseData; + this.responseHeaders = responseHeaders == null ? "" : responseHeaders; + this.children = children == null + ? Collections.emptyList() + : Collections.unmodifiableList(new ArrayList<>(children)); + } + + public String getLabel() { + return label; + } + + public boolean isSuccessful() { + return successful; + } + + public String getResponseCode() { + return responseCode; + } + + public String getResponseMessage() { + return responseMessage; + } + + public String getResponseData() { + return responseData; + } + + public String getResponseHeaders() { + return responseHeaders; + } + + public List getChildren() { + return children; + } +} diff --git a/src/test/resources/jmeter-regression/5.6.3/BUG_62847.jmx b/src/test/resources/jmeter-regression/5.6.3/BUG_62847.jmx new file mode 100644 index 00000000..34136103 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/BUG_62847.jmx @@ -0,0 +1,322 @@ + + + + + + false + true + true + + + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + true + 8 + + + + ${__jm__loop__idx} == 3 + false + + + + false + true + false + + + + + + false + true + false + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + 1 + 0 + 0 + + + + + continue_loop + + + + true + + + false + + + + + ${continue_loop} + + + + ${__jm__while__idx} == 3 + false + + + + false + true + false + + + + 1 + 0 + 0 + + + + + continue_loop + + + + false + + + false + + + + + + + false + true + false + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + 1 + 0 + 0 + + + + + idx_matchNr + idx_1 + idx_2 + idx_3 + idx_4 + idx_5 + idx_6 + idx_7 + idx_8 + + + + 8 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + + + false + + + + + idx + currentIdx + true + + + + ${__jm__for_idx__idx} == 3 + false + + + + false + true + false + + + + 1 + 0 + 0 + + + + + continue_loop + + + + false + + + false + + + + + + + false + true + false + + + + + false + + saveConfig + + + false + false + true + + true + true + true + false + false + false + false + false + false + false + true + false + false + false + false + 0 + + + BUG_62847.csv + + + + false + + saveConfig + + + false + false + true + + true + true + true + false + false + true + true + false + false + true + false + false + false + false + false + 0 + + + BUG_62847.xml + + + + false + + saveConfig + + + true + true + true + + true + true + true + true + false + true + true + false + false + false + true + false + false + false + true + 0 + true + true + true + true + true + true + + + + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/Bug54685.jmx b/src/test/resources/jmeter-regression/5.6.3/Bug54685.jmx new file mode 100644 index 00000000..1bd4d0ec --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/Bug54685.jmx @@ -0,0 +1,155 @@ + + + + + + false + false + + + + REFERENCE + reference + = + + + JSESSIONID + jsessionId + = + + + + + + + + continue + + false + 1 + + 1 + 1 + 1364309240000 + 1364309240000 + false + + + + + + + + + Sleep_Time + 5 + = + + + Sleep_Mask + 0x03 + = + + + Label + sample_variables=${__P(sample_variables,'undef')} REFERENCE=${REFERENCE} JSESSIONID=${JSESSIONID} + = + + + ResponseCode + + = + + + ResponseMessage + + = + + + Status + OK + = + + + SamplerData + + = + + + ResultData + + = + + + + org.apache.jmeter.protocol.java.test.JavaTest + + + + + false + + saveConfig + + + false + false + true + + true + true + false + true + false + false + false + false + false + false + true + false + false + false + false + 0 + true + + + Bug54685.csv + + + + false + + saveConfig + + + false + false + true + + true + true + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + true + + + Bug54685.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/HTMLParserTestFile_2.jmx b/src/test/resources/jmeter-regression/5.6.3/HTMLParserTestFile_2.jmx new file mode 100644 index 00000000..6600b890 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/HTMLParserTestFile_2.jmx @@ -0,0 +1,124 @@ + + + + + + false + false + + + + + + + + continue + + false + 1 + + 1 + 1 + 1317685259000 + 1317685259000 + false + + + + + + + + + + + file + + testfiles/HTMLParserTestFile_2.html + GET + false + false + false + false + true + + + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + true + true + true + false + true + true + false + true + true + false + true + 0 + true + true + true + true + true + + + HTMLParserTestFile_2.xml + + + + false + + saveConfig + + + true + false + true + + true + true + true + true + true + false + false + false + false + false + true + false + false + false + true + 0 + true + true + true + true + true + + + HTMLParserTestFile_2.csv + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/Http4ImplDigestAuth.jmx b/src/test/resources/jmeter-regression/5.6.3/Http4ImplDigestAuth.jmx new file mode 100644 index 00000000..0f7e250a --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/Http4ImplDigestAuth.jmx @@ -0,0 +1,218 @@ + + + + + + false + true + + + + has_auth_header + false + = + + + + + + + + + + + jmeter.apache.org + + + + / + 6 + + + + + + 200 + + + + continue + + false + 1 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + + https://jigsaw.w3.org/HTTP/Digest/ + guest + guest + + test + DIGEST + + + http://httpbin.org/digest-auth/ + user + passwd + + me@kennethreitz.com + DIGEST + + + + + + + + + jigsaw.w3.org + + https + + /HTTP/Digest/ + GET + true + false + true + false + + + + + + + + Digest Authentication test page + + Assertion.response_data + false + 16 + + + + + + + + + httpbin.org + + http + + /digest-auth/auth/user/passwd + GET + true + false + true + false + + + + + + + authenticated + $.authenticated + 1 + nv_authenticated + + + + + true + + Assertion.response_data + false + 16 + + variable + authenticated + + + + + + Authorization: Digest + + + Assertion.request_headers + false + 16 + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + false + true + false + false + false + false + 0 + + + Http4ImplDigestAuth.csv + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + Http4ImplDigestAuth.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/Http4ImplPreemptiveBasicAuth.jmx b/src/test/resources/jmeter-regression/5.6.3/Http4ImplPreemptiveBasicAuth.jmx new file mode 100644 index 00000000..970baf5f --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/Http4ImplPreemptiveBasicAuth.jmx @@ -0,0 +1,471 @@ + + + + + + false + true + + + + has_auth_header + false + = + + + + + + + + + + + jmeter.apache.org + + https + + / + 6 + 2000 + 5000 + + + + 200 + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + + http://localhost:8081/ + ${login} + ${password} + + * + + + + + + + + + + + + + / + GET + true + false + true + false + + + + + + + ; + + Http4ImplPreemptiveBasicAuth-data.csv + false + false + true + shareMode.group + false + login,password + + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + + http://localhost:8081/ + ${login} + ${password} + + * + + + + + + ; + + Http4ImplPreemptiveBasicAuth-data.csv + false + false + true + shareMode.group + false + login,password + + + + + + + + + https + + / + GET + true + false + true + false + + + + + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + + + + + https + + / + GET + true + false + true + false + + + + + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + + + + + https + + / + GET + true + false + true + false + + + + + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + has_auth_header + + + + true + + + false + + + + + + https://jmeter.apache.org/ + ${login} + ${password} + + * + + + + + + ; + + Http4ImplPreemptiveBasicAuth-data.csv + false + false + true + shareMode.group + false + login,password,headerValue + + + + + + + + + + + / + GET + true + false + true + false + + + + + + + + continue + + false + 2 + + 1 + 0 + 1485612143000 + 1485612143000 + false + + + + + + + has_auth_header + + + + true + + + false + + + + + + https://jmeter.apache.org/ + ${login} + ${password} + + * + + + + + + ; + + Http4ImplPreemptiveBasicAuth-data.csv + false + false + true + shareMode.group + false + login,password,headerValue + + + + + + + + + https + + / + GET + true + false + true + false + + + + + + + + 9796e1f7-1b35-434c-b29f-36c589b72ea9 + + + boolean mustHaveAuthHeader = vars["has_auth_header"].toBoolean(); +String requestHeaders = SampleResult.getRequestHeaders(); +if(mustHaveAuthHeader) { + if(requestHeaders.contains("Authorization")) { + if(requestHeaders.contains("Authorization: Basic "+vars["headerValue"])) { + AssertionResult.setFailure(false); + } else { + AssertionResult.setFailure(true); + AssertionResult.setFailureMessage("Wrong authorization header value:"+requestHeaders); + } + } else { + AssertionResult.setFailure(true); + AssertionResult.setFailureMessage("Request has no authorization header while it must have:"+requestHeaders); + } +} else { + if(requestHeaders.contains("Authorization")) { + AssertionResult.setFailure(true); + AssertionResult.setFailureMessage("Request has authorization header while it shouldn't have any:"+requestHeaders); + } +} + + groovy + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + false + true + false + false + false + false + 0 + + + Http4ImplPreemptiveBasicAuth.csv + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + Http4ImplPreemptiveBasicAuth.xml + + + + + true + + + + 8081 + 0 + 25 + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/ResponseDecompression.jmx b/src/test/resources/jmeter-regression/5.6.3/ResponseDecompression.jmx new file mode 100644 index 00000000..9350526d --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/ResponseDecompression.jmx @@ -0,0 +1,249 @@ + + + + + + false + false + + + + + + + + + + + + + https + + + 6 + 5000 + 30000 + + + + + + User-Agent + Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:52.0) Gecko/20100101 Firefox/52.0 + + + Accept + text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 + + + Accept-Language + en-US,en;q=0.5 + + + Cache-Control + max-age=0 + + + + + + continue + + false + 1 + + 1 + 1 + 1493581595000 + 1493581595000 + false + + + false + + + + + + + Content-Encoding: deflate + content-encoding: deflate + + Assertion.response_headers + false + 48 + + + + + + + Accept-Encoding + deflate + + + + + + + + + www.bing.com + + + + / + GET + true + false + true + false + + + + + + + + Microsoft + bing + + Assertion.response_data + false + 16 + + + + + + + 1 + 0 + 0 + + + + 500 + 100 + + + + + + + + Content-Encoding: br + content-encoding: br + + Assertion.response_headers + false + 48 + + + + + + + Accept-Encoding + br + + + + + + + + + www.facebook.com + + + + / + GET + true + false + true + false + + + + + + + + facebook + + Assertion.response_data + false + 16 + + + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + false + false + false + false + false + false + false + false + false + false + 0 + + + ResponseDecompression.csv + + + + false + + saveConfig + + + false + false + true + + true + true + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + ResponseDecompression.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/SlowCharsFeature.jmx b/src/test/resources/jmeter-regression/5.6.3/SlowCharsFeature.jmx new file mode 100644 index 00000000..17a8e2ee --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/SlowCharsFeature.jmx @@ -0,0 +1,383 @@ + + + + + + false + false + + + + + + + + + + Range + bytes=0-7000 + + + + + + continue + + false + 1 + + 1 + 1 + 1485469603000 + 1485469603000 + false + + + + + + true + + + import org.apache.jmeter.util.JMeterUtils; + +JMeterUtils.setProperty("httpclient.socket.http.cps","1500"); +JMeterUtils.setProperty("httpclient.socket.https.cps","1500"); +SampleResult.setBytes(0); + groovy + + + + + continue + + false + 1 + + 1 + 1 + 1485465167000 + 1485465167000 + false + + + + + + true + + + if(SampleResult.getTime() < 5000) { + AssertionResult.setFailure(true); + AssertionResult.setFailureMessage("Sampler should have taken more than 5 seconds"); +} + + groovy + + + + SizeAssertion.response_data + 7001 + 1 + + + + + 206 + + Assertion.response_code + false + 8 + + + + + + HTTP/1.1 206 Partial [Cc]ontent + + Assertion.response_headers + false + 2 + + + + + + + + + + jmeter.apache.org + + https + + + 6 + 3000 + 10000 + + + + ${__jexl3("${__P(jmeter.httpsampler,)}" != "Java",)} + false + true + Java implementation does not support slow chars for HTTP + + + + + + + + + + + / + GET + true + false + true + false + + + + + + + + + + + + + https + + / + GET + true + false + true + false + + + + + + + + Apache JMeter + + Assertion.response_data + false + 16 + + + + + + + + + + + analytics.usa.gov + + + + + 6 + 3000 + 10000 + + + + + + + + + https + + + GET + true + false + true + false + + HttpClient4 + + + + + + + <meta property="og:title" content="analytics.usa.gov | The US government's web traffic." /> + + Assertion.response_data + false + 16 + + + + + + javax.net.ssl.SSLHandshakeException + handshake_failure + + Assertion.response_data + false + 20 + + + + + + + + + + + https + + + GET + true + false + true + false + + HttpClient4 + 15000 + 60000 + + + + + <meta property="og:title" content="analytics.usa.gov | The US government's web traffic." /> + + Assertion.response_data + false + 16 + + + + + + javax.net.ssl.SSLHandshakeException + handshake_failure + + Assertion.response_data + false + 20 + + + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + false + false + false + false + false + true + false + false + false + false + 0 + + + SlowCharsFeature_${__property(jmeter.httpsampler,,HttpClient4)}.csv + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + SlowCharsFeature_${__property(jmeter.httpsampler,,HttpClient4)}.xml + + + + true + + saveConfig + + + true + true + true + + true + true + true + true + true + true + true + true + true + true + false + true + true + false + true + 0 + true + true + true + true + true + true + true + true + true + + + SlowCharsFeature_${__property(jmeter.httpsampler,,HttpClient4)}-errors.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TEST_GET.jmx b/src/test/resources/jmeter-regression/5.6.3/TEST_GET.jmx new file mode 100644 index 00000000..162c30ac --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TEST_GET.jmx @@ -0,0 +1,7 @@ + + + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TEST_HTTP.jmx b/src/test/resources/jmeter-regression/5.6.3/TEST_HTTP.jmx new file mode 100644 index 00000000..ed39afc8 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TEST_HTTP.jmx @@ -0,0 +1,1738 @@ + + + + + + false + true + + + + + true + + + + stoptest + + false + 1 + + 1 + 1 + 1488116764000 + 1488116764000 + false + + + + + + true + + + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = new HttpMirrorServer(8081, 10, 10); +props.put("MIRROR_SERVER", mirrorServer); + +mirrorServer.start(); + groovy + + + + + continue + + false + 1 + + 1 + 1 + 1488116271000 + 1488116271000 + false + + + + + + + + + + + + + /test + GET + true + false + true + false + + + + + + + + GET /test HTTP/1.1 + Host: localhost:8081 + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + + + + TEST_GET.jmx + file + text/xml + + + + + + + + + + + /test?name=value + GET + true + false + true + false + + + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + GET + Host: localhost:8081 + User-Agent: + /test?name=value + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + + + false + value1 + = + true + name1 + + + + + + + + /test?name0=value0 + GET + true + false + true + false + + + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + GET + Host: localhost:8081 + User-Agent: + /test?name0=value0&name1=value1 + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + ${__jexl3("${__P(jmeter.httpsampler,)}" != "Java",)} + false + true + + + + true + + + + false + Get with body + = + + + + + + + + /test + GET + true + false + true + false + + + + + + + + Content-Length: 13 + Content-Type: text/plain + Content-Type: + + Assertion.request_headers + false + 16 + + + + + + GET /test HTTP/1.1 + Host: localhost:8081 + User-Agent: + Get with body + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + + + + localhost + 8081 + + + + 6 + 3000 + 30000 + + + + + continue + + false + 1 + + 1 + 1 + 1488116271000 + 1488116271000 + false + + + + + + + + + www.example.org + + + + + 6 + 3000 + 30000 + + + + + + + + + + + / + GET + true + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + + Accept-Encoding: gzip + + Assertion.request_headers + false + 16 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + Example Domain + + Assertion.response_data + false + 16 + + + + + + + + + + + + + / + GET + true + false + true + false + + + + + + + + Content-Encoding: gzip + + Assertion.response_headers + false + 20 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + Accept-Encoding: gzip + + Assertion.request_headers + false + 20 + + + + + + Example Domain + + Assertion.response_data + false + 16 + + + + + + + continue + + false + 1 + + 1 + 1 + 1488116271000 + 1488116271000 + false + + + + + + + + + localhost + 8081 + http + + + 6 + 3000 + 30000 + + + + + + + + + + + / + GET + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + X-SetHeaders + Location:http://localhost:8081/redirect + + + X-ResponseStatus + 302 + + + + + + + Accept-Encoding: gzip + + Assertion.request_headers + false + 16 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + ${__jexl3("${__P(jmeter.httpsampler,)}" != "Java",)} + false + true + + + + true + + + + false + Method has body + = + + + + + + + + / + GET + false + false + true + false + + + + BUG 60682 + + + + + + Accept-Encoding + gzip + + + X-SetHeaders + Location:http://localhost:8081/redirect + + + X-ResponseStatus + 302 + + + + + + + Accept-Encoding: gzip + + Assertion.request_headers + false + 16 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 16 + + + + + + + true + + + + false + We have a body + = + + + + + + + + / + GET + true + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + X-SetHeaders + Location:http://localhost:8081/redirect + + + X-ResponseStatus + 302 + + + + + + + Exceeded maximum number of redirects + + Assertion.response_message + true + 16 + + + + + + Content-Length: + Content-Type: + + Assertion.request_headers + false + 20 + + + + + + + + + + + + + / + GET + false + true + true + false + + + + + + + + + Accept-Encoding + gzip + + + X-SetHeaders + Location:http://localhost:8081/redirect + + + X-ResponseStatus + 302 + + + + + + + Non HTTP response message + + Assertion.response_message + true + 16 + + + + + + + continue + + false + 1 + + 1 + 1 + 1488116271000 + 1488116271000 + false + + + + + + + + + localhost + 8081 + http + + + 6 + 3000 + 30000 + + + + + + + false + value1 + = + true + name1 + + + + + + + + /test?name0=value0 + POST + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + + POST /test?name0=value0 HTTP/1.1 + Accept-Encoding: gzip + Content-Length: 12 + Host: localhost:8081 + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + + + false + value1 + = + true + name1 + + + + + + + + /test?name0=value0 + POST + false + false + true + true + + + + + + + + + Accept-Encoding + gzip + + + + + + false + boundary + boundary=(.*) + $1$ + nv_boundary + 1 + + + + + POST /test?name0=value0 HTTP/1.1 + Accept-Encoding: gzip + Host: localhost:8081 + User-Agent: + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="name1" + Content-Type: text/plain + Content-Transfer-Encoding: 8bit + value1 + boundary=${boundary} + ${boundary} + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + + + false + value1 + = + true + name1 + + + false + value2 + = + true + 安_param + + + + + + + UTF-8 + /test?name0=value0 + POST + false + false + true + true + true + + + + + + + + + Accept-Encoding + gzip + + + + + + false + boundary + boundary=(.*) + $1$ + nv_boundary + 1 + + + + + POST /test?name0=value0 HTTP/1.1 + Accept-Encoding: gzip + Host: localhost:8081 + User-Agent: + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="name1" + value1 + boundary=${boundary} + ${boundary} + + Assertion.response_data + false + 16 + Content-Type: multipart/form-data; boundary= + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + Workaround as Mirror Server corrupts encoding + true + + + String textToCheck = 'Content-Disposition: form-data; name="安_param"'; +if(prev.getSamplerData().indexOf(textToCheck) < 0) { + AssertionResult.setFailure(true); + AssertionResult.setFailureMessage("Request does not contains '"+textToCheck+"'"); +} + + groovy + + + + + true + + + + false + Veni vidi vici + = + + + + + + + + /test?name0=value0 + POST + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + request_headers + contentType + Content-Type: (.*) + $1$ + nv_contentType + 1 + + + + + POST /test?name0=value0 HTTP/1.1 + Accept-Encoding: gzip + Content-Length: 14 + Host: localhost:8081 + User-Agent: + Veni vidi vici + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + nv_contentType + text/plain; charset=UTF-8 + + Assertion.response_data + false + 40 + + variable + contentType + + + + + + + + user.properties + fileName + text/plain + + + + + + + + + + + /test?name0=value0 + POST + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + false + boundary + boundary=(.*) + $1$ + nv_boundary + 1 + + + + + POST /test?name0=value0 HTTP/1.1 + Accept-Encoding: gzip + Host: localhost:8081 + User-Agent: + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Type: text/plain + Content-Transfer-Encoding: binary + boundary=${boundary} + ${boundary} + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + + + user.properties + + text/plain + + + + + + + + + + + /test?name0=value0 + POST + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + + Assertion.response_data + false + 20 + + + + + + Sample user.properties file + Host: localhost:8081 + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + Content-Type: text/plain + Content-Length: + + Assertion.request_headers + false + 16 + + + + + + + + + user.properties + + text/plain + + + + + + + + + + + /test?name0=value0 + POST + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + Content-Type + text/plain + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + PUT http://localhost:8081/test?name0=value0 + + Assertion.response_data + false + 20 + + + + + + Sample user.properties file + User-Agent: + Host: localhost:8081 + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + Content-Type: text/plain + Content-Length: + + Assertion.request_headers + false + 16 + + + + + + + + + user.properties + + text/plain + + + + + + + + + + + /test?name0=value0 + PUT + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + Content-Type + text/plain + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + + Assertion.response_data + false + 20 + + + + + + Sample user.properties file + PUT /test?name0=value0 HTTP/1.1 + Host: localhost:8081 + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + Content-Type: text/plain + Content-Length: + + Assertion.request_headers + false + 16 + + + + + + true + + + + false + Body of Put + = + + + + + + + + /test?name0=value0 + PUT + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + Content-Type + text/plain + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + + Assertion.response_data + false + 20 + + + + + + PUT /test?name0=value0 HTTP/1.1 + Body of Put + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + Content-Type: text/plain + Content-Length: + + Assertion.request_headers + false + 16 + + + + + + ${__jexl3("${__P(jmeter.httpsampler,)}" == "Java",)} + false + true + + + + + + + false + value1 + = + true + name1 + + + + + + + + /test?name0=value0 + PUT + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + Content-Type: text/plain + + Assertion.response_data + false + 20 + + + + + + PUT /test?name0=value0 HTTP/1.1 + Host: localhost:8081 + User-Agent: + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + + Assertion.request_headers + false + 16 + + + + + + + ${__jexl3("${__P(jmeter.httpsampler,)}" != "Java",)} + false + true + + + + + + + false + value1 + = + true + name1 + + + + + + + + /test?name0=value0 + PUT + false + false + true + false + + + + + + + + + Accept-Encoding + gzip + + + + + + + Content-Type: multipart/form-data; boundary= + Content-Disposition: form-data; name="fileName"; filename="user.properties" + Content-Transfer-Encoding: binary + boundary= + Content-Type: text/plain + + Assertion.response_data + false + 20 + + + + + + PUT /test?name0=value0 HTTP/1.1 + Host: localhost:8081 + User-Agent: + name1=value1 + + Assertion.response_data + false + 16 + + + + + + Connection: keep-alive + Accept-Encoding: gzip + + Assertion.request_headers + false + 16 + + + + + + + + stoptest + + false + 1 + + 1 + 1 + 1488116768000 + 1488116768000 + false + + + + + + groovy + + + true + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = (HttpMirrorServer) props.get("MIRROR_SERVER"); +mirrorServer.stopServer(); + + + + + false + + saveConfig + + + false + false + true + + true + false + true + true + false + false + false + false + false + false + false + false + false + false + false + 0 + + + TEST_HTTP_${__P(jmeter.httpsampler,)}.csv + + + + false + + saveConfig + + + false + false + true + + true + false + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + TEST_HTTP_${__P(jmeter.httpsampler,)}.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TEST_HTTPS.jmx b/src/test/resources/jmeter-regression/5.6.3/TEST_HTTPS.jmx new file mode 100644 index 00000000..6ffa072c --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TEST_HTTPS.jmx @@ -0,0 +1,194 @@ + + + + + + false + false + + + + + + + + continue + + false + 1 + + 1 + 1 + 1450720695000 + 1450720695000 + false + + + + + + + + + jmeter.apache.org + + https + + + GET + true + false + true + false + + HttpClient4 + + + + + + + Apache JMeter + + Assertion.response_data + false + 16 + + + + + javax.net.ssl.SSLHandshakeException + handshake_failure + + Assertion.response_data + false + 20 + + + + + + + + www.apache.org + + https + + + GET + true + false + true + false + + + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + false + false + false + false + false + true + false + false + false + false + 0 + true + true + + + TEST_HTTPS.csv + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + false + false + true + false + false + false + false + false + 0 + true + true + + + TEST_HTTPS.xml + + + + true + + saveConfig + + + false + false + true + + true + true + true + true + false + true + true + true + false + true + false + false + false + false + false + 0 + true + true + + + TEST_HTTPS.err + + + + + true + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TestCookieManager.jmx b/src/test/resources/jmeter-regression/5.6.3/TestCookieManager.jmx new file mode 100644 index 00000000..9f6c5519 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TestCookieManager.jmx @@ -0,0 +1,628 @@ + + + + + + false + true + true + + + + + + + + + + + localhost + 8081 + + + + 6 + + + + + + stoptest + + false + 1 + + 1 + 1 + 1488116764000 + 1488116764000 + false + + + + + + 877b8f57-5272-402c-bc11-cddbd4e69d71 + + + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = new HttpMirrorServer(8081, 10, 10); +props.put("MIRROR_SERVER", mirrorServer); + +mirrorServer.start(); + groovy + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + value1 + localhost + / + false + 0 + true + true + + + value3 + localhost + / + true + 0 + true + true + + + value2 + localhost + / + false + 0 + true + true + + + true + + + + + + + localhost + 8081 + + + / + GET + true + false + true + false + + + + + + + + Cookie: myCookie=value1; myCookie2=value2 + + Assertion.response_data + false + 16 + + + + + Cookie Data: + myCookie=value1; myCookie2=value2 + + Assertion.request_data + false + 16 + + + + + + + + localhost + 8082 + https + + / + GET + true + false + true + false + + + + + + + + Cookie Data: + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + Assertion.request_data + true + 16 + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + value1 + localhost + / + false + 0 + true + true + + + value3 + localhost + / + true + 0 + true + true + + + value2 + localhost + / + false + 0 + true + true + + + true + + + + + + + + + + + + 6 + Java + + + + + + + + + localhost + 8081 + + + / + GET + true + false + true + false + + + + + + + + Cookie: myCookie=value1; myCookie2=value2 + + Assertion.response_data + false + 16 + + + + + Cookie Data: + myCookie=value1; myCookie2=value2 + + Assertion.request_data + false + 16 + + + + + + + + localhost + 8082 + https + + / + GET + true + false + true + false + + + + + + + + Cookie Data: + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + Assertion.request_data + true + 16 + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + Cookie + myCookie=value1; myCookie2=value2 + + + + + + + + + localhost + 8081 + + + / + GET + true + false + true + false + + + + + + + + Cookie Data: + myCookie=value1; myCookie2=value2 + + Assertion.request_data + false + 16 + + + + + Cookie: myCookie=value1; myCookie2=value2 + + Assertion.response_data + false + 16 + + + + + + + + localhost + 8082 + https + + / + GET + true + false + true + false + + + + + + + + + Cookie + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + + + + + + Cookie Data: + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + Assertion.request_data + true + 16 + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + Cookie + myCookie=value1; myCookie2=value2 + + + + + + + + + + + + + + 6 + Java + + + + + + + + + localhost + 8081 + + + / + GET + true + false + true + false + + + + + + + + Cookie: myCookie=value1; myCookie2=value2 + + Assertion.response_data + false + 16 + + + + + Cookie Data: + myCookie=value1; myCookie2=value2 + + Assertion.request_data + false + 16 + + + + + + + + localhost + 8082 + https + + / + GET + true + false + true + false + + + + + + + + + Cookie + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + + + + + + Cookie Data: + myCookie=value1; mySecureCookie=value3; myCookie2=value2 + + Assertion.request_data + true + 16 + + + + + + stoptest + + false + 1 + + 1 + 1 + 1488116768000 + 1488116768000 + false + + + + + + groovy + + + 6e41dc10-a2fc-4d6b-a7a2-e76a1ceb22c5 + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = (HttpMirrorServer) props.get("MIRROR_SERVER"); +mirrorServer.stopServer(); + + + + + false + + saveConfig + + + false + false + true + + true + false + true + true + false + false + false + false + false + false + true + false + false + false + false + 0 + true + true + + + TestCookieManager.csv + + + + false + + saveConfig + + + false + false + true + + true + false + true + true + false + true + true + false + false + true + false + false + false + false + false + 0 + true + true + + + TestCookieManager.xml + + + + + true + + + + 8081 + 0 + 25 + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TestHeaderManager.jmx b/src/test/resources/jmeter-regression/5.6.3/TestHeaderManager.jmx new file mode 100644 index 00000000..8f1f0f4c --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TestHeaderManager.jmx @@ -0,0 +1,372 @@ + + + + + Test Header Manager merge feature + false + true + true + + + + + + + + + + Header1 + val1 + + + Header2 + val2 + + + Header3 + val3 + + + + + + + + + localhost + 8081 + + + + 6 + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + test-header-manager-mirror + + + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = new HttpMirrorServer(8081, 10, 10); +props.put("MIRROR_SERVER", mirrorServer); + +mirrorServer.start(); + groovy + + + + + continue + + false + 1 + + 1 + 1 + 1487511062000 + 1487511062000 + false + + + + + + + + Header5 + val5 + + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + + Header1 + val1_0 + + + Header2 + val2_0_overriden + + + Header4 + val4_0 + + + + + + + Header1: val1_0\b + + Header2: val2_0_overriden\b + Header4: val4_0\b + Header5: val5\b + Header3: val3\b + + Assertion.request_headers + true + 2 + + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + + Header1 + val1_1 + + + Header2 + val2_1_overriden + + + Header4 + val4_1 + + + + + + + Header1: val1_1\b + Header2: val2_1_overriden\b + Header4: val4_1\b + Header5: val5\b + Header3: val3\b + + Assertion.request_headers + true + 2 + + + + + + Header1: val1\b + Header2: val2\b + + Assertion.request_headers + true + 6 + + + + + + + continue + + false + 1 + + 1 + 1 + 1487511062000 + 1487511062000 + false + + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Header1: val1\b + Header2: val2\b + Header3: val3\b + + Assertion.request_headers + true + 2 + + + + + + Header4 + Header5 + + Assertion.request_headers + true + 20 + + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + test-header-manager-mirror-stop + + + import org.apache.jmeter.protocol.http.control.HttpMirrorServer; + +HttpMirrorServer mirrorServer = (HttpMirrorServer) props.get("MIRROR_SERVER"); +if (mirrorServer != null) { + mirrorServer.stopServer(); +} + groovy + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + false + + saveConfig + + + false + false + true + + false + false + true + true + false + false + false + false + false + false + false + false + false + false + false + 0 + + + TestHeaderManager.csv + + + + false + + saveConfig + + + false + false + true + + false + false + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + TestHeaderManager.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TestKeepAlive.jmx b/src/test/resources/jmeter-regression/5.6.3/TestKeepAlive.jmx new file mode 100644 index 00000000..a6c580f8 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TestKeepAlive.jmx @@ -0,0 +1,465 @@ + + + + + + false + false + + + + validationAfterInactivity + 2000 + = + + + ttl + 30000 + = + + + + + + + + + + + jmeter.apache.org + + https + + + jmeter.apache.org + true + 6 + 3000 + 10000 + + + + + false + + + + continue + + false + 1 + + 1 + 1 + 1487437592000 + 1487437592000 + false + + + + + + true + + + import org.apache.jmeter.util.JMeterUtils; +int validationAfterInactivity = vars["validationAfterInactivity"].toInteger(); +int ttl = vars["ttl"].toInteger(); + +JMeterUtils.setProperty("httpclient4.validate_after_inactivity", validationAfterInactivity.toString()); +JMeterUtils.setProperty("httpclient4.time_to_live", ttl.toString()); + groovy + + + + + continue + + false + 1 + + 1 + 1 + 1487436097000 + 1487436097000 + false + + + + + + + + + + + + + + GET + true + false + false + false + + + + + + + + Connection: close + + Assertion.response_headers + false + 16 + + + + + + Connection: close + + Assertion.request_headers + false + 16 + + + + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=100 + + Assertion.response_headers + false + 16 + + + + + true + keepAliveTimeout + Keep-Alive: timeout=(.+?), max= + $1$ + nv_keepAliveTimeout + 1 + + + + true + + + int validationAfterInactivity = vars["validationAfterInactivity"].toInteger(); +int ttl = vars["ttl"].toInteger(); +int firstPause = validationAfterInactivity - 300; +int secondPause = validationAfterInactivity + 500; +int thirdPause = ttl-(firstPause+secondPause); + +vars.put("firstPause",firstPause.toString()); +vars.put("secondPause",secondPause.toString()); +vars.put("thirdPause",thirdPause.toString()); + groovy + + + + + true + 1 + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=99 + + Assertion.response_headers + false + 16 + + + + + + 1 + 0 + 0 + + + + ${firstPause} + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=98 + + Assertion.response_headers + false + 16 + + + + + + 1 + 0 + 0 + + + + ${secondPause} + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=97 + + Assertion.response_headers + false + 16 + + + + + + 1 + 0 + 0 + + + + ${thirdPause} + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=100 + + Connection has exceeded its TTL + Assertion.response_headers + false + 16 + + + + + + 1 + 0 + 0 + + + + ${keepAliveTimeout}000 + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + Keep-Alive: timeout=30, max=100 + + Connection has exceeded its TTL + Assertion.response_headers + false + 16 + + + + + + + + Connection: keep-alive + + Assertion.request_headers + false + 16 + + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + false + false + false + false + false + false + false + false + false + false + 0 + + + TestKeepAlive.csv + + + + false + + saveConfig + + + false + false + true + + true + true + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + TestKeepAlive.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/TestRedirectionPolicies.jmx b/src/test/resources/jmeter-regression/5.6.3/TestRedirectionPolicies.jmx new file mode 100644 index 00000000..0f24954f --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/TestRedirectionPolicies.jmx @@ -0,0 +1,465 @@ + + + + + + false + true + true + + + + + + + + + + + httpbin.org + + + + /redirect/1 + 6 + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + + + + + + + GET + false + true + true + false + + + + + + + + 200 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==0 + groovy + + + + url + $..url + 1 + nv_url + + + + + https://httpbin.org/get + + + Assertion.response_data + false + 16 + variable + url + + + + + + + + + + + + + GET + false + false + true + false + + + + + + + + 302 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==0 + groovy + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + 200 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==2 + groovy + + + + url + $..url + 1 + nv_url + + + + + https://httpbin.org/get + + + Assertion.response_data + false + 16 + variable + url + + + + + + continue + + false + 1 + + 1 + 1 + false + + + + + + + + + + + + + + GET + false + false + true + false + + + + + + + + 302 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==0 + groovy + + + + + + + + + + + + + GET + true + false + true + false + + + + + + + + 200 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==2 + groovy + + + + url + $..url + 1 + nv_url + + + + + https://httpbin.org/get + + + Assertion.response_data + false + 16 + variable + url + + + + + + + + + + + + + GET + false + true + true + false + + + + + + + + 200 + + + Assertion.response_code + false + 16 + + + + true + + + assert prev.getSubResults().length==0 + groovy + + + + url + $..url + 1 + nv_url + + + + + https://httpbin.org/get + + + Assertion.response_data + false + 16 + variable + url + + + + + + false + + saveConfig + + + false + false + true + + true + true + true + true + false + false + false + false + false + false + false + false + false + false + false + 0 + + + TestRedirectionPolicies.csv + + + + false + + saveConfig + + + false + false + true + + true + true + false + true + false + true + true + false + false + true + false + false + false + false + false + 0 + + + TestRedirectionPolicies.xml + + + + true + + saveConfig + + + true + true + true + + true + true + true + true + true + true + true + true + true + true + false + true + true + false + true + 0 + true + true + true + true + true + true + true + true + true + + + TestRedirectionPolicies-errors.xml + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/jmeter-batch.properties b/src/test/resources/jmeter-regression/5.6.3/jmeter-batch.properties new file mode 100644 index 00000000..a93d6f13 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/jmeter-batch.properties @@ -0,0 +1,40 @@ +################################################################################ +# Apache JMeter Property file for batch runs +################################################################################ + +## Licensed to the Apache Software Foundation (ASF) under one or more +## contributor license agreements. See the NOTICE file distributed with +## this work for additional information regarding copyright ownership. +## The ASF licenses this file to You under the Apache License, Version 2.0 +## (the "License"); you may not use this file except in compliance with +## the License. You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. + +# Revert to original default mode +mode=Standard + +# Since JMeter 2.12, defaults for this property is true +jmeter.save.saveservice.thread_counts=false +# Since JMeter 3.0, defaults for this property is true +jmeter.save.saveservice.idle_time=false +# Since JMeter 3.1, defaults for this property is true +jmeter.save.saveservice.connect_time=false +# Since JMeter 3.1, defaults for this property is true +jmeter.save.saveservice.sent_bytes=false +# Since JMeter 5.0, defaults for this property is true +jmeter.save.saveservice.url=false + +# add some context in case tests fail (.jtl files are not compared) +jmeter.save.saveservice.responseHeaders=true +jmeter.save.saveservice.output_format=xml + +HTTPResponse.parsers=htmlParser +htmlParser.className=org.apache.jmeter.protocol.http.parser.LagartoBasedHtmlParser +htmlParser.types=text/html application/xhtml+xml application/xml text/xml diff --git a/src/test/resources/jmeter-regression/5.6.3/log4j2-batch.xml b/src/test/resources/jmeter-regression/5.6.3/log4j2-batch.xml new file mode 100644 index 00000000..65595d2a --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/log4j2-batch.xml @@ -0,0 +1,49 @@ + + + + + + + + + %d %p %c{1.}: %m%n + + + + + + %d %p %c{1.}: %m%n + + + + + + + + + + + + + + + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2.html b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2.html new file mode 100644 index 00000000..b28f2490 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2.html @@ -0,0 +1,1277 @@ + + + + + + + + + + + + + +JMeter - User's Manual: Elements of a Test Plan + + + + + + + + + + +
+ + + +Jakarta + +JMeter +
+ + + + + + + + + + +
+
+
+

About

+ +

Download

+ +

Documentation

+ +

Tutorials (PDF format)

+ +

Community

+ +

Foundation

+ +
+ + + + + + +
+ + + + + +
+
+ + + + +
+ +4. Elements of a Test Plan +
+
+

+The Test Plan object has a checkbox called "Functional Testing". If selected, it +will cause JMeter to record the data returned from the server for each sample. If you have +selected a file in your test listeners, this data will be written to file. This can be useful if +you are doing a small run to ensure that JMeter is configured correctly, and that your server +is returning the expected results. The consequence is that the file will grow huge quickly, and +JMeter's performance will suffer. This option should be off if you are doing stress-testing (it +is off by default). +

+

+If you are not recording the data to file, this option makes no difference. +

+

+You can also use the Configuration button on a listener to decide what fields to save. +

+ + + + +
+ +4.1 ThreadGroup + +
+
+

+Thread group elements are the beginning points of any test plan. +All controllers and samplers must be under a thread group. +Other elements, e.g. Listeners, may be placed directly under the test plan, +in which case they will apply to all the thread groups. +As the name implies, the thread group +element controls the number of threads JMeter will use to execute your test. The +controls for a thread group allow you to: + +

    +
  • +Set the number of threads +
  • + + +
  • +Set the ramp-up period +
  • + + +
  • +Set the number of times to execute the test +
  • + + +
+

+

+Each thread will execute the test plan in its entirety and completely independently +of other test threads. Multiple threads are used to simulate concurrent connections +to your server application. +

+

+The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of +threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then +JMeter will take 100 seconds to get all 10 threads up and running. Each thread will +start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads +and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds. +

+

+Ramp-up needs to be long enough to avoid too large a work-load at the start +of a test, and short enough that the last threads start running before +the first ones finish (unless one wants that to happen). + +

+

+ +Start with Ramp-up = number of threads and adjust up or down as needed. + +

+

+By default, the thread group is configured to loop once through its elements. +

+

+Version 1.9 introduces a test run + +scheduler + +. + Click the checkbox at the bottom of the Thread Group panel to reveal extra fields + in which you can enter the start and end times of the run. + When the test is started, JMeter will wait if necessary until the start-time has been reached. + At the end of each cycle, JMeter checks if the end-time has been reached, and if so, the run is stopped, + otherwise the test is allowed to continue until the iteration limit is reached. +

+

+Alternatively, one can use the relative delay and duration fields. + Note that delay overrides start-time, and duration over-rides end-time. +

+
+

+ + + + +
+ +4.2 Controllers + +
+
+

+ +JMeter has two types of Controllers: Samplers and Logical Controllers. +These drive the processing of a test. + +

+

+Samplers tell JMeter to send requests to a server. For +example, add an HTTP Request Sampler if you want JMeter +to send an HTTP request. You can also customize a request by adding one +or more Configuration Elements to a Sampler. For more +information, see + + +Samplers + +. +

+

+Logical Controllers let you customize the logic that JMeter uses to +decide when to send requests. For example, you can add an Interleave +Logic Controller to alternate between two HTTP Request Samplers. +For more information, see + +Logical Controllers + +. +

+
+

+ + + + +
+ +4.2.1 Samplers + +
+
+

+ +Samplers tell JMeter to send requests to a server and wait for a response. +They are processed in the order they appear in the tree. +Controllers can be used to modify the number of repetitions of a sampler. + +

+

+ +JMeter samplers include: + +

    + + +
  • +FTP Request +
  • + + +
  • +HTTP Request +
  • + + +
  • +JDBC Request +
  • + + +
  • +Java object request +
  • + + +
  • +LDAP Request +
  • + + +
  • +SOAP/XML-RPC Request +
  • + + +
  • +WebService (SOAP) Request +
  • + + +
+ +Each sampler has several properties you can set. +You can further customize a sampler by adding one or more Configuration Elements to the Test Plan. + +

+

+If you are going to send multiple requests of the same type (for example, +HTTP Request) to the same server, consider using a Defaults Configuration +Element. Each controller has one or more Defaults elements (see below). +

+

+Remember to add a Listener to your test plan to view and/or store the +results of your requests to disk. +

+

+If you are interested in having JMeter perform basic validation on +the response of your request, add an + +Assertion + + to +the sampler. For example, in stress testing a web application, the server +may return a successful "HTTP Response" code, but the page may have errors on it or +may be missing sections. You could add assertions to check for certain HTML tags, +common error strings, and so on. JMeter lets you create these assertions using regular +expressions. +

+

+ +JMeter's built-in samplers + +

+
+

+ + + + +
+ +4.2.2 Logic Controllers + +
+
+

+Logic Controllers let you customize the logic that JMeter uses to +decide when to send requests. +Logic Controllers can change the order of requests coming from their +child elements. They can modify the requests themselves, cause JMeter to repeat +requests, etc. + +

+

+To understand the effect of Logic Controllers on a test plan, consider the +following test tree: +

+

+ + +

    + + +
  • +Test Plan +
  • + + +
      + + +
    • +Thread Group +
    • + + +
        + + +
      • +Once Only Controller +
      • + + + + + +
      • +Load Search Page (HTTP Sampler) +
      • + + +
      • +Interleave Controller +
      • + + +
          + + +
        • +Search "A" (HTTP Sampler) +
        • + + +
        • +Search "B" (HTTP Sampler) +
        • + + +
        • +HTTP default request (Configuration Element) +
        • + + +
        + + +
      • +HTTP default request (Configuration Element) +
      • + + +
      • +Cookie Manager (Configuration Element) +
      • + + +
      + + +
    + + +
+ + +

+

+The first thing about this test is that the login request will be executed only +the first time through. Subsequent iterations will skip it. This is due to the +effects of the +Once Only Controller +. +

+

+After the login, the next Sampler loads the search page (imagine a +web application where the user logs in, and then goes to a search page to do a search). This +is just a simple request, not filtered through any Logic Controller. +

+

+After loading the search page, we want to do a search. Actually, we want to do +two different searches. However, we want to re-load the search page itself between +each search. We could do this by having 4 simple HTTP request elements (load search, +search "A", load search, search "B"). Instead, we use the +Interleave Controller + which passes on one child request each time through the test. It keeps the +ordering (ie - it doesn't pass one on at random, but "remembers" its place) of its +child elements. Interleaving 2 child requests may be overkill, but there could easily have +been 8, or 20 child requests. +

+

+Note the +HTTP Request Defaults + that +belongs to the Interleave Controller. Imagine that "Search A" and "Search B" share +the same PATH info (an HTTP request specification includes domain, port, method, protocol, +path, and arguments, plus other optional items). This makes sense - both are search requests, + hitting the same back-end search engine (a servlet or cgi-script, let's say). Rather than + configure both HTTP Samplers with the same information in their PATH field, we + can abstract that information out to a single Configuration Element. When the Interleave + Controller "passes on" requests from "Search A" or "Search B", it will fill in the blanks with + values from the HTTP default request Configuration Element. So, we leave the PATH field + blank for those requests, and put that information into the Configuration Element. In this +case, this is a minor benefit at best, but it demonstrates the feature. +

+

+The next element in the tree is another HTTP default request, this time added to the +Thread Group itself. The Thread Group has a built-in Logic Controller, and thus, it uses +this Configuration Element exactly as described above. It fills in the blanks of any +Request that passes through. It is extremely useful in web testing to leave the DOMAIN +field blank in all your HTTP Sampler elements, and instead, put that information +into an HTTP default request element, added to the Thread Group. By doing so, you can +test your application on a different server simply by changing one field in your Test Plan. +Otherwise, you'd have to edit each and every Sampler. +

+

+The last element is a +HTTP Cookie Manager +. A Cookie Manager should be added to all web tests - otherwise JMeter will +ignore cookies. By adding it at the Thread Group level, we ensure that all HTTP requests +will share the same cookies. +

+

+Logic Controllers can be combined to achieve various results. See the list of + +built-in +Logic Controllers + +. +

+
+

+ + + + +
+ +4.2.3 Test Fragments + +
+
+

+The Test Fragment element is a special type of + +controller + + that +exists on the Test Plan tree at the same level as the Thread Group element. It is distinguished +from a Thread Group in that it is not executed unless it is +referenced by either a +Module Controller + or an +Include_Controller +. + +

+

+This element is purely for code re-use within Test Plans and was introduced in Version 2.5 +

+
+

+ + + + +
+ +4.3 Listeners + +
+
+

+Listeners provide access to the information JMeter gathers about the test cases while +JMeter runs. The +Graph Results + listener plots the response times on a graph. +The "View Results Tree" Listener shows details of sampler requests and +responses, and can display basic HTML and XML representations of the +response. +Other listeners provide summary or aggregation information. + +

+

+ +Additionally, listeners can direct the data to a file for later use. +Every listener in JMeter provides a field to indicate the file to store data to. +There is also a Configuration button which can be used to choose which fields to save, and whether to use CSV or XML format. + + +Note that all Listeners save the same data; the only difference is in the way the data is presented on the screen. + + + +

+

+ +Listeners can be added anywhere in the test, including directly under the test plan. +They will collect data only from elements at or below their level. + +

+

+There are several + +listeners + + +that come with JMeter. +

+
+

+ + + + +
+ +4.4 Timers + +
+
+

+By default, a JMeter thread sends requests without pausing between each request. +We recommend that you specify a delay by adding one of the available timers to +your Thread Group. If you do not add a delay, JMeter could overwhelm your server by +making too many requests in a very short amount of time. +

+

+The timer will cause JMeter to delay a certain amount of time + +before + + each +sampler which is in its + +scope + +. +

+

+ +If you choose to add more than one timer to a Thread Group, JMeter takes the sum of +the timers and pauses for that amount of time before executing the samplers to which the timers apply. +Timers can be added as children of samplers or controllers in order to restrict the samplers to which they are applied. + +

+

+ +To provide a pause at a single place in a test plan, one can use the +Test Action + Sampler. + +

+
+

+ + + + +
+ +4.5 Assertions + +
+
+

+Assertions allow you to assert facts about responses received from the +server being tested. Using an assertion, you can essentially "test" that your +application is returning the results you expect it to. +

+

+For instance, you can assert that the response to a query will contain some +particular text. The text you specify can be a Perl-style regular expression, and +you can indicate that the response is to contain the text, or that it should match +the whole response. +

+

+You can add an assertion to any Sampler. For example, you can +add an assertion to a HTTP Request that checks for the text, "</HTML>". JMeter +will then check that the text is present in the HTTP response. If JMeter cannot find the +text, then it will mark this as a failed request. +

+

+ +Note that assertions apply to all samplers which are in its + +scope + +. +To restrict the assertion to a single sampler, add the assertion as a child of the sampler. + +

+

+To view the assertion results, add an Assertion Listener to the Thread Group. +Failed Assertions will also show up in the Tree View and Table Listeners, +and will count towards the error %age for example in the Aggregate and Summary reports. + +

+
+

+ + + + +
+ +4.6 Configuration Elements + +
+
+

+A configuration element works closely with a Sampler. Although it does not send requests +(except for +HTTP Proxy Server +), it can add to or modify requests. +

+

+A configuration element is accessible from only inside the tree branch where you place the element. +For example, if you place an HTTP Cookie Manager inside a Simple Logic Controller, the Cookie Manager will +only be accessible to HTTP Request Controllers you place inside the Simple Logic Controller (see figure 1). +The Cookie Manager is accessible to the HTTP requests "Web Page 1" and "Web Page 2", but not "Web Page 3". +

+

+Also, a configuration element inside a tree branch has higher precedence than the same element in a "parent" +branch. For example, we defined two HTTP Request Defaults elements, "Web Defaults 1" and "Web Defaults 2". +Since we placed "Web Defaults 1" inside a Loop Controller, only "Web Page 2" can access it. The other HTTP +requests will use "Web Defaults 2", since we placed it in the Thread Group (the "parent" of all other branches). +

+


+Figure 1 - + Test Plan Showing Accessibility of Configuration Elements +

+

+

+ +
+The +User Defined Variables + Configuration element is different. +It is processed at the start of a test, no matter where it is placed. +For simplicity, it is suggested that the element is placed only at the start of a Thread Group. + +
+

+
+

+ + + + +
+ +4.7 Pre-Processor Elements + +
+
+

+A Pre-Processor executes some action prior to a Sampler Request being +made. +If a Pre-Processor is attached to a Sampler element, then it will +execute just prior to that sampler element running. +A Pre-Processor is most often used to modify the settings of a Sample +Request just before it runs, or to update variables that aren't +extracted from response text. +See the + + +scoping rules + + + for more details on when Pre-Processors are executed. +

+
+

+ + + + +
+ +4.8 Post-Processor Elements + +
+
+

+A Post-Processor executes some action after a Sampler Request has been made. +If a Post-Processor is attached to a Sampler element, then it will execute just after that sampler element runs. +A Post-Processor is most often used to process the response data, often to extract values from it. +See the + + +scoping rules + + + for more details on when Post-Processors are executed. +

+
+

+ + + + +
+ +4.9 Execution order + +
+
+
    + + +
  1. +Configuration elements +
  2. + + +
  3. +Pre-Processors +
  4. + + +
  5. +Timers +
  6. + + +
  7. +Sampler +
  8. + + +
  9. +Post-Processors (unless SampleResult is null) +
  10. + + +
  11. +Assertions (unless SampleResult is null) +
  12. + + +
  13. +Listeners (unless SampleResult is null) +
  14. + + +
+

+

+ +
+Please note that Timers, Assertions, Pre- and Post-Processors are only processed if there is a sampler to which they apply. +Logic Controllers and Samplers are processed in the order in which they appear in the tree. +Other test elements are processed according to the scope in which they are found, and the type of test element. +[Within a type, elements are processed in the order in which they appear in the tree]. + +
+

+

+ +For example, in the following test plan: + +

    + + +
  • +Controller +
  • + + +
      + + +
    • +Post-Processor 1 +
    • + + +
    • +Sampler 1 +
    • + + +
    • +Sampler 2 +
    • + + +
    • +Timer 1 +
    • + + +
    • +Assertion 1 +
    • + + +
    • +Pre-Processor 1 +
    • + + +
    • +Timer 2 +
    • + + +
    • +Post-Processor 2 +
    • + + +
    + + +
+ +The order of execution would be: + +
+Pre-Processor 1
+Timer 1
+Timer 2
+Sampler 1
+Post-Processor 1
+Post-Processor 2
+Assertion 1
+
+Pre-Processor 1
+Timer 1
+Timer 2
+Sampler 2
+Post-Processor 1
+Post-Processor 2
+Assertion 1
+
+
+ + +

+
+

+ + + + +
+ +4.10 Scoping Rules + +
+
+

+ +The JMeter test tree contains elements that are both hierarchical and +ordered. Some elements in the test trees are strictly hierarchical +(Listeners, Config Elements, Post-Procesors, Pre-Processors, Assertions, + Timers), and some are primarily ordered (controllers, samplers). When +you create your test plan, you will create an ordered list of sample +request (via Samplers) that represent a set of steps to be executed. +These requests are often organized within controllers that are also +ordered. Given the following test tree: +

+


+Example test tree +

+

+The order of requests will be, One, Two, Three, Four. +

+

+Some controllers affect the order of their subelements, and you can read about these specific controllers in + +the component reference + +. +

+

+Other elements are hierarchical. An Assertion, for instance, is hierarchical in the test tree. +If its parent is a request, then it is applied to that request. If its +parent is a Controller, then it affects all requests that are descendants of +that Controller. In the following test tree: +

+


+Hierarchy example +

+

+Assertion #1 is applied only to Request One, while Assertion #2 is applied to Requests Two and Three. +

+

+Another example, this time using Timers: +

+


+complex example +

+

+In this example, the requests are named to reflect the order in which +they will be executed. Timer #1 will apply to Requests Two, Three, and +Four (notice how order is irrelevant for hierarchical elements). +Assertion #1 will apply only to Request Three. Timer #2 will affect all + the requests. +

+

+Hopefully these examples make it clear how configuration (hierarchical) +elements are applied. If you imagine each Request being passed up the +tree branches, to its parent, then to its parent's parent, etc, and each + time collecting all the configuration elements of that parent, then you + will see how it works. +

+ + +The Configuration elements Header Manager, Cookie Manager and Authorization manager are +treated differently from the Configuration Default elements. +The settings from the Configuration Default elements are merged into a set of values that the Sampler has access to. +However, the settings from the Managers are not merged. +If more than one Manager is in the scope of a Sampler, +only one Manager is used, but there is currently no way to specify + +which + + is used. + + +
+

+ + + + +
+ +4.11 Properties and Variables + +
+
+

+ +JMeter + +properties + + are defined in jmeter.properties (see + +Gettting Started - Configuring JMeter + + for more details). + +
+
+ +Properties are global to jmeter, and are mostly used to define some of the defaults JMeter uses. +For example the property remote_hosts defines the servers that JMeter will try to run remotely. +Properties can be referenced in test plans +- see + +Functions - read a property + + - +but cannot be used for thread-specific values. + +

+

+ +JMeter + +variables + + are local to each thread. The values may be the same for each thread, or they may be different. + +
+
+ +If a variable is updated by a thread, only the thread copy of the variable is changed. +For example the +Regular Expression Extractor + Post-Processor +will set its variables according to the sample that its thread has read, and these can be used later +by the same thread. +For details of how to reference variables and functions, see + +Functions and Variables + + + +

+

+ +Note that the values defined by the +Test Plan + and the +User Defined Variables + configuration element +are made available to the whole test plan at startup. +If the same variable is defined by multiple UDV elements, then the last one takes effect. +Once a thread has started, the initial set of variables is copied to each thread. +Other elements such as the + +User Parameters + Pre-Processor or +Regular Expression Extractor + Post-Processor +may be used to redefine the same variables (or create new ones). These redefinitions only apply to the current thread. + +

+

+ +The + +setProperty + + function can be used to define a JMeter property. +These are global to the test plan, so can be used to pass information between threads - should that be needed. + +

+

+

+ +
Both variables and properties are case-sensitive. +
+

+
+

+ + + + +
+ +4.12 Using Variables to parameterise tests + +
+
+

+ +Variables don't have to vary - they can be defined once, and if left alone, will not change value. +So you can use them as short-hand for expressions that appear frequently in a test plan. +Or for items which are constant during a run, but which may vary between runs. +For example, the name of a host, or the number of threads in a thread group. + +

+

+ +When deciding how to structure a Test Plan, +make a note of which items are constant for the run, but which may change between runs. +Decide on some variable names for these - +perhaps use a naming convention such as prefixing them with C_ or K_ or using uppercase only +to distinguish them from variables that need to change during the test. +Also consider which items need to be local to a thread - +for example counters or values extracted with the Regular Expression Post-Processor. +You may wish to use a different naming convention for these. + +

+

+ +For example, you might define the following on the Test Plan: + +

+HOST             www.example.com
+THREADS          10
+LOOPS            20
+
+
+ +You can refer to these in the test plan as ${HOST} ${THREADS} etc. +If you later want to change the host, just change the value of the HOST variable. +This works fine for small numbers of tests, but becomes tedious when testing lots of different combinations. +One solution is to use a property to define the value of the variables, for example: + +
+HOST             ${__P(host,www.example.com)}
+THREADS          ${__P(threads,10)}
+LOOPS            ${__P(loops,20)}
+
+
+ +You can then change some or all of the values on the command-line as follows: + +
+jmeter ... -Jhost=www3.example.org -Jloops=13
+
+
+ + +

+
+

+
+

+

+ + + + + + +
+ + + + + +
+
+
+
+
+
+Copyright © 1999-2011, Apache Software Foundation +
+
+
+Apache, Apache JMeter, JMeter, the Apache feather, and the Apache JMeter logo are +trademarks of the Apache Software Foundation. + +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner.htm b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner.htm new file mode 100644 index 00000000..55dbc5c5 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner.htm @@ -0,0 +1,6 @@ + + + + + + \ No newline at end of file diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner_data/2011-na-234x60.png b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner_data/2011-na-234x60.png new file mode 100644 index 00000000..7dc685c3 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/halfbanner_data/2011-na-234x60.png differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/http-config-example.png b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/http-config-example.png new file mode 100644 index 00000000..5bb36f42 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/http-config-example.png differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/jakarta-logo.gif b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/jakarta-logo.gif new file mode 100644 index 00000000..049cf822 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/jakarta-logo.gif differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/logo.jpg b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/logo.jpg new file mode 100644 index 00000000..304363ab Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/logo.jpg differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping1.png b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping1.png new file mode 100644 index 00000000..bfbc5eb7 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping1.png differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping2.png b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping2.png new file mode 100644 index 00000000..c18ca840 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping2.png differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping3.png b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping3.png new file mode 100644 index 00000000..d263fa29 Binary files /dev/null and b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/scoping3.png differ diff --git a/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/style.css b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/style.css new file mode 100644 index 00000000..7d3a6829 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/testfiles/HTMLParserTestFile_2_files/style.css @@ -0,0 +1,39 @@ +/* + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +*/ + +/*Shows the value of the name attribute when hovered*/ +/* Disabled +a[name]:hover:after{ + content: " #" attr(name); + font-size: 90%; + text-decoration: none; +} +*/ + +/* + * Hide class="sectionlink", except when an enclosing heading + * has the :hover property. + * Used to hide the ¶ marker for generating internal links + */ +.sectionlink { + display: none; +} +:hover > .sectionlink { + display: inline; + /* Green so shows up on section headings too */ + color: rgb(0,255,0); +} diff --git a/src/test/resources/jmeter-regression/5.6.3/user.properties b/src/test/resources/jmeter-regression/5.6.3/user.properties new file mode 100644 index 00000000..b3504dd5 --- /dev/null +++ b/src/test/resources/jmeter-regression/5.6.3/user.properties @@ -0,0 +1,2 @@ +# Sample user.properties file +sample=true