Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 17 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
# Reinforcement Learning (RL)
> **Reinforcement Learning (RL)** brings Thompson Sampling multi-armed bandit A/B testing to Drupal, automatically optimizing content variants based on real user engagement without third-party services. Built by [DXPR](https://dxpr.com).
>
> [Getting Started](https://dxpr.com/c/getting-started) |
> [Pricing](https://dxpr.com/pricing) |
> [Try Free Demo](https://dxpr.com/try)

# Reinforcement Learning (RL) - Adaptive A/B Testing for Drupal with Thompson Sampling

Multi-armed bandit experiments in Drupal using Thompson Sampling algorithm for
efficient A/B testing that minimizes lost conversions.
Expand Down Expand Up @@ -424,11 +430,6 @@ RL provides optional cache management for web components:
- Blocks displaying A/B tested content
- Components needing frequent RL score updates

## Related Modules

- [AI Sorting](https://www.drupal.org/project/ai_sorting) - Intelligent content
ordering for Drupal Views

## Technical Implementation

Full algorithm details available in source code:
Expand Down Expand Up @@ -539,3 +540,13 @@ docker compose --profile lint run --rm drupal-check
- [Thompson Sampling Paper](https://www.jstor.org/stable/2332286) - Original research
- [Finite-time Analysis](https://homes.di.unimi.it/~cesa-bianchi/Pubblicazioni/ml-02.pdf) -
Mathematical foundations

---

## Related Modules

- [RL Sorting](https://www.drupal.org/project/rl_sorting) - Views sort plugin that uses RL Thompson Sampling to order content by real engagement
- **RL Page Title** (bundled submodule) - A/B test page titles on any path, including nodes, Views displays, and custom controllers
- **RL Menu Link** (bundled submodule) - A/B test menu link labels for menu_link_content entities and YAML-defined links
- [Analyze](https://www.drupal.org/project/analyze) - Content analysis and quality scoring for Drupal
- [AI Content Strategy](https://www.drupal.org/project/ai_content_strategy) - AI-driven content strategy recommendations for Drupal
35 changes: 23 additions & 12 deletions docs/rl_project_desc.html
Original file line number Diff line number Diff line change
@@ -1,22 +1,26 @@
<p>This module is included in <a href="https://www.drupal.org/project/dxpr_cms">DXPR CMS</a>.</p>

<blockquote>The <strong>Reinforcement Learning (RL)</strong> module implements A/B testing in the most efficient and effective way possible, minizing lost conversions using machine learning.</blockquote>
<blockquote>The <strong>Reinforcement Learning (RL)</strong> module implements A/B testing in the most efficient and effective way possible, minimizing lost conversions using machine learning.</blockquote>


<p><strong>Thompson Sampling</strong> is a learning-while-doing method. Each time a visitor lands on your site the algorithm "rolls the dice" based on what it has learned so far. Variants that have performed well roll larger numbers, so they are shown more often, while weak copies still get a small chance to prove themselves. This simple trick means the system can discover winners very quickly without stopping normal traffic.</p>

<p>Traditional A/B tests run for a fixed horizonsay two weeksduring which half your visitors keep seeing the weaker version. Thompson Sampling avoids this waste. As soon as the algorithm has even a little evidence it quietly shifts most traffic to the better variant, saving conversions and shortening the wait for useful insights.</p>
<p>Traditional A/B tests run for a fixed horizon (say two weeks) during which half your visitors keep seeing the weaker version. Thompson Sampling avoids this waste. As soon as the algorithm has even a little evidence it quietly shifts most traffic to the better variant, saving conversions and shortening the wait for useful insights.</p>

<p>For full details of what goes on behind the curtains check the source code:
<a href="https://git.drupalcode.org/project/rl/-/blob/1.x/src/Service/ThompsonCalculator.php" target="_blank" rel="noopener">ThompsonCalculator.php</a>.
</p>

<h3>RL Modules</h3>
<h3>RL Ecosystem</h3>

<ul>
<li><a href="https://www.drupal.org/project/ai_sorting"><strong>AI Sorting</strong></a> - Intelligent content ordering/switching for Drupal Views</li>
<li><a href="https://www.drupal.org/project/rl_sorting"><strong>RL Sorting</strong></a> - Views sort plugin that uses RL Thompson Sampling to order content by real engagement</li>
<li><strong>RL Page Title</strong> (bundled) - A/B test page titles on any path: nodes, Views displays, custom controllers</li>
<li><strong>RL Menu Link</strong> (bundled) - A/B test menu link labels for menu_link_content entities and YAML-defined links</li>
</ul>

<p>RL is part of the <a href="https://dxpr.com/c/marketing-cms">DXPR marketing CMS</a> stack, where it works alongside DXPR Builder and DXPR Theme to optimize content through automated experimentation.</p>

<h3>Features</h3>

<ul>
Expand Down Expand Up @@ -50,7 +54,7 @@ <h3>Thompson Sampling</h3>

<div class="note-tip">
<h4>Prefer a turnkey demo site?</h4>
<p>Spin up <strong>DXPR CMS</strong>Drupal pre-configured with DXPR Builder, DXPR Theme, RL (Reinforcement Learning) module, and security best practices.</p>
<p>Spin up <strong>DXPR CMS</strong>, Drupal pre-configured with DXPR Builder, DXPR Theme, RL (Reinforcement Learning) module, and security best practices.</p>
<p><a href="https://www.drupal.org/project/dxpr_cms" title="DXPR CMS platform">Get DXPR CMS »</a></p>
</div>

Expand Down Expand Up @@ -193,13 +197,13 @@ <h4>Does RL store my experiment's variants?</h4>
<p>Different modules keep the live variant list in different places:</p>

<ul>
<li><strong>ai_sorting</strong> the content returned by a View</li>
<li><strong>rl_page_title</strong> fields on a content entity</li>
<li><strong>rl_menu_link</strong> labels in a menu link</li>
<li><strong>DXPR Builder</strong> slots inside a block component</li>
<li><strong>ai_sorting</strong>: the content returned by a View</li>
<li><strong>rl_page_title</strong>: fields on a content entity</li>
<li><strong>rl_menu_link</strong>: labels in a menu link</li>
<li><strong>DXPR Builder</strong>: slots inside a block component</li>
</ul>

<p>On each call, your module passes its current list <code>getThompsonScores($id, NULL, $arms)</code> in PHP or <code>Drupal.rl.decide(id, arms)</code> in JS and RL matches it against the stored stats to pick a winner. A newly added variant is automatically in play on the next render, and a removed one stops appearing. No second save step can fall out of sync with your module's UI.</p>
<p>On each call, your module passes its current list (<code>getThompsonScores($id, NULL, $arms)</code> in PHP or <code>Drupal.rl.decide(id, arms)</code> in JS) and RL matches it against the stored stats to pick a winner. A newly added variant is automatically in play on the next render, and a removed one stops appearing. No second save step can fall out of sync with your module's UI.</p>

<h4>When do I pick a winner and end an experiment?</h4>

Expand All @@ -208,12 +212,19 @@ <h4>When do I pick a winner and end an experiment?</h4>
<p>Two patterns, depending on what you are testing:</p>

<ul>
<li><strong>Converging tests</strong> a better page title, a clearer checkout button, a stronger hero image. Once the report shows a confident winner, lock it in and move on to the next test.</li>
<li><strong>Evergreen experiments</strong> blog post lists where reader interest drifts week to week, banner ads that fade as returning visitors tune them out, seasonal calls to action. Leave them running. Thompson Sampling will follow the winner as it shifts.</li>
<li><strong>Converging tests</strong>: a better page title, a clearer checkout button, a stronger hero image. Once the report shows a confident winner, lock it in and move on to the next test.</li>
<li><strong>Evergreen experiments</strong>: blog post lists where reader interest drifts week to week, banner ads that fade as returning visitors tune them out, seasonal calls to action. Leave them running. Thompson Sampling will follow the winner as it shifts.</li>
</ul>

<p>In both cases the loser of a pair just stops receiving traffic on its own, so there is no urgency to declare a winner by hand. If you are used to fixed-horizon A/B tools, this is the biggest mental shift: there is no "test complete" flag to chase.</p>

<h3>Related Modules</h3>
<ul>
<li><a href="https://www.drupal.org/project/rl_sorting">RL Sorting</a> - Views sort plugin that uses RL Thompson Sampling to order content by real engagement</li>
<li><a href="https://www.drupal.org/project/analyze">Analyze</a> - Content analysis and quality scoring for Drupal</li>
<li><a href="https://www.drupal.org/project/ai_content_strategy">AI Content Strategy</a> - AI-driven content strategy recommendations for Drupal</li>
</ul>

<h3>Resources</h3>
<ul>
<li><a href="https://everyday-data-science.tigyog.app/a-b-testing" target="_blank" rel="noopener">Free Micro-Course on Thompson Sampling</a></li>
Expand Down