Skip to content

NJ/NY and crawl data updates - #141

Merged
mccoogp merged 8 commits into
mainfrom
NJ/NY-Updates
Oct 21, 2026
Merged

mccoogp merged 8 commits into
mainfrom
NJ/NY-Updates

Conversation

@mccoogp

@mccoogp mccoogp commented Oct 13, 2026

Copy link
Copy Markdown
Member

The majority of these changes are updating the UI with the findings of privacy-tech-lab/gpc-web-crawler#345

Across the UI, NJ has been renamed to NJ/NY, and whenever it is selected a warning pops up with a longer explanation at the end which points them to more information.
Screenshot 2026-10-13 at 1 34 25 PM
The same popup appears when selecting NJ/NY in the data table and GPC breakdown. This popup only occurs when NJ/NY is selected.
When clicking on the NJ/NY warning popup, it takes you to a separate page that breaks down the findings described in https://docs.google.com/spreadsheets/d/1fVdlDKdnzLb1XFmGooXaQbuF7XYzTx34jUsT6kWvmyQ/edit?gid=567337877#gid=567337877.
I would love both @KyleQ-hub and @RubyFri to thoroughly verify all of the information on this page, and take over edits of the page if they would like.

The AugSep has been added to the crawler along with redownloading all of the previous crawl data because it was never updated with the redone post-processing of privacy-tech-lab/gpc-web-crawler#313

I've also made two smaller changes to edit columns in accordance with Ruby's comments in #135. The compliance results descriptions now include "None" as an option. Site ID has also been added back as an option for edit columns.

@mccoogp
mccoogp requested review from KyleQ-hub and RubyFri October 13, 2026 17:35
@RubyFri

RubyFri commented Oct 14, 2026

Copy link
Copy Markdown
Member

I made some major edits to this branch including making tables/charts fit without having to scroll within them, and also changing a lot of the language used on the page to more accurately reflect what we want to say.

Happy to discuss my changes in our meeting today or another one later @mccoogp, let me know.

@mccoogp

mccoogp commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

I realized I added in the AugSep and NJ_NY data directly through the generated datasts.json, but this is in git ignore because it's supposed to be handled by the generator. It wasn't working for all of these added datasets because it didn't know how to deal with multiple state names and because of the sep vs sept naming inconsistency. This is fixed now so all of the data should load properly.

I also made some small edits to the NJ page.
I fixed a couple of small typos, mostly just grammar fixes.
I also realized both versions of this page made it sound like we no longer use the NJ Mullvad VPN, "We have historically performed our "New Jersey" crawl using MullvadVPN's Secaucus, NJ connection." So I made some changes in the background to be slightly more clear that we are continuing to use Mullvad and that the Express VPN was a one time investigation. I also added a final sentence in the implications about clearly stating that despite these findings, we are continuing to crawl with Mullvad and it led to our decision to consider it a NJ_NY crawl.

@RubyFri

RubyFri commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Looked through all changes, I approve! @mccoogp feel free to merge and close all issues associated with this PR.

@mccoogp
mccoogp merged commit 6a3f445 into main Oct 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants