Skip to content

data.md

Latest commit

 

History

History
99 lines (71 loc) · 3.09 KB

data.md

File metadata and controls

99 lines (71 loc) · 3.09 KB
Raw

Downloading Data Using BugBug

BugBug relies on various types of data, such as bugs, commits, issues, and crash reports, to build its models. Although all this data is publicly available through different APIs, retrieving it every time we train a model is not an efficient solution. Hence, a copy of the data is saved as downloadable compressed files through a simple API.

Note: You can use the data outside this project by using BugBug as a dependency (pip install bugbug).

Bugzilla Bugs

from bugbug import bugzilla, db

# Downland the latest version if the data set if it is not already downloaded
db.download(bugzilla.BUGS_DB)

# Iterate over all bugs in the dataset
for bug in bugzilla.get_bugs():
    # This is the same as if you retrieved the bug through Bugzilla REST API:
    # https://bmo.readthedocs.io/en/latest/api/core/v1/bug.html
    print(bug["id"])

Uplift Data

Here is an example of how to extract uplift requests and approvals from Bugzilla bug histories:

from bugbug import bugzilla, db

db.download(bugzilla.BUGS_DB)

for bug in bugzilla.get_bugs():
    for history in bug["history"]:
        for change in history["changes"]:
            if change["added"].startswith("approval-mozilla"):
                uplift_tags = change["added"].split(", ")
                for uplift_tag in uplift_tags:
                    release_channel = uplift_tag[len("approval-mozilla-") : -1]
                    if uplift_tag.endswith("?"):
                        print(
                            f"Uplift: Requested \tBug {bug['id']}\t{history['when']} \t{release_channel}"
                        )
                    elif uplift_tag.endswith("+"):
                        print(
                            f"Uplift: Approved  \tBug {bug['id']}\t{history['when']} \t{release_channel}"
                        )

Performance Alerts and Regressions

from bugbug import db
from bugbug.perf import datasets

db.download(datasets.PERF_REGRESSIONS_DB)

for regression in datasets.get_perf_regressions():
    # One record per confirmed Perfherder regression summary, with the culprit
    # push, the pinned culprit commits, the outcome and the regressed
    # performance tests.
    print(regression["summary_id"], regression["revision"], regression["outcome"])

See the performance alerts pipeline for the raw snapshots, the labeling policy and the commit level dataset used by the performance regression predictor.

Phabricator Revisions

from bugbug import phabricator, db

db.download(phabricator.REVISIONS_DB)

for revision in phabricator.get_revisions():
    # The revision here combines the results retrieved from two API endpoints:
    # https://phabricator.services.mozilla.com/conduit/method/differential.revision.search/
    # https://phabricator.services.mozilla.com/conduit/method/transaction.search/
    print(revision["id"])

Repository Commits

from bugbug import repository, db

db.download(repository.COMMITS_DB)

for commit in repository.get_commits():
    print(commit["node"])

Github Issues

TODO

Mozilla Crash Reports

TODO