Skip to content

Notifications: Email alerts for eval completion #837

@Ayush8923

Description

@Ayush8923

Is your feature request related to a problem?
Currently, users associated with a project do not receive notifications when an eval run completes. They have to manually check the platform, which is inconvenient for long-running eval workflows.

Describe the solution you'd like

  • Trigger notification only after an eval run is completed.
  • Fetch all users associated with the project.
  • Send an email notification to all project users.
  • Include relevant details in the email:
    • Eval name
    • Project name
    • Completion status
    • Completion timestamp
    • Link to view eval results
  • Ensure duplicate emails are not triggered.
  • Add proper logging for the notification flow.
  • Handle failed email delivery gracefully.
Original issue

Describe the current behavior
Implement email notifications for eval runs so that all users associated with a project are notified once an eval execution is completed. Currently, users need to manually check the platform to know whether an eval has completed, which is inconvenient for long-running eval workflows.

Describe the enhancement you'd like

  • Trigger notification only after an eval run is completed.
  • Fetch all users associated with the corresponding project.
  • Send an email notification to all connected project users.
  • Include relevant details in the email:
    • Eval name
    • Project name
    • Completion status
    • Completion timestamp
    • Link to view eval results
  • Ensure duplicate emails are not triggered.
  • Add proper logging for the notification flow.
  • Handle failed email delivery gracefully.

Why is this enhancement needed?

  • All users connected to the project receive an email after eval completion.
  • Email contains eval details and result link.
  • Notification is triggered only once per completed eval run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    To Do

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions