Skip to content

Comments

Improve PR Bot reviewer assignment / pinging#37663

Open
kennknowles wants to merge 1 commit intoapache:masterfrom
kennknowles:pr-bot
Open

Improve PR Bot reviewer assignment / pinging#37663
kennknowles wants to merge 1 commit intoapache:masterfrom
kennknowles:pr-bot

Conversation

@kennknowles
Copy link
Member

  1. It missed review comments, so would ping even when review was active.
  2. It would not treat Github UI review assignments the same as "R:" comments

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

1. It missed review comments, so would ping even when review was active.
2. It would not treat Github UI review assignments the same as "R:"
   comments
@kennknowles
Copy link
Member Author

R: @damccorm

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @kennknowles, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refines the PR bot's ability to manage reviewer assignments and pings. It addresses previous shortcomings where the bot would incorrectly ping active reviews or fail to recognize reviewers assigned through GitHub's native UI. The changes ensure a more intelligent and less intrusive review process by integrating GitHub's reviewer assignment mechanisms more deeply into the bot's operational logic.

Highlights

  • Improved Review Activity Detection: The PR bot now correctly identifies active reviews by checking individual review comments, preventing unnecessary pings when a review is already underway.
  • Unified Reviewer Assignment Logic: GitHub UI review assignments are now treated equivalently to 'R:' comments, ensuring the bot's logic for assigning and pinging reviewers is consistent across both methods.
  • Enhanced Committer Assignment Logic: The bot now checks if any GitHub-assigned reviewer is a committer, influencing whether it needs to assign an additional committer.
  • Explicit Handling for Review Requests: The bot now explicitly processes the 'review_requested' GitHub action, ensuring timely updates to reviewer assignments.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • scripts/ci/pr-bot/findPrsNeedingAttention.ts
    • Corrected isSlowReview function to check individual review comment authors, preventing false positives for active reviews.
    • Extended reviewersToExclude in assignToNewReviewers to include users explicitly requested as reviewers via GitHub's UI.
    • Modified processPull to include GitHub-requested reviewers when constructing the list of reviewers to ping for slow reviews.
  • scripts/ci/pr-bot/processNewPrs.ts
    • Added isAnyGithubReviewerCommitter function to determine if any GitHub-requested reviewer is a committer.
    • Modified processPull to consider both bot-assigned and GitHub-requested reviewers when checking if reviewers are already assigned.
    • Enhanced processPull to attempt to derive a label for GitHub-assigned reviewers if not found in the bot's state, and updated committer check logic to include GitHub-assigned committers.
  • scripts/ci/pr-bot/processPrUpdate.ts
    • Updated areReviewersAssigned to fetch PR details and check for pull.requested_reviewers in addition to bot-assigned reviewers.
    • Added a new else if condition to processPrUpdate to explicitly handle the review_requested GitHub action, triggering setNextActionReviewers.
Activity
  • No human activity (comments, reviews, or progress updates) was detected on this pull request.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@kennknowles
Copy link
Member Author

Mildly AI assisted, so feel free to point out areas where I just mis-reviewed the results.

@kennknowles
Copy link
Member Author

/gemini review

@github-actions
Copy link
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@kennknowles
Copy link
Member Author

Also I don't know how to / how best to test, if it is feasible.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request significantly improves the PR bot's ability to handle reviewer assignments and pings by integrating GitHub UI-assigned reviewers. This addresses key limitations mentioned in the PR description, ensuring that the bot's logic is more comprehensive and accurate. The changes correctly incorporate requested_reviewers into various parts of the workflow, such as excluding them from new assignments, including them in slow review pings, and considering them when determining if a committer needs to be assigned. However, there are a couple of areas that could be further refined to enhance correctness and robustness.

({ user: login }) => login !== pullAuthor && login !== BOT_NAME
)
) {
if (comment.user.login !== pullAuthor && comment.user.login !== BOT_NAME) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The fix applied here correctly addresses the logic for reviewComments. However, a similar logical error exists in the preceding loop for issue.listComments (lines 75-83 in the full file). The comments.some check in that loop should be comment.user.login !== pullAuthor && comment.user.login !== BOT_NAME to correctly identify if a specific issue comment was made by a non-author/non-bot. Without this, the bot might incorrectly determine a PR is not slow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is correct, we should apply the change above as well

Comment on lines +218 to +224
if (!labelOfReviewer) {
let isGithubReviewer = false;
if (pull.requested_reviewers && pull.requested_reviewers.some((r: any) => r.login === approver)) isGithubReviewer = true;

if (isGithubReviewer && pull.labels && pull.labels.length > 0) {
labelOfReviewer = pull.labels[0].name;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic introduced to guess a label for a GitHub-assigned reviewer by picking pull.labels[0].name might not always be accurate. If a PR has multiple labels, simply taking the first one could lead to an incorrect association between the reviewer and the PR's content area. This could result in committers being assigned to areas they are not specialized in. Consider a more robust way to infer the label, or explicitly require labels to be present for proper committer assignment.

Copy link
Contributor

@damccorm damccorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM - thanks for contributing!

Also I don't know how to / how best to test, if it is feasible.

Sadly, the answer is just merge the PR. You can add all of this to a fork and try it there, but it is all one-off testing. Merging and monitoring is usually much easier

({ user: login }) => login !== pullAuthor && login !== BOT_NAME
)
) {
if (comment.user.login !== pullAuthor && comment.user.login !== BOT_NAME) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is correct, we should apply the change above as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants