Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] OCPBUGS-32250: Prevent UniqueHost from stopping plugin chain #574

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

gcs278
Copy link
Contributor

@gcs278 gcs278 commented Apr 15, 2024

Fix UniqueHost so that it doesn't completely stop the plugin chain if the route is not "activated" because the the ResourceVersion on the route didn't change. This is a problem for resyncs as the ResourceVersion doesn't change, but the plugin change should continue.

WIP because I need to test more, and possibly develop E2Es to reproduce this scenario.

@openshift-ci-robot openshift-ci-robot added the jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. label Apr 15, 2024
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 15, 2024
@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 15, 2024
@openshift-ci-robot
Copy link
Contributor

@gcs278: This pull request references Jira Issue OCPBUGS-32250, which is invalid:

  • expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Fix UniqueHost so that it doesn't completely stop the plugin chain if the route is not "activated" because the the ResourceVersion on the route didn't change. This is a problem for resyncs as the ResourceVersion doesn't change, but the plugin change should continue.

WIP because I need to test more, and possibly develop E2Es to reproduce this scenario.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Apr 15, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from gcs278. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gcs278 gcs278 changed the title [WIP] OCPBUGS-32250: Prvent UniqueHost from stopping plugin chain [WIP] OCPBUGS-32250: Prevent UniqueHost from stopping plugin chain Apr 15, 2024
Remove the ResourceVersion check in the UniqueHost Plugin to ensure it
"activates" routes even when there are no changes. If routes aren't
"activated", the UniqueHost Plugin won't call downstream plugins.
If the downstream plugins aren't called, then the route status won't
have a chance to resync.

Resyncing route status is vital for after periods of status contention,
which can occur when rolling out new versions of the router. If the
new router loses the status update battle to the old router and the old
router is subsequently terminated, the new router needs to resync the
status to achieve the right state.

Removing this ResourceVersion check should only impact resyncs, as
routes are only reconciled without changes during resyncs, except at
initial startup.
@gcs278
Copy link
Contributor Author

gcs278 commented Jun 25, 2024

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 25, 2024
@openshift-ci-robot
Copy link
Contributor

@gcs278: This pull request references Jira Issue OCPBUGS-32250, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.17.0) matches configured target version for branch (4.17.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @lihongan

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from lihongan June 25, 2024 15:00
@lihongan
Copy link
Contributor

/jira refresh

@openshift-ci-robot
Copy link
Contributor

@lihongan: This pull request references Jira Issue OCPBUGS-32250, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.17.0) matches configured target version for branch (4.17.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @ShudiLi

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from ShudiLi June 26, 2024 03:50
@ShudiLi
Copy link
Member

ShudiLi commented Jun 28, 2024

/test e2e-metal-ipi-ovn-ipv6

@ShudiLi
Copy link
Member

ShudiLi commented Jun 28, 2024

/label qe-approved
tested it with 4.17.0-0.ci.test-2024-06-28-012349-ci-ln-34x60yk-latest

1.
% oc get clusterversion
NAME      VERSION                                                   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.17.0-0.ci.test-2024-06-28-012349-ci-ln-34x60yk-latest   True        False         10m     Cluster version is 4.17.0-0.ci.test-2024-06-28-012349-ci-ln-34x60yk-latest

2.
% oc -n openshift-ingress get pod router-default-7897f79b-5fbsr -oyaml | grep -i image
    image: registry.build02.ci.openshift.org/ci-ln-34x60yk/stable@sha256:66d3c2205cf42b2e2c03b4dcd0ef4d4d4e2575a17e1edce616402d72cf419ca1

3. replaced the router image in router-contention-resync-test.sh
4.
% ./router-contention-resync-test.sh 
deployment.apps/router-contention1 created
deployment.apps/router-contention2 created
Waiting for router-contention1 deployment to rollout
deployment "router-contention1" successfully rolled out
Waiting for router-contention2 deployment to rollout
deployment "router-contention2" successfully rolled out
Creating route
route.route.openshift.io/route-contention created
Waiting for the router to get a status...
Contention host: route-contention-default.router-contention2.local
Deleting router-contention2...
deployment.apps "router-contention2" deleted
waiting for router to resync, and the status to change to router-contention1
waiting for router to resync, and the status to change to router-contention1
waiting for router to resync, and the status to change to router-contention1
...
waiting for router to resync, and the status to change to router-contention1
Success: Found status changed to router-contention1, no bug
% 

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Jun 28, 2024
@ShudiLi
Copy link
Member

ShudiLi commented Jul 2, 2024

/test e2e-metal-ipi-ovn-ipv6

Copy link
Contributor

openshift-ci bot commented Jul 2, 2024

@gcs278: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-bot
Copy link
Contributor

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

@openshift-ci openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 1, 2024
@gcs278
Copy link
Contributor Author

gcs278 commented Oct 7, 2024

/remove-lifecycle stale

@openshift-ci openshift-ci bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants