Skip to content

AWS: improve interactions with AWS API #40443

@marseel

Description

@marseel

I've been talking offline a little bit with @antonipp about #39543 and newly introduced flag --aws-pagination-enabled

To summarize our discussion:

  • In case of listing interfaces, there is no clear number/limit of interfaces at which DescribeNetworkInterfaces fails with error OperationNotPermitted for calls without pagination.
  • Paginated results are usually significantly slower + they use API more heavily, potentially resulting in more throttling

Proposed improvement:

Instead of always using paginated or unpaginated API calls, let's always try unpaginated request and only if it fails (for any reason), try paginated api calls.

This has following benefits:

  • In happy case (unpaginated call works) it's fast
  • No need to have additional flag - easier from operational point of view
  • Paginated api calls sometimes do not have a predictable performance: operator: add flag to control AWS API pagination #39543 (comment)
  • Unpaginated API Calls for interfaces should work almost always if subnet-id/subnet-tags is specified (to be verified)
    And small drawback:
  • If unpaginated call didn't work, we did performed only one unnecessary API call out of tens of paginated API calls

This sounds like a good trade off to me, but I would appreciate others' feedback as well.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/eniImpacts ENI based IPAM.help-wantedPlease volunteer for this by adding yourself as an assignee!integration/cloudRelated to integration with cloud environments such as AKS, EKS, GKE, etc.sig/scalabilityImpacts how well Cilium handles a high rate of events or churn.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions