Description

Observed Behavior:
BLUF: Karpenter's node overlay performs a Deep Copy of instance type with offering every time when returning instance type with Node Overlay applied which significantly increase memory. https://github.com/kubernetes-sigs/karpenter/blob/main/pkg/controllers/nodeoverlay/store.go#L135

I see a huge increase in memory when enabling Node Overlay.

Note the red box, when enabling node overlay we saw almost a 4x increase in memory (from 2GB - 8GB spikes)

When Karpenter is returning instance types from GetInstanceType causing memory to blow up.

** What is happening **:
Karpenter builts an update dictionary which applies itself against the underlying instance types return from cloud provider instance type. https://github.com/kubernetes-sigs/karpenter/blob/main/pkg/controllers/nodeoverlay/store.go#L121-L148. However doing this, Karpenter has to perform a deep copy of each instance type for every node pool. This effectively multiples the size of offering + instance type cache * number of node pool * each invocation of GetInstanceType.

When user creates a broadly applicable Node Overlay or to be specific create a updates cache that has many node pools and many instance types, Karpenter will suffer heavy memory usage during its normal operation.
Before overlay:

After overlay:

Expected Behavior:
Karpenter shouldn't scale so poorly.

Reproduction Steps (Please include YAML):

• Create 1 node pool with very constrained instance types.
• Measure memory
• Create a very permissive Node Overlay
• Measure Memory

Node Pool instance type related requirements

    - key: karpenter.k8s.aws/instance-size
        operator: NotIn
        values:
        - metal
    - key: karpenter.k8s.aws/instance-size
        operator: In
        values:
        - 12xlarge
        - 16xlarge
        - 18xlarge
        - 24xlarge
        - 2xlarge
        - 32xlarge
        - 3xlarge
        - 4xlarge
        - 6xlarge
        - 8xlarge
        - 9xlarge
        - large
        - xlarge
      - key: karpenter.k8s.aws/instance-category
        operator: In
        values:
        - m
        - r
        - c
      - key: karpenter.k8s.aws/instance-generation
        operator: In
        values:
        - "6"
        - "7"

Node Overlay

    - key: karpenter.sh/capacity-type
      operator: In
      values:
      - spot

Versions:

• Kubernetes Version (kubectl version): 1.8.2 AWS Karpenter provider

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment