Skip to content

K8s plugin: Support custom CNI manifest - #2254

Merged
katiewasnothere merged 2 commits into
apple:mainfrom
katiewasnothere:k8s_support_other_cnis
Sep 15, 2026
Merged

katiewasnothere merged 2 commits into
apple:mainfrom
katiewasnothere:k8s_support_other_cnis

Conversation

@katiewasnothere

@katiewasnothere katiewasnothere commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Type of Change

  • New feature

Motivation and Context

This PR adds a new flag to the k8s create command to allow users to specify an optional path to a CNI manifest that they wish to use for the cluster instead of the bundled kindnet CNI manifest.

I tested this with cilium by doing the following:

% helm template cilium cilium/cilium \
    --version 1.20.1 \
    --namespace kube-system \
    --set ipam.mode=cluster-pool \
    --set ipam.operator.clusterPoolIPv4PodCIDRList='{<TARGETCIDR>}' \
    --set kubeProxyReplacement=false > /tmp/cilium.yaml
% container k8s create --cni /tmp/cilium.yaml

Testing

  • Tested locally
  • Added/updated tests
  • Added/updated docs

Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Code Coverage

Tier Line Coverage
Unit 24.85%
Integration 66.64%
Combined 76.06%

@egernst

egernst commented Sep 10, 2026

Copy link
Copy Markdown
Contributor
  • Consider adding the example usage in the PR description?
  • Can you add example to the kubernetes.md?

@egernst egernst left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool -- looks good Kathryn. Thanks for the updated docs/example

@egernst
egernst self-requested a review September 14, 2026 23:39
+ "\(kubeconfigEnv) kubectl apply -f /tmp/kindnet.yaml"
log.info("Applying CNI manifest", metadata: ["node": "\(nodeID)"])
let manifest = try await loadCNIManifest(path: cniManifestPath, log: log)
let apply = "\(kubeconfigEnv) kubectl apply -f - <<'EOF'\n\(manifest)\nEOF"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can things get weird here if the user submits a yaml that "accidentally" includes EOF? Should we feed it a real stdin, or base64-encoding the payload, or?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think a particularly large manifest may hit E2BIG - I think this would also be solved if we considered a pipe?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what the right answer for this is from a UX perspective. I'd like to investigate that in a separate PR if that's okay with you.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think the UX would be the same, just a matter of how we handle it. I'm fine doing this as a follow up. #2273


static func loadCNIManifest(path: String?, log: Logger) async throws -> String {
if let path {
guard let contents = try? String(contentsOfFile: path, encoding: .utf8) else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we do/catch so we can print out the actual error (could be a read error as well?)

Comment thread docs/kubernetes.md Resolved
}
}

@Test func customPathTakesPrecedenceOverBundledCNI() async throws {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is effectively covering the same as customPathReturnsItsContents().

A local friend of mine had this suggestion:

Line 76 already asserted result == contents, and contents is the literal "custom-cni-marker". So by the time you reach line 77, result is known to be that string, which self-evidently doesn't contain "kindnet". The assertion is true as a consequence of the previous line — it has no independent power to catch anything.

Signed-off-by: Kathryn Baldauf <k_baldauf@apple.com>
@katiewasnothere
katiewasnothere merged commit 57f0b93 into apple:main Sep 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants