summaryrefslogtreecommitdiffhomepage
path: root/cmd/k8s-operator/deploy
AgeCommit message (Collapse)AuthorFilesLines
2026-01-23all: remove AUTHORS file and references to itWill Norris10-10/+10
This file was never truly necessary and has never actually been used in the history of Tailscale's open source releases. A Brief History of AUTHORS files --- The AUTHORS file was a pattern developed at Google, originally for Chromium, then adopted by Go and a bunch of other projects. The problem was that Chromium originally had a copyright line only recognizing Google as the copyright holder. Because Google (and most open source projects) do not require copyright assignemnt for contributions, each contributor maintains their copyright. Some large corporate contributors then tried to add their own name to the copyright line in the LICENSE file or in file headers. This quickly becomes unwieldy, and puts a tremendous burden on anyone building on top of Chromium, since the license requires that they keep all copyright lines intact. The compromise was to create an AUTHORS file that would list all of the copyright holders. The LICENSE file and source file headers would then include that list by reference, listing the copyright holder as "The Chromium Authors". This also become cumbersome to simply keep the file up to date with a high rate of new contributors. Plus it's not always obvious who the copyright holder is. Sometimes it is the individual making the contribution, but many times it may be their employer. There is no way for the proejct maintainer to know. Eventually, Google changed their policy to no longer recommend trying to keep the AUTHORS file up to date proactively, and instead to only add to it when requested: https://opensource.google/docs/releasing/authors. They are also clear that: > Adding contributors to the AUTHORS file is entirely within the > project's discretion and has no implications for copyright ownership. It was primarily added to appease a small number of large contributors that insisted that they be recognized as copyright holders (which was entirely their right to do). But it's not truly necessary, and not even the most accurate way of identifying contributors and/or copyright holders. In practice, we've never added anyone to our AUTHORS file. It only lists Tailscale, so it's not really serving any purpose. It also causes confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header in other open source repos which don't actually have an AUTHORS file, so it's ambiguous what that means. Instead, we just acknowledge that the contributors to Tailscale (whoever they are) are copyright holders for their individual contributions. We also have the benefit of using the DCO (developercertificate.org) which provides some additional certification of their right to make the contribution. The source file changes were purely mechanical with: git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g' Updates #cleanup Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d Signed-off-by: Will Norris <will@tailscale.com>
2026-01-21cmd/k8s-operator,k8s-operator: Allow the use of multiple tailnets (#18344)David Bond7-0/+345
This commit contains the implementation of multi-tailnet support within the Kubernetes Operator Each of our custom resources now expose the `spec.tailnet` field. This field is a string that must match the name of an existing `Tailnet` resource. A `Tailnet` resource looks like this: ```yaml apiVersion: tailscale.com/v1alpha1 kind: Tailnet metadata: name: example # This is the name that must be referenced by other resources spec: credentials: secretName: example-oauth ``` Each `Tailnet` references a `Secret` resource that contains a set of oauth credentials. This secret must be created in the same namespace as the operator: ```yaml apiVersion: v1 kind: Secret metadata: name: example-oauth # This is the name that's referenced by the Tailnet resource. namespace: tailscale stringData: client_id: "client-id" client_secret: "client-secret" ``` When created, the operator performs a basic check that the oauth client has access to all required scopes. This is done using read actions on devices, keys & services. While this doesn't capture a missing "write" permission, it catches completely missing permissions. Once this check passes, the `Tailnet` moves into a ready state and can be referenced. Attempting to use a `Tailnet` in a non-ready state will stall the deployment of `Connector`s, `ProxyGroup`s and `Recorder`s until the `Tailnet` becomes ready. The `spec.tailnet` field informs the operator that a `Connector`, `ProxyGroup`, or `Recorder` must be given an auth key generated using the specified oauth client. For backwards compatibility, the set of credentials the operator is configured with are considered the default. That is, where `spec.tailnet` is not set, the resource will be deployed in the same tailnet as the operator. Updates https://github.com/tailscale/corp/issues/34561
2026-01-08cmd/k8s-operator/e2e: run self-contained e2e tests with devcontrol (#17415)Tom Proctor3-50/+14
* cmd/k8s-operator/e2e: run self-contained e2e tests with devcontrol Adds orchestration for more of the e2e testing setup requirements to make it easier to run them in CI, but also run them locally in a way that's consistent with CI. Requires running devcontrol, but otherwise supports creating all the scaffolding required to exercise the operator and proxies. Updates tailscale/corp#32085 Change-Id: Ia7bff38af3801fd141ad17452aa5a68b7e724ca6 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com> * cmd/k8s-operator/e2e: being more specific on tmp dir cleanup Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk> --------- Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com> Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk> Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
2025-12-17cmd/k8s-operator: fixes helm template for oauth secret volume mount (#18230)Tom Meadows2-23/+27
Fixes #18228 Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
2025-12-15cmd/k8s-operator: fix statefulset template yaml indentation (#18194)Tom Meadows2-8/+8
Fixes #17000 Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
2025-11-20cmd/k8s-operator: add multi replica support for recorders (#17864)David Bond2-0/+16
This commit adds the `spec.replicas` field to the `Recorder` custom resource that allows for a highly available deployment of `tsrecorder` within a kubernetes cluster. Many changes were required here as the code hard-coded the assumption of a single replica. This has required a few loops, similar to what we do for the `Connector` resource to create auth and state secrets. It was also required to add a check to remove dangling state and auth secrets should the recorder be scaled down. Updates: https://github.com/tailscale/tailscale/issues/17965 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-11-19cmd/k8s-operator: fix type comparison in apiserver proxy template (#17981)Raj Singh1-3/+3
ArgoCD sends boolean values but the template expects strings, causing "incompatible types for comparison" errors. Wrap values with toString so both work. Fixes #17158 Signed-off-by: Raj Singh <raj@tailscale.com>
2025-11-19cmd/k8s-operator: default to stable image (#17848)David Bond2-3/+3
This commit modifies the helm/static manifest configuration for the k8s-operator to prefer the stable image tag. This avoids making those using static manifests seeing unstable behaviour by default if they do not manually make the change. This is managed for us when using helm but not when generating the static manifests. Updates https://github.com/tailscale/tailscale/issues/10655 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-11-07cmd/k8s-operator: support workload identity federationTom Proctor3-8/+41
The feature is currently in private alpha, so requires a tailnet feature flag. Initially focuses on supporting the operator's own auth, because the operator is the only device we maintain that uses static long-lived credentials. All other operator-created devices use single-use auth keys. Testing steps: * Create a cluster with an API server accessible over public internet * kubectl get --raw /.well-known/openid-configuration | jq '.issuer' * Create a federated OAuth client in the Tailscale admin console with: * The issuer from the previous step * Subject claim `system:serviceaccount:tailscale:operator` * Write scopes services, devices:core, auth_keys * Tag tag:k8s-operator * Allow the Tailscale control plane to get the public portion of the ServiceAccount token signing key without authentication: * kubectl create clusterrolebinding oidc-discovery \ --clusterrole=system:service-account-issuer-discovery \ --group=system:unauthenticated * helm install --set oauth.clientId=... --set oauth.audience=... Updates #17457 Change-Id: Ib29c85ba97b093c70b002f4f41793ffc02e6c6e9 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-10-17cmd/k8s-operator: allow pod tolerations on nameservers (#17260)David Bond2-0/+86
This commit modifies the `DNSConfig` custom resource to allow specifying [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) on the nameserver pods. This will allow users to dictate where their nameserver pods are located within their clusters. Fixes: https://github.com/tailscale/tailscale/issues/17092 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-10-02cmd/k8s-operator: add .gitignore for generated chart CRDs (#17406)Tom Proctor1-0/+10
Add a .gitignore for the chart version of the CRDs that we never commit, because the static manifest CRD files are the canonical version. This makes it easier to deploy the CRDs via the helm chart in a way that reflects the production workflow without making the git checkout "dirty". Given that the chart CRDs are ignored, we can also now safely generate them for the kube-generate-all Makefile target without being a nuisance to the state of the git checkout. Added a slightly more robust repo root detection to the generation logic to make sure the command works from the context of both the Makefile and the image builder command we run for releases in corp. Updates tailscale/corp#32085 Change-Id: Id44a4707c183bfaf95a160911ec7a42ffb1a1287 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-09-30cmd/k8s-operator: add DNS policy and config support to ProxyClass (#16887)Raj Singh2-0/+112
DNS configuration support to ProxyClass, allowing users to customize DNS resolution for Tailscale proxy pods. Fixes #16886 Signed-off-by: Raj Singh <raj@tailscale.com>
2025-09-29cmd/k8s-operator: add replica support to nameserver (#17246)David Bond2-0/+10
This commit modifies the `DNSConfig` custom resource to allow specifying a replica count when deploying a nameserver. This allows deploying nameservers in a HA configuration. Updates https://github.com/tailscale/corp/issues/32589 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-09-25k8s-operator: add IPv6 support for DNS records (#16691)Raj Singh2-2/+0
This change adds full IPv6 support to the Kubernetes operator's DNS functionality, enabling dual-stack and IPv6-only cluster support. Fixes #16633 Signed-off-by: Raj Singh <raj@tailscale.com>
2025-09-03cmd/k8s-operator: update connector example (#17020)David Bond1-1/+2
This commit modifies the connector example to use the new hostname prefix and replicas fields Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-09-02cmd/k8s-operator: allow specifying replicas for connectors (#16721)David Bond2-4/+90
This commit adds a `replicas` field to the `Connector` custom resource that allows users to specify the number of desired replicas deployed for their connectors. This allows users to deploy exit nodes, subnet routers and app connectors in a highly available fashion. Fixes #14020 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-07-31cmd/k8s-operator,k8s-operator: allow setting a `priorityClassName` (#16685)Lee Briggs2-0/+12
* cmd/k8s-operator,k8s-operator: allow setting a `priorityClassName` Fixes #16682 Signed-off-by: Lee Briggs <lee@leebriggs.co.uk> * Update k8s-operator/apis/v1alpha1/types_proxyclass.go Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Signed-off-by: Lee Briggs <jaxxstorm@users.noreply.github.com> * run make kube-generate-all Change-Id: I5f8f16694fdc181b048217b9f05ec2ee2aa04def Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com> --------- Signed-off-by: Lee Briggs <lee@leebriggs.co.uk> Signed-off-by: Lee Briggs <jaxxstorm@users.noreply.github.com> Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-07-21cmd/k8s-operator: Allow specifying cluster ips for nameservers (#16477)David Bond2-2/+16
This commit modifies the kubernetes operator's `DNSConfig` resource with the addition of a new field at `nameserver.service.clusterIP`. This field allows users to specify a static in-cluster IP address of the nameserver when deployed. Fixes #14305 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-07-21all-kube: create Tailscale Service for HA kube-apiserver ProxyGroup (#16572)Tom Proctor2-22/+90
Adds a new reconciler for ProxyGroups of type kube-apiserver that will provision a Tailscale Service for each replica to advertise. Adds two new condition types to the ProxyGroup, TailscaleServiceValid and TailscaleServiceConfigured, to post updates on the state of that reconciler in a way that's consistent with the service-pg reconciler. The created Tailscale Service name is configurable via a new ProxyGroup field spec.kubeAPISserver.ServiceName, which expects a string of the form "svc:<dns-label>". Lots of supporting changes were needed to implement this in a way that's consistent with other operator workflows, including: * Pulled containerboot's ensureServicesUnadvertised and certManager into kube/ libraries to be shared with k8s-proxy. Use those in k8s-proxy to aid Service cert sharing between replicas and graceful Service shutdown. * For certManager, add an initial wait to the cert loop to wait until the domain appears in the devices's netmap to avoid a guaranteed error on the first issue attempt when it's quick to start. * Made several methods in ingress-for-pg.go and svc-for-pg.go into functions to share with the new reconciler * Added a Resource struct to the owner refs stored in Tailscale Service annotations to be able to distinguish between Ingress- and ProxyGroup- based Services that need cleaning up in the Tailscale API. * Added a ListVIPServices method to the internal tailscale client to aid cleaning up orphaned Services * Support for reading config from a kube Secret, and partial support for config reloading, to prevent us having to force Pod restarts when config changes. * Fixed up the zap logger so it's possible to set debug log level. Updates #13358 Change-Id: Ia9607441157dd91fb9b6ecbc318eecbef446e116 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-07-09cmd/{k8s-operator,k8s-proxy}: add kube-apiserver ProxyGroup type (#16266)Tom Proctor6-29/+142
Adds a new k8s-proxy command to convert operator's in-process proxy to a separately deployable type of ProxyGroup: kube-apiserver. k8s-proxy reads in a new config file written by the operator, modelled on tailscaled's conffile but with some modifications to ensure multiple versions of the config can co-exist within a file. This should make it much easier to support reading that config file from a Kube Secret with a stable file name. To avoid needing to give the operator ClusterRole{,Binding} permissions, the helm chart now optionally deploys a new static ServiceAccount for the API Server proxy to use if in auth mode. Proxies deployed by kube-apiserver ProxyGroups currently work the same as the operator's in-process proxy. They do not yet leverage Tailscale Services for presenting a single HA DNS name. Updates #13358 Change-Id: Ib6ead69b2173c5e1929f3c13fb48a9a5362195d8 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-07-09cmd/k8s-operator/deploy: clarify helm install notes (#16449)Tom Proctor1-1/+3
Based on feedback that it wasn't clear what the user is meant to do with the output of the last command, clarify that it's an optional command to explore what got created. Updates #13427 Change-Id: Iff64ec6d02dc04bf4bbebf415d7ed1a44e7dd658 Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-07-07cmd/k8s-operator: Allow custom ingress class names (#16472)David Bond4-1/+9
This commit modifies the k8s operator to allow for customisation of the ingress class name via a new `OPERATOR_INGRESS_CLASS_NAME` environment variable. For backwards compatibility, this defaults to `tailscale`. When using helm, a new `ingress.name` value is provided that will set this environment variable and modify the name of the deployed `IngressClass` resource. Fixes https://github.com/tailscale/tailscale/issues/16248 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-07-07cmd/k8s-operator: always set ProxyGroup status conditions (#16429)Tom Proctor2-2/+8
Refactors setting status into its own top-level function to make it easier to ensure we _always_ set the status if it's changed on every reconcile. Previously, it was possible to have stale status if some earlier part of the provision logic failed. Updates #16327 Change-Id: Idab0cfc15ae426cf6914a82f0d37a5cc7845236b Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-07-04cmd/k8s-operator: Move login server value to top-level (#16470)David Bond2-4/+4
This commit modifies the operator helm chart values to bring the newly added `loginServer` field to the top level. We felt as though it was a bit confusing to be at the `operatorConfig` level as this value modifies the behaviour or the operator, api server & all resources that the operator manages. Updates https://github.com/tailscale/corp/issues/29847 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-07-02cmd/k8s-operator: Allow configuration of login server (#16432)David Bond3-0/+7
This commit modifies the kubernetes operator to allow for customisation of the tailscale login url. This provides some data locality for people that want to configure it. This value is set in the `loginServer` helm value and is propagated down to all resources managed by the operator. The only exception to this is recorder nodes, where additional changes are required to support modifying the url. Updates https://github.com/tailscale/corp/issues/29847 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-06-27cmd/k8s-operator, k8s-operator: support Static Endpoints on ProxyGroups (#16115)Tom Meadows4-0/+111
updates: #14674 Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
2025-06-25cmd/k8s-operator: Add NOTES.txt to Helm chart (#16364)David Bond1-0/+25
This commit adds a NOTES.txt to the operator helm chart that will be written to the terminal upon successful installation of the operator. It includes a small list of knowledgebase articles with possible next steps for the actor that installed the operator to the cluster. It also provides possible commands to use for explaining the custom resources. Fixes #13427 Signed-off-by: David Bond <davidsbond93@gmail.com>
2025-05-19cmd/k8s-operator,kube/kubetypes,k8s-operator/apis: reconcile L3 HA Services ↵Tom Meadows2-0/+11
(#15961) This reconciler allows users to make applications highly available at L3 by leveraging Tailscale Virtual Services. Many Kubernetes Service's (irrespective of the cluster they reside in) can be mapped to a Tailscale Virtual Service, allowing access to these Services at L3. Updates #15895 Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
2025-05-19{cmd,}/k8s-operator: support IRSA for Recorder resources (#15913)Tom Proctor2-0/+60
Adds Recorder fields to configure the name and annotations of the ServiceAccount created for and used by its associated StatefulSet. This allows the created Pod to authenticate with AWS without requiring a Secret with static credentials, using AWS' IAM Roles for Service Accounts feature, documented here: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html Fixes #15875 Change-Id: Ib0e15c0dbc357efa4be260e9ae5077bacdcb264f Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2025-04-15k8s-operator: add age column to all custom resources (#15663)Satyam Soni6-0/+30
This change introduces an Age column in the output for all custom resources to enhance visibility into their lifecycle status. Fixes #15499 Signed-off-by: satyampsoni <satyampsoni@gmail.com>
2025-03-28cmd/k8s-operator,k8s-operator: enable HA Ingress again. (#15453)Irbe Krumina2-2/+2
Re-enable HA Ingress again that was disabled for 1.82 release. This reverts commit fea74a60d529bcccbc8ded74644256bb6f6c7727. Updates tailscale/corp#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-03-26cmd/k8s-operator,k8s-operator: disable HA Ingress before stable release (#15433)v1.83.0-preIrbe Krumina2-2/+2
Temporarily make sure that the HA Ingress reconciler does not run, as we do not want to release this to stable just yet. Updates tailscale/corp#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-03-21cmd/k8s-operator,k8s-operator: allow optionally using LE staging endpoint ↵Irbe Krumina2-0/+32
for Ingress (#15360) cmd/k8s-operator,k8s-operator: allow using LE staging endpoint for Ingress Allow to optionally use LetsEncrypt staging endpoint to issue certs for Ingress/HA Ingress, so that it is easier to experiment with initial Ingress setup without hiting rate limits. Updates tailscale/corp#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-03-19cmd/k8s-operator: configure proxies for HA Ingress to run in cert share mode ↵Irbe Krumina2-1/+2
(#15308) cmd/k8s-operator: configure HA Ingress replicas to share certs Creates TLS certs Secret and RBAC that allows HA Ingress replicas to read/write to the Secret. Configures HA Ingress replicas to run in read-only mode. Updates tailscale/corp#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-02-04cmd/k8s-operator: reinstate HA Ingress reconciler (#14887)Irbe Krumina2-2/+2
This change: - reinstates the HA Ingress controller that was disabled for 1.80 release - fixes the API calls to manage VIPServices as the API was changed - triggers the HA Ingress reconciler on ProxyGroup changes Updates tailscale/tailscale#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-01-30cmd/k8s-operator: temporarily disable HA Ingress controller (#14833)Irbe Krumina2-2/+2
The HA Ingress functionality is not actually doing anything valuable yet, so don't run the controller in 1.80 release yet. Updates tailscale/tailscale#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-01-30cmd/k8s-operator: check that cluster traffic is routed to egress ProxyGroup ↵Irbe Krumina2-1/+11
Pod before marking it as ready (#14792) This change builds on top of #14436 to ensure minimum downtime during egress ProxyGroup update rollouts: - adds a readiness gate for ProxyGroup replicas that prevents kubelet from marking the replica Pod as ready before a corresponding readiness condition has been added to the Pod - adds a reconciler that reconciles egress ProxyGroup Pods and, for each that is not ready, if cluster traffic for relevant egress endpoints is routed via this Pod- if so add the readiness condition to allow kubelet to mark the Pod as ready. During the sequenced StatefulSet update rollouts kubelet does not restart a Pod before the previous replica has been updated and marked as ready, so ensuring that a replica is not marked as ready allows to avoid a temporary post-update situation where all replicas have been restarted, but none of the new ones are yet set up as an endpoint for the egress service, so cluster traffic is dropped. Updates tailscale/tailscale#14326 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-01-17go.{mod,sum},cmd/{k8s-operator,derper,stund}/depaware.txt: bump kube deps ↵Irbe Krumina6-72/+252
(#14601) Updates kube deps and mkctr, regenerates kube yamls with the updated tooling. Updates#cleanup Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-01-09cmd/k8s-operator,k8s-operator: allow users to set custom labels for the ↵Irbe Krumina2-0/+28
optional ServiceMonitor (#14475) * cmd/k8s-operator,k8s-operator: allow users to set custom labels for the optional ServiceMonitor Updates tailscale/tailscale#14381 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2025-01-08cmd/k8s-operator,k8s-operator: support ingress ProxyGroup type (#14548)Irbe Krumina2-2/+24
Currently this does not yet do anything apart from creating the ProxyGroup resources like StatefulSet. Updates tailscale/corp#24795 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-12-20cmd/k8s-operator,k8s-operator: include top-level CRD descriptions (#14435)Tom Proctor3-0/+34
When reading https://doc.crds.dev/github.com/tailscale/tailscale/tailscale.com/ProxyGroup/v1alpha1@v1.78.3 I noticed there is no top-level description for ProxyGroup and Recorder. Add one to give some high-level direction. Updates #cleanup Change-Id: I3666c5445be272ea5a1d4d02b6d5ad4c23afb09f Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2024-12-03cmd/k8s-operator/deploy/chart: allow reading OAuth creds from a CSI driver's ↵Oliver Rahner3-4/+30
volume and annotating operator's Service account (#14264) cmd/k8s-operator/deploy/chart: allow reading OAuth creds from a CSI driver's volume and annotating operator's Service account Updates #14264 Signed-off-by: Oliver Rahner <o.rahner@dke-data.com>
2024-12-03cmd/k8s-operator,k8s-operator,go.mod: optionally create ServiceMonitor (#14248)Irbe Krumina3-0/+69
* cmd/k8s-operator,k8s-operator,go.mod: optionally create ServiceMonitor Adds a new spec.metrics.serviceMonitor field to ProxyClass. If that's set to true (and metrics are enabled), the operator will create a Prometheus ServiceMonitor for each proxy to which the ProxyClass applies. Additionally, create a metrics Service for each proxy that has metrics enabled. Updates tailscale/tailscale#11292 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-12-03cmd/k8s-operator,docs/k8s: run tun mode proxies in privileged containers ↵Irbe Krumina3-23/+25
(#14262) We were previously relying on unintended behaviour by runc where all containers where by default given read/write/mknod permissions for tun devices. This behaviour was removed in https://github.com/opencontainers/runc/pull/3468 and released in runc 1.2. Containerd container runtime, used by Docker and majority of Kubernetes distributions bumped runc to 1.2 in 1.7.24 https://github.com/containerd/containerd/releases/tag/v1.7.24 thus breaking our reference tun mode Tailscale Kubernetes manifests and Kubernetes operator proxies. This PR changes the all Kubernetes container configs that run Tailscale in tun mode to privileged. This should not be a breaking change because all these containers would run in a Pod that already has a privileged init container. Updates tailscale/tailscale#14256 Updates tailscale/tailscale#10814 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-11-22cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user ↵Tom Proctor2-2/+88
metrics (#14035) containerboot: Adds 3 new environment variables for containerboot, `TS_LOCAL_ADDR_PORT` (default `"${POD_IP}:9002"`), `TS_METRICS_ENABLED` (default `false`), and `TS_DEBUG_ADDR_PORT` (default `""`), to configure metrics and debug endpoints. In a follow-up PR, the health check endpoint will be updated to use the `TS_LOCAL_ADDR_PORT` if `TS_HEALTHCHECK_ADDR_PORT` hasn't been set. Users previously only had access to internal debug metrics (which are unstable and not recommended) via passing the `--debug` flag to tailscaled, but can now set `TS_METRICS_ENABLED=true` to expose the stable metrics documented at https://tailscale.com/kb/1482/client-metrics at `/metrics` on the addr/port specified by `TS_LOCAL_ADDR_PORT`. Users can also now configure a debug endpoint more directly via the `TS_DEBUG_ADDR_PORT` environment variable. This is not recommended for production use, but exposes an internal set of debug metrics and pprof endpoints. operator: The `ProxyClass` CRD's `.spec.metrics.enable` field now enables serving the stable user metrics documented at https://tailscale.com/kb/1482/client-metrics at `/metrics` on the same "metrics" container port that debug metrics were previously served on. To smooth the transition for anyone relying on the way the operator previously consumed this field, we also _temporarily_ serve tailscaled's internal debug metrics on the same `/debug/metrics` path as before, until 1.82.0 when debug metrics will be turned off by default even if `.spec.metrics.enable` is set. At that point, anyone who wishes to continue using the internal debug metrics (not recommended) will need to set the new `ProxyClass` field `.spec.statefulSet.pod.tailscaleContainer.debug.enable`. Users who wish to opt out of the transitional behaviour, where enabling `.spec.metrics.enable` also enables debug metrics, can set `.spec.statefulSet.pod.tailscaleContainer.debug.enable` to false (recommended). Separately but related, the operator will no longer specify a host port for the "metrics" container port definition. This caused scheduling conflicts when k8s needs to schedule more than one proxy per node, and was not necessary for allowing the pod's port to be exposed to prometheus scrapers. Updates #11292 --------- Co-authored-by: Kristoffer Dalby <kristoffer@tailscale.com> Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2024-11-22cmd/k8s-operator/deploy: ensure that operator can write kube state Events ↵Irbe Krumina2-0/+16
(#14177) A small follow-up to #14112- ensures that the operator itself can emit Events for its kube state store changes. Updates tailscale/tailscale#14080 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-11-20Make the deployment of an IngressClass optional, default to true (#14153)James Stocker2-0/+5
Fixes tailscale/tailscale#14152 Signed-off-by: James Stocker jamesrstocker@gmail.com Co-authored-by: James Stocker <james.stocker@intenthq.co.uk>
2024-11-19kube/{kubeapi,kubeclient},ipn/store/kubestore,cmd/{containerboot,k8s-operato ↵Irbe Krumina4-0/+27
r}: emit kube store Events (#14112) Adds functionality to kube client to emit Events. Updates kube store to emit Events when tailscaled state has been loaded, updated or if any errors where encountered during those operations. This should help in cases where an error related to state loading/updating caused the Pod to crash in a loop- unlike logs of the originally failed container instance, Events associated with the Pod will still be accessible even after N restarts. Updates tailscale/tailscale#14080 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-11-11cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app ↵Irbe Krumina2-12/+94
connector via Connector spec (#13950) * cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app connector via Connector spec Updates tailscale/tailscale#11113 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2024-10-30cmd/k8s-operator,k8s-operator: add topology spread constraints to ProxyClass ↵Irbe Krumina2-0/+352
(#13959) Now when we have HA for egress proxies, it makes sense to support topology spread constraints that would allow users to define more complex topologies of how proxy Pods need to be deployed in relation with other Pods/across regions etc. Updates tailscale/tailscale#13406 Signed-off-by: Irbe Krumina <irbe@tailscale.com>