It is possible to run the api-syncagent component without running a fully fleged KDP installation. All that is necessary is a running kcp installation.
Before installing the api-syncagent it is necessary to create an APIExport on kcp. This is
automatically generated by KDP if you create a Service object.
The APIExport should be empty, because it is updated later by the api-syncagent. An example file
could look like this:
apiVersion: apis.kcp.io/v1alpha1
kind: APIExport
metadata:
name: test.kubermatic.io
spec: {}
Create a file with a similar content (you most likely want to change the name, as that is the API group under which your published resources will be made available) and create it in a kcp workspace of your choice:
# use the kcp kubeconfig
$ export KUBECONFIG=/path/to/kcp.kubeconfig
# nativagate to the workspace where the APIExport should exist
$ kubectl ws :workspace:you:want:to:create:it
# create it
$ kubectl create --filename apiexport.yaml
apiexport/test.example.com created
The Sync Agent can be installed into any namespace, but in our example we are going with kcp-system.
It doesn’t necessarily have to live in the same Kubernetes cluster where it is synchronizing data
to, but that is the common setup. Ultimately the Sync Agent synchronizes data between two kube
endpoints.
Now that the APIExport is created, switch to the Kubernetes cluster from which you wish to
publish resources. You will need to ensure that a kubeconfig with access to
the kcp workspace that the APIExport has been created in is stored as a Secret on this cluster.
Make sure that the kubeconfig points to the right workspace (not necessarily the root workspace).
This can be done via a command like this:
$ kubectl create secret generic kcp-kubeconfig \
--namespace kcp-system \
--from-file "kubeconfig=admin.kubeconfig"
The Sync Agent is shipped as a Helm chart and to install it, the next step is preparing a values.yaml
file for the Sync Agent Helm chart. We need to pass the target APIExport, a name for the Sync Agent
itself and a reference to the kubeconfig secret we just created.
# Required: the name of the APIExport in kcp that this Sync Agent is supposed to serve.
apiExportName: test.example.com
# Required: This Agent's public name, will be shown in the KDP Platform and used to signal ownership over locally synced objects.
# This value must be a valid Kubernetes label value, see
# https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set
# for more information.
# Changing this value after the fact will make the agent ignore previously created objects,
# so beware and relabel if necessary.
agentName: unique-test
# Required: Name of the Kubernetes Secret that contains a "kubeconfig" key,
# with the kubeconfig provided by kcp to access it.
kcpKubeconfig: kcp-kubeconfig
Once this values.yaml file is prepared, install a recent development build of the Sync Agent:
helm repo add kcp https://kcp-dev.github.io/helm-charts
helm repo update
helm install kcp-api-syncagent kcp/api-syncagent \
--values values.yaml \
--namespace kcp-system
Two kcp-api-syncagent Pods should start in the kcp-system namespace. If they crash you will need to
identify the reason from container logs. A possible issue is that the provided kubeconfig does not
have permissions against the target kcp workspace.
The Sync Agent usually requires additional RBAC on the service cluster to function properly. The Helm chart will automatically allow it to read CRDs, namespaces and Secrets, but depending on how you configure your PublishedResources, additional permissions need to be created.
For example, if the Sync Agent is meant to create Certificate objects (defined by cert-manager),
you would need to grant it permissions on those:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: 'api-syncagent:unique-test'
rules:
- apiGroups:
- cert-manager.io
resources:
- certificates
verbs:
- get
- list
- watch
- create
- update
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: 'api-syncagent:unique-test'
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: 'api-syncagent:unique-test'
subjects:
- kind: ServiceAccount
name: 'kcp-api-syncagent'
namespace: kcp-system
NB: Even though the PublishedResources might only create/update Certificates in a single namespace,
due to the inner workings of the Agent they will still be watched (cached) cluster-wide. So you can
tighten permissions on create/update operations to certain namespaces, but watch permissions
need to be granted cluster-wide.
The Helm chart is installed on the service cluster and so cannot provision the necessary RBAC for
the Sync Agent within kcp. Usually whoever creates the APIExport is also responsible for creating
the RBAC rules that grant the Agent access.
The Sync Agent needs to
APIExport,APIResourceSchemas andAPIExport.This can be achieved by applying RBAC like this in the workspace where the APIExport resides:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: api-syncagent-mango
rules:
# manage its APIExport
- apiGroups:
- apis.kcp.io
resources:
- apiexports
resourceNames:
- test.example.com
verbs:
- get
- list
- watch
- patch
- update
# manage APIResourceSchemas
- apiGroups:
- apis.kcp.io
resources:
- apiresourceschemas
verbs:
- get
- list
- watch
- create
# access the virtual workspace
- apiGroups:
- apis.kcp.io
resources:
- apiexports/content
resourceNames:
- test.example.com
verbs:
- '*'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: api-syncagent-columbo:mango-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: api-syncagent-mango
subjects:
- kind: User
name: api-syncagent-mango
Two api-syncagent Pods should start in the kdp-system namespace. If they crash you will need to
identify the reason from container logs. A possible issue is that the provided kubeconfig does not
have permissions against the target kcp workspace.
Once the api-syncagent Pods are up and running, you should be able to follow Publishing Resources. Be aware that the remaining KDP documentation assumes you are working on a KDP installation, and as such might show you screenshots of the KDP dashboard. Such a UI to manage resources does not exist if you use the api-syncagent without, and you will need to create objects from the command line directly.
Once resources have been published through the api-syncagent, they can be consumed on the kcp side (i.e. objects on kcp will be synced back and forth with the service cluster). Follow the manual steps to consume services.