ci/gitops with oci artifact, github actions, google artifact registry and config sync
Since Anthos Config Management 1.13.0, you can now deploy OCI artifacts and Helm charts the GitOps way with Config Sync.
In this article, we will show how you can package and push an OCI artifact to Google Artifact Registry with GitHub actions (using Workload Identity Federation) and oras
, and then how you can deploy this OCI artifact with Config Sync (using Workload Identity).
Objectives
- Set up Workload Identity Federation with a dedicated Google Service Account (Artifact Registry writer)
- Create a Google Artifact Registry repository
- Package and push an OCI artifact in Google Artifact Registry with GitHub actions (using Workload Identity Federation) and
oras
- Create a GKE cluster and enable Config Sync
- Set up Workload Identity with a dedicated Google Service Account (Artifact Registry reader)
- Sync an OCI artifact from Google Artifact Registry with Config Sync (using Workload Identity)
Costs
This tutorial uses billable components of Google Cloud, including the following:
Note: In this case, Config Sync is free, see more details here.
Use the pricing calculator to generate a cost estimate based on your projected usage.
Before you begin
This guide assumes that you have owner IAM permissions for your Google Cloud project. In production, you do not require owner permission.
Verify that billing is enabled for your project.
This guide also assumes that you have a GitHub account.
Set up your environment
Here are the tools you will need:
Note: You can use the Google Cloud Shell which has all these tools already installed.
Initialize the common variables used throughout this tutorial:
PROJECT_ID=FIXME-WITH-YOUR-PROJECT-ID
REGION=us-east4
ZONE=us-east4-a
To avoid repeating the --project
in the commands throughout this tutorial, let’s set the current project:
gcloud config set project ${PROJECT_ID}
Use the gh
tool to create this GitHub repository:
REPO_NAME=my-oci-artifact
cd ~/
gh auth login
git config --global init.defaultBranch main
gh repo create ${REPO_NAME} --private --clone
Let’s capture the GitHub owner value that you will reuse later in this tutorial:
GITHUB_REPO_OWNER=$(gh repo view ${REPO_NAME} --json owner --jq .owner.login)
Set up Workload Identity Federation
Workload Identity Federation lets you access resources directly, using a short-lived access token, and eliminates the maintenance and security burden associated with service account keys.
Create a dedicated Google Service Account which will push the OCI artifact in Artifact Registry later:
PACKAGER_GSA_NAME=oci-artifacts-packager
gcloud iam service-accounts create ${PACKAGER_GSA_NAME}
Create a Workload Identity Pool:
WI_POOL_NAME=oci-artifacts-packager-wi-pool
gcloud iam workload-identity-pools create ${WI_POOL_NAME} \
--location global \
--display-name ${WI_POOL_NAME}
WI_POOL_ID=$(gcloud iam workload-identity-pools describe ${WI_POOL_NAME} \
--location global \
--format='get(name)')
Create a Workload Identity Provider with GitHub actions in that pool:
gcloud iam workload-identity-pools providers create-oidc ${WI_POOL_NAME} \
--location global \
--workload-identity-pool ${WI_POOL_NAME} \
--display-name ${WI_POOL_NAME} \
--attribute-mapping "google.subject=assertion.repository,attribute.actor=assertion.actor,attribute.aud=assertion.aud,attribute.repository=assertion.repository" \
--issuer-uri "https://token.actions.githubusercontent.com"
WI_POOL_PROVIDER_ID=$(gcloud iam workload-identity-pools providers describe ${WI_POOL_NAME} \
--location global \
--workload-identity-pool ${WI_POOL_NAME} \
--format='get(name)')
Allow authentications from the Workload Identity Provider to impersonate the Service Account created above:
gcloud iam service-accounts add-iam-policy-binding ${PACKAGER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com \
--role "roles/iam.workloadIdentityUser" \
--member "principalSet://iam.googleapis.com/${WI_POOL_ID}/attribute.repository/${GITHUB_REPO_OWNER}/${REPO_NAME}"
Set up the Artifact Registry repository
Create a Google Artifact Registry repository:
gcloud services enable artifactregistry.googleapis.com
ARTIFACT_REGISTRY_REPOSITORY=oci-artifacts
gcloud artifacts repositories create ${ARTIFACT_REGISTRY_REPOSITORY} \
--location ${REGION} \
--repository-format docker
Allow the Google Service Account to push in Artifact Registry:
gcloud artifacts repositories add-iam-policy-binding ${ARTIFACT_REGISTRY_REPOSITORY} \
--location ${REGION} \
--member "serviceAccount:${PACKAGER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
--role roles/artifactregistry.writer
Package and push an OCI artifact in Google Artifact Registry
Create the Namespace
resource:
cd ~/${REPO_NAME}
cat <<EOF> test-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: test
EOF
Commit this Namespace
resource in the GitHub repository:
git add . && git commit -m "Create Namespace resource" && git push origin main
Set the environment variables as secrets for the GitHub actions pipeline:
gh secret set PROJECT_ID -b"${PROJECT_ID}"
gh secret set ARTIFACT_REGISTRY_REPOSITORY -b"${ARTIFACT_REGISTRY_REPOSITORY}"
gh secret set ARTIFACT_REGISTRY_HOST_NAME -b"${REGION}-docker.pkg.dev"
gh secret set PACKAGER_GSA_ID -b"${PACKAGER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
gh secret set WI_POOL_PROVIDER_ID -b"${WI_POOL_PROVIDER_ID}"
Define a GitHub actions pipeline to package and push the OCI artifact in Google Artifact Registry using oras
:
mkdir .github && mkdir .github/workflows
cat <<'EOF' > .github/workflows/ci-oci-gar.yaml
name: ci-oci-gar
permissions:
contents: read
id-token: write
on:
push:
branches:
- main
pull_request:
env:
PACKAGE_NAME: my-oci-artifact
IMAGE_TAG: 0.1.0
jobs:
job:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: google-github-actions/auth@v0
with:
workload_identity_provider: '${{ secrets.WI_POOL_PROVIDER_ID }}'
service_account: '${{ secrets.PACKAGER_GSA_ID }}'
token_format: 'access_token'
- uses: google-github-actions/setup-gcloud@v0
with:
version: latest
- name: login to artifact registry
run: |
gcloud auth configure-docker ${{ secrets.ARTIFACT_REGISTRY_HOST_NAME }} --quiet
- name: package
run: |
tar -cf $PACKAGE_NAME.tar $(ls *.yaml)
- name: oras push
if: ${{ github.event_name == 'push' }}
run: |
oras push oci://${{ secrets.ARTIFACT_REGISTRY_HOST_NAME }}/${{ secrets.PROJECT_ID }}/${{ secrets.ARTIFACT_REGISTRY_REPOSITORY }}/$PACKAGE_NAME:$IMAGE_TAG $PACKAGE_NAME.tar
EOF
This GitHub Actions pipeline allows to execute a series of commands: gcloud auth configure-docker
, tar
and eventually, if it’s a push
in main
branch, oras push
will be executed. Also, this pipeline is triggered as soon as there is a push
in main
branch as well as for any pull requests. You can adapt this flow and these conditions for your own needs.
You can see that we use the google-github-actions/auth
action to establish authentication to Google Cloud using Workload Identity Federation. To make this action working we need to have permissions.id-token: write
. Then gcloud auth configure-docker
allows to authenticate against the Artifact Registry, the next oras push
command will reuse this authentication mechanism.
That’s also great to see that oras
is installed by default on the GitHub actions runners (ubuntu-latest
used here).
Commit this GitHub actions pipeline in the GitHub repository:
git add . && git commit -m "Create GitHub actions pipeline" && git push origin main
Wait until the associated run is successfully completed:
gh run list
See that your OCI artifact has been uploaded in the Google Artifact Registry repository:
gcloud artifacts docker images list ${REGION}-docker.pkg.dev/${PROJECT_ID}/$ARTIFACT_REGISTRY_REPOSITORY/${REPO_NAME}
Now that we have built and store our OCI artifact, let’s provision the GKE cluster with Config Sync ready to eventually deploy this OCI artifact.
Create your GKE cluster and enable Config Sync
Create a GKE cluster registered in a Fleet to enable Config Management:
gcloud services enable container.googleapis.com
CLUSTER_NAME=cluster-oci-test
gcloud container clusters create ${CLUSTER_NAME} \
--workload-pool=${PROJECT_ID}.svc.id.goog \
--zone ${ZONE}
gcloud services enable gkehub.googleapis.com
gcloud container fleet memberships register ${CLUSTER_NAME} \
--gke-cluster ${ZONE}/${CLUSTER_NAME} \
--enable-workload-identity
gcloud beta container fleet config-management enable
Install Config Sync in this GKE cluster:
cat <<EOF > acm-config.yaml
applySpecVersion: 1
spec:
configSync:
enabled: true
EOF
gcloud beta container fleet config-management apply \
--membership ${CLUSTER_NAME} \
--config acm-config.yaml
Sync an OCI artifact from Google Artifact Registry
Create a dedicated Google Cloud Service Account with the fine granular access to that Artifact Registry repository with the roles/artifactregistry.reader
role:
ARTIFACTS_READER_GSA_NAME=oci-artifacts-reader
gcloud iam service-accounts create ${ARTIFACTS_READER_GSA_NAME} \
--display-name ${ARTIFACTS_READER_GSA_NAME}
gcloud artifacts repositories add-iam-policy-binding ${ARTIFACT_REGISTRY_REPOSITORY} \
--location ${REGION} \
--member "serviceAccount:${ARTIFACTS_READER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
--role roles/artifactregistry.reader
Allow Config Sync to synchronize resources for a specific RootSync
via Workload Identity:
ROOT_SYNC_NAME=root-sync-oci
gcloud iam service-accounts add-iam-policy-binding \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:${PROJECT_ID}.svc.id.goog[config-management-system/root-reconciler-${ROOT_SYNC_NAME}]" \
${ARTIFACTS_READER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
Deploy the RootSync
in order to sync the private OCI artifact:
cat << EOF | kubectl apply -f -
apiVersion: configsync.gke.io/v1beta1
kind: RootSync
metadata:
name: ${ROOT_SYNC_NAME}
namespace: config-management-system
spec:
sourceFormat: unstructured
sourceType: oci
oci:
image: ${REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REGISTRY_REPOSITORY}/my-oci-artifact:0.1.0
dir: .
auth: gcpserviceaccount
gcpServiceAccountEmail: ${ARTIFACTS_READER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
EOF
Note that we set the spec.oci.auth: gcpserviceaccount
and spec.oci.gcpServiceAccountEmail: ${ARTIFACTS_READER_GSA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com
values to be able to access and sync the private OCI artifact.
Check the status of the sync:
nomos status \
--contexts=$(kubectl config current-context)
Verify that the Namespace
is synced:
kubectl get ns test
And voilà! You just deployed a private OCI artifact hosted in Google Artifact Registry with Config Sync.
Conclusion
In this article, you were able to package and push an OCI artifact in Google Artifact Registry thanks to oras
with GitHub Actions using Workload Identity Federation. At the end, you saw how you can sync a private OCI artifact with the spec.oci.auth: gcpserviceaccount
setup on the RootSync
using Workload Identity. This demonstrates that both GitHub Actions and Config Sync support an highly secured (key-less) approach to connect to Google Artifact Registry.
Cleaning up
To avoid incurring charges to your Google Cloud account, you can delete the resources used in this tutorial.
Unregister the GKE cluster from the Fleet:
gcloud container fleet memberships unregister ${CLUSTER_NAME} \
--project=${PROJECT_ID} \
--gke-cluster=${ZONE}/${CLUSTER_NAME}
Delete the GKE cluster:
gcloud container clusters delete ${CLUSTER_NAME} \
--zone ${ZONE}
Delete the Artifact Registry repository:
gcloud artifacts repositories delete ${ARTIFACT_REGISTRY_REPOSITORY} \
--location ${REGION}
What’s next
- Deploy OCI artifacts and Helm charts the GitOps way with Config Sync
- Sync Helm charts from Artifact Registry
- Sync OCI artifacts from Artifact Registry
- CI/GitOps with Helm, GitHub Actions, GitHub Container Registry and Config Sync
- CI/GitOps with Helm, GitHub Actions, Google Artifact Registry and Config Sync