Introduction

MPAS (Multi Product Pipeline Automation System) enables the release of complex software systems in a fully automated way. It is based on the Open component model and uses Kubernetes as a runtime environment.

MPAS provides a cloud-native operational model for packaging and running applications. It takes an opinionated approach that considers the packaging unit to be an Open Component Model (OCM) component, Git to be the configuration interface and GitOps to be the deployment methodology.

Therefore an environment in which MPAS can be bootstrapped requires only the following cloud-native primitives:

  • Kubernetes Cluster
  • OCI Registry
  • Git Provider (currently we support Gitea and GitHub)

MPAS has a strong focus on tooling and our CLI helps with automating all of the hard stuff.

Take a look at the getting started guide to get going straight away or peruse the architecture documentation to get a technical overview of how MPAS functions.

Setup

We provide a number of options for installing MPAS.

Note: If you are managing an MPAS environment then you'll also need to take a look at our bootstrap section which guides you through the process of deploying MPAS.

Installation

Install using Binaries

To install the MPAS CLI, you can download the binaries for major platforms from the GitHub releases page.

Install using Homebrew

You can also install via homebrew for macOS and Linux:

brew install open-component-model/tap/mpas

Install using curl-bash

To install with bash for macOS or Linux execute the following command:

curl -sfL https://raw.githubusercontent.com/open-component-model/mpas/main/install.sh | sh -

Bootstrap

The mpas bootstrap command deploys the following components to your cluster:

  • Flux: A Kubernetes operator that will install and manage the other components.
  • ocm-controller: A Kubernetes controller that enables the automated deployment of software using the Open Component Model and Flux.
  • git-controller: A Kubernetes controller that will create pull requests in the target Github repository when changes are made to the cluster.
  • replication-controller: A Kubernetes controller that replicates everything defined and bundled in an OCM component version (and that the consumer subscribed to) into the local OCI registry of the cluster.
  • mpas-product-controller: A Kubernetes controller responsible for creating the custom resource Product.
  • mpas-project-controller: A Kubernetes controller responsible for bootstrapping a whole project and creating relevant access credentials, service accounts, roles and the main repository. It reconciles the Project resource.

Besides the above components, the mpas bootstrap command will also push the corresponding component manifests to the target Git repository and configure Flux to continuously update the installed components from the target Git repository.

After the mpas bootstrap command is executed, the cluster is ready to deploy software in a GitOps fashion using the Open Component Model and MPAS.

{{% alert color="danger" title="Required permissions" %}} To bootstrap MPAS, the person running the command must have cluster admin rights for the target Kubernetes cluster. It is also required that the person running the command to be the owner of the GitHub repository, or to have admin rights of a GitHub organization. {{% /alert %}}

Bootstrap for GitHub

GitHub Personal Access Token (PAT)

For accessing the GitHub API, the boostrap command requires a GitHub personal access token (PAT) with administration permissions.

The GitHub PAT can be exported as environment variable:

export GITHUB_TOKEN=<your-github-pat>

If the GITHUB_TOKEN environment variable is not set, the mpas bootstrap command will prompt for the GitHub PAT.

{{% alert color="danger" title="PAT secret" %}} Note that the GitHub PAT is stored in the cluster as a Kubernetes Secret named flux-system inside the flux-system namespace. {{% /alert %}}

Personal account

Run the bootstrap for a repository on your personal GitHub account:

mpas bootstrap github \
  --owner=<your-github-username> \
  --repository=<your-github-repository> \
  --path=clusters/my-cluster \
  --personal

If the specified repository does not exist, the mpas bootstrap command will create it as a private repository. If you wish to create a public repository, you can use the --private=false flag.

Organization

If you want to bootstrap MPAS for a repository owned by an GitHub organization, it is recommended to create a dedicated GitHub user for MPAS and use that user to bootstrap the repository.

Run the bootstrap for a repository owned by a GitHub organization:

mpas bootstrap github \
  --owner=<your-github-organization> \
  --repository=<your-github-repository> \
  --path=clusters/my-cluster \

Bootstrap for Gitea

Gitea API token

For accessing the Gitea API, the boostrap command requires a Gitea API token with administration permissions.

The Gitea API Token can be exported as an environment variable:

export GITEA_TOKEN=<your-gitea-api-token>

If the GITEA_TOKEN environment variable is not set, the mpas bootstrap command will prompt for the Gitea API token.

{{% alert color="danger" title="API Token secret" %}} Note that the Gitea API Token is stored in the cluster as a Kubernetes Secret named flux-system inside the flux-system namespace. {{% /alert %}}

Personal account

Run bootstrap for a repository on your personal Gitea account:

mpas bootstrap gitea \
  --owner=<your-gite  -username> \
  --repository=<your-gitea-repository> \
  --path=clusters/my-cluster \
  --personal

If the specified repository does not exist, the mpas bootstrap command will create it as a private repository. If you wish to create a public repository, you can use the --private=false flag.

Organization

If you want to bootstrap MPAS for a repository owned by an Gitea organization, it is recommended to create a dedicated Gitea user for MPAS and use that user to bootstrap the repository.

Run the bootstrap for a repository owned by a Gitea organization:

mpas bootstrap gitea \
  --owner=<your-gitea-organization> \
  --repository=<your-gitea-repository> \
  --path=clusters/my-cluster

Bootstrap for an air-gapped environment

If you want to bootstrap MPAS for a repository in an air-gapped environment, only Gitea is supported at the moment.

Export the bootstrap components bundle

To bootstrap MPAS in an air-gapped environment, you need to export the bootstrap components bundle from the MPAS default registry.

mpas bootstrap \
  --export \
  --export-path=/tmp

The above command will export the bootstrap components archive to /tmp/mpas-bundle.tar.gz.

It is then possible to import the bootstrap components bundle into an air-gapped environment registry and use it to bootstrap MPAS for a repository in that environment.

mpas bootstrap gitea \
  --owner=<your-gitea-organization> \
  --repository=<your-gitea-repository> \
  --from-file=/tmp/mpas-bundle.tar.gz \
  --registry=<your-air-gapped-registry> \
  --path=clusters/my-cluster

The above command will copy the bootstrap components from the bundle archive to the specified air-gapped registry and bootstrap MPAS for the specified repository.

Getting Started

This tutorial shows you how to bootstrap MPAS to a Kubernetes cluster and deploy a simple application.

Prerequisites

  • A Kubernetes cluster
  • A GitHub access token with repo scope
  • kubectl

Objectives

  • Bootstrap MPAS to a Kubernetes cluster
  • Deploy a simple application

Install the MPAS CLI

The MPAS CLI is the primary tool for interacting with MPAS. It can be used to bootstrap MPAS to a Kubernetes cluster.

To install the MPAS CLI using brew:

brew install open-component-model/tap/mpas

For other installation methods, see the installation guide.

Bootstrap MPAS

Export your GitHub access token

The MPAS CLI uses your GitHub access token to authenticate with GitHub. To create a GitHub access token, see the GitHub documentation.

export GITHUB_TOKEN=<your-github-access-token>
export GITHUB_USER=<your-username>

Bootstrap MPAS

To bootstrap MPAS to your Kubernetes cluster, run the following command. If nothing is specified it will use the KUBECONFIG specified in the user's environment. It is also possible to specify a dedicated config using the --kubeconfig option. 

mpas bootstrap github \
  --owner=$GITHUB_USER \
  --repository=mpas-bootstrap \
  --path=./clusters/my-cluster \
  --personal

This command will create a new Github repository called mpas-bootstrap and bootstrap MPAS to your Kubernetes cluster. The following components will be installed:

  • Flux: A Kubernetes operator that will install and manage the other components.
  • ocm-controller: A Kubernetes controller that enables the automated deployment of software components using the Open Component Model and Flux.
  • git-controller: A Kubernetes controller that will create pull requests in the target Github repository when changes are made to the cluster.
  • replication-controller: A Kubernetes controller that keeps keep component versions in the cluster up-to-date with a version defined by the consumer in the ComponentSubscription resource.
  • mpas-product-controller: A Kubernetes controller, responsible for creating a product. Reconciles the Product resource.
  • mpas-project-controller: A Kubernetes controller responsible for bootstrapping a whole project. Creates relevant access credentials, service accounts, roles and the main GitOps repository and reconciles the Project resource.

The output of the bootstrap is similar to the following:

Running mpas bootstrap ...
 ✓   Preparing Management repository mpas-test
 ✓   Fetching bootstrap component from ghcr.io/open-component-model/mpas-bootstrap-component
 ✓   Installing flux with version v2.1.0
 ✓   Installing cert-manager with version v1.13.1
 ✓   Reconciling infrastructure components
 ✓   Waiting for cert-manager to be available
 ✓   Generating external-secrets-operator manifest with version v0.9.6
 ✓   Generating git-controller manifest with version v0.9.0
 ✓   Generating mpas-product-controller manifest with version v0.6.0
 ✓   Generating mpas-project-controller manifest with version v0.5.0
 ✓   Generating ocm-controller manifest with version v0.14.1
 ✓   Generating replication-controller manifest with version v0.8.0
 ✓   Generate certificate manifests
 ✓   Reconciling infrastructure components
 ✓   Waiting for components to be ready

Bootstrap completed successfully!

After completing the bootstrap process, the target github repository will contain yaml manifests for the components to be installed on the cluster and Flux will apply all of them to get the components installed. Furthermore the installed Flux components will be configured to watch the target github repository for changes in the path ./clusters/my-cluster.

Clone the git repository

Clone the mpas-bootstrap repository to your local machine:

git clone https://github.com/$GITHUB_USER/mpas-bootstrap
cd mpas-bootstrap

Deploy podinfo application

The podinfo application has been packaged as an OCM component and can be retrieved from Github.

  1. Create a secret containing your GitHub credentials that will be used by MPAS to create your project repository.
kubectl create secret generic \
  github-access \
  --from-literal=username=$GITHUB_USER \
  --from-literal=password=$GITHUB_TOKEN \
  -n mpas-system
  1. Create a project that will contain the podinfo application.

Let's create a directory for the project:

mkdir -p ./clusters/my-cluster/podinfo

Then, create a project.yaml file in the ./clusters/my-cluster/podinfo directory:

mpas create project podinfo-application \
  --owner=$GITHUB_USER \
  --provider=github \
  --visibility=public \
  --already-exists-policy=fail \
  --branch=main \
  --secret-ref=github-access \
  --email=$MY_EMAIL \
  --message=xxx \
  --author=mpas-admin \
  --maintainers=$GITHUB_USER \
  --prune \
  --personal \
  --export  >> ./clusters/my-cluster/podinfo/project.yaml

Then, apply the project to the cluster in a gitOps fashion:

git add --all && git commit -m "Add podinfo project" && git push

Flux will detect the changes and apply the project to the cluster.

This will create in the cluster a namespace for the project, a serviceaccount, and RBAC. It will also create a GitHub repository for the project, and configure Flux to manage the project's resources.

  1. Add the needed secrets to the namespace

Flux is used to deploy all workloads in a gitOps way. Flux needs a secret in the project namespace that will be used to communicate with github:

kubectl create secret generic \
  github-access \
  --from-literal=username=$GITHUB_USER \
  --from-literal=password=$GITHUB_TOKEN \
  -n mpas-podinfo-application

Note The credentials should have access to GitHub packages.

As part of step 2, a serviceaccount was created for the project. We will use this service account to provide the necessary permissions to pull from the ghcr registry.

First, create a secret containing the credentials for the service account:

kubectl create secret docker-registry github-registry-key --docker-server=ghcr.io \
  --docker-username=$GITHUB_USER --docker-password=$GITHUB_TOKEN \
  --docker-email=<MY_EMAIL> -n mpas-podinfo-application

Then, patch the service account to use the secret:

kubectl patch serviceaccount mpas-podinfo-application -p '{"imagePullSecrets": [{"name": "github-registry-key"}]}' \
  -n mpas-podinfo-application
  1. Clone the project repository
git clone https://github.com/$GITHUB_USER/mpas-podinfo-application
cd mpas-podinfo-application
  1. Add the podinfo component subscription

Create a file under ./subscriptions/ that will contains the subscription declaration.

mpas create cs podinfo-subscription \
  --component=ocm.software/mpas/podinfo \
  --semver=">=v1.0.0" \
  --source-url=ghcr.io/open-component-model/mpas \
  --source-secret-ref=github-access \
  --target-url=ghcr.io/$GITHUB_USER \
  --target-secret-ref=github-access \
  --namespace=mpas-podinfo-application  \
  --export >> ./subscriptions/podinfo.yaml

Then, apply the ComponentSubscription to the project in a gitOps fashion:

git add --all && git commit -m "Add podinfo subscription" && git push

Flux will detect the changes and apply the subscription to the cluster.

This will replicate the product referenced by the ComponentSubscription spec.component field from defined registry in the spec.source.url to the spec.destination.url registry.

  1. Add a target for the podinfo application

The target will define where the application will be installed

cat <<EOF >> ./targets/podinfo.yaml
apiVersion: mpas.ocm.software/v1alpha1
kind: Target
metadata:
  name: podinfo-kubernetes-target
  namespace: mpas-podinfo-application
  labels:
    target.mpas.ocm.software/ingress-enabled: "true" # This label is defined by the component that will use it to select an appropriate target to deploy to.
spec:
  type: kubernetes
  access:
    targetNamespace: podinfo
  serviceAccountName: podinfo-sa
  selector:
    matchLabels:
      mpas.ocm.software/target-selector: podinfo-kubernetes-target
  interval: 5m0s
EOF

Then, apply the Target to the project in a gitOps fashion:

git add --all && git commit -m "Add a target for podinfo" && git push

Flux will detect the changes and apply the target to the cluster.

In order for the Target to reach a Ready state, the needed secrets should be created in the podinfo namespace.

First, create a secret containing the credentials for the service account:

kubectl create secret docker-registry github-registry-key --docker-server=ghcr.io \
  --docker-username=$GITHUB_USER --docker-password=$GITHUB_TOKEN \
  --docker-email=<MY_EMAIL> -n podinfo

Then, add a label to allow the target to select it using the label selector:

kubectl label secret github-registry-key mpas.ocm.software/target-selector=podinfo-kubernetes-target -n podinfo
  1. Deploy the podinfo application

In order to deploy the podinfo application, we need to create a ProductDeploymentGenerator resource:

mpas create pdg podinfo \
  --service-account=mpas-podinfo-application \
  --subscription-name=podinfo-subscription \
  --subscription-namespace=mpas-podinfo-application  \
  --namespace=mpas-podinfo-application \
  --export >> ./generators/podinfo.yaml

Then, apply the ProductDeploymentGenerator to the project in a gitOps fashion:

git add --all && git commit -m "Add podinfo deployment generator" && git push

Flux will detect the changes and apply the resource to the cluster.

This will create a pull request in the project repository with the ProductDeployment resource that will deploy the podinfo application.

Go to the project repository and retrieve the pull request. It should contain a ProductDeployment declaration that provides the configuration and all steps needed to deploy the product, as well as a values.yaml file. The values file contains values that should be used to configure the different resources that are part of the product to be deployed. There is a check that should pass before merging the pull request.

Once the pull request is merged, Flux will detect the changes and deploy the application to the cluster.

After a moment the ProductDeployment should be deployed successfully. It is possible to verify this with the command:

kubectl describe productdeployment -n mpas-podinfo-application

***TODO:

The deployed helmRepository does not have the secret set that would permit pulling the helmcharts. So we need to add a step for the user to make the chart public until we address this shortcoming. The deployed podinfo deployment doesn't have the pullImageSecret. In order to solve this, the user should update the config.cue to add a serviceAccountName. The name should be the one generated by target.

The result should look something like:

Name:         podinfo
Namespace:    mpas-podinfo-application
Labels:       kustomize.toolkit.fluxcd.io/name=mpas-podinfo-application-products
              kustomize.toolkit.fluxcd.io/namespace=mpas-system
API Version:  mpas.ocm.software/v1alpha1
Kind:         ProductDeployment
Metadata:
...
Status:
  Conditions:
    Last Transition Time:  2023-09-14T10:14:41Z
    Message:               Reconciliation success
    Observed Generation:   1
    Reason:                Succeeded
    Status:                True
    Type:                  Ready
  Observed Generation:     1

The application is deployed in the mpas-podinfo-application namespace.

Administration

Project Setup

Credentials

Product Management

Subscriptions

Deployment

Configuration

Product Authoring

Product Description

Architecture

Troubleshooting

Frequently Asked Questions