Deploy Charmed Airbyte¶
In this step you deploy the Charmed Airbyte application and integrate it with the supporting charms deployed in the previous step.
Deploy Charmed Airbyte¶
Deploy Airbyte using the official charm:
juju deploy airbyte-k8s --channel 2/stable --trust
Verify the deployment:
juju status --watch 2s
Initially, Airbyte is in a blocked state with a message such as:
database relation not ready
You add the relations in the next steps.
Integrate Airbyte with PostgreSQL¶
Airbyte depends on PostgreSQL to store metadata, configuration, and job history. Add the relation between PostgreSQL and Airbyte:
juju integrate postgresql-k8s airbyte-k8s
The database relation is satisfied, and Airbyte moves on to check its connection to Temporal:
temporal relation not ready
Set up Temporal¶
Airbyte offloads workflow execution to Temporal, so Temporal must be operational before Airbyte can run syncs. First, bring Temporal up by relating it to its own backends:
temporal-k8s— the Temporal workflow engine, which stores its default and visibility data in PostgreSQLtemporal-admin-k8s— registers the Temporal namespace and provides admin capabilities
Add the relations:
juju integrate temporal-k8s:db postgresql-k8s:database
juju integrate temporal-k8s:visibility postgresql-k8s:database
juju integrate temporal-k8s:admin temporal-admin-k8s:admin
juju integrate temporal-k8s:temporal-host-info temporal-admin-k8s:temporal-host-info
Once temporal-k8s and temporal-admin-k8s reach active, create the default Temporal namespace that Airbyte uses:
juju run temporal-admin-k8s/0 cli args="operator namespace --namespace default create"
Then relate Airbyte to Temporal, so that it receives the Temporal server address:
juju integrate airbyte-k8s:temporal-host-info temporal-k8s:temporal-host-info
The Temporal relation is satisfied, and Airbyte moves on to check its object storage:
minio relation not ready
Integrate Airbyte with MinIO¶
Airbyte requires object storage for logs, artifacts, and state. Add the relation between MinIO and Airbyte:
juju integrate minio airbyte-k8s
With PostgreSQL, Temporal and MinIO related, the airbyte-bootloader container runs its one-time initialization; you may briefly see waiting for airbyte-auth-secrets while it creates the auth secret, after which airbyte-k8s/0 settles to active.
After all relations are applied, check the status:
juju status --relations
All applications (airbyte-k8s, temporal-k8s, temporal-admin-k8s, postgresql-k8s, minio) should eventually show active status. At this point, Airbyte is fully operational:
Model Controller Cloud/Region Version SLA Timestamp
airbyte-model airbyte-controller microk8s/localhost 3.6.11 unsupported 14:00:29+03:00
App Version Status Scale Charm Channel Rev Address Exposed Message
airbyte-k8s v2.0.0 active 1 airbyte-k8s 2/stable 37 10.x.x.x no
minio res:oci-image@7f2474f active 1 minio ckf-1.10/stable 459 10.x.x.x no
postgresql-k8s 14.15 active 1 postgresql-k8s 14/stable 495 10.x.x.x no
temporal-admin-k8s 1.23.1 active 1 temporal-admin-k8s 1.23/stable 28 10.x.x.x no
temporal-k8s 1.23.1 active 1 temporal-k8s 1.23/stable 68 10.x.x.x no
Unit Workload Agent Address Ports Message
airbyte-k8s/0* active idle 10.x.x.x
minio/0* active idle 10.x.x.x 9000-9001/TCP
postgresql-k8s/0* active idle 10.x.x.x
temporal-admin-k8s/0* active idle 10.x.x.x
temporal-k8s/0* active idle 10.x.x.x
Integration provider Requirer Interface Type Message
airbyte-k8s:airbyte-peer airbyte-k8s:airbyte-peer airbyte peer
minio:object-storage airbyte-k8s:object-storage object-storage regular
postgresql-k8s:database airbyte-k8s:db postgresql_client regular
postgresql-k8s:database temporal-k8s:db postgresql_client regular
postgresql-k8s:database temporal-k8s:visibility postgresql_client regular
temporal-admin-k8s:admin temporal-k8s:admin temporal regular
temporal-admin-k8s:peer temporal-admin-k8s:peer temporal-admin peer
temporal-k8s:peer temporal-k8s:peer temporal peer
temporal-k8s:temporal-host-info airbyte-k8s:temporal-host-info temporal-host-info regular
temporal-k8s:temporal-host-info temporal-admin-k8s:temporal-host-info temporal-host-info regular
Charmed Airbyte is up and running. Continue to run your first sync.