Upgrade Podman-based installations (Podman Quadlets)

Includes updates for CM 8.13.0-1.

This article describes how to upgrade an existing Podman Quadlets Installation of Smart ID Certificate Manager (CM).

Sequential upgrade process

You must upgrade the database schema and configuration files for each intermediary version. All the necessary database schema and configuration file changes for the intermediary versions are supplied in the distribution zip of the target version, so there is no need to download the distribution zips for the intermediary versions separately.

Example:

Starting upgrade from 8.10.0, you must upgrade the database schema and configuration files to 8.11.0 and thereafter to 8.12.0, using the supplied files in the distribution zip for 8.12.0.

Prerequisites

Quadlet installation of Certificate Manager (CM) installed with version CM 8.10.x.
Podman version 4.9.4 or later.
Administrator's Workbench, Registration Authority, and Certificate Controller clients from CM distributable package.
Access to the new version's distributable package including updated container images ,database migration scripts and configuration files.

Step-by-step instruction

1. Pre-upgrade preparations

Ensure database accessibility
Verify that the database is accessible and functioning properly before proceeding:
BASH
```
podman exec -ti <cf-server-container-name> bash
# Use JDBC or other database tools to test the connection
```
Stop all running containers
Use the following commands to stop the running containers:
BASH
```
systemctl --user stop cf-server
systemctl --user stop pgw
```
Backup the database.
As part of the backup, make sure to backup the cmdb database, please refer to the relevant database documentation for this step.
Verify that the container is stopped
Check that all containers are stopped:
BASH
```
podman ps --all
```
Ensure no containers from the previous deployment are running.

Backup Volumes (optional but highly recommended)
If something goes wrong, the backup can be restored, and the system can return to its previous state.
Volumes that should be backed up are certs, config, bin and any relevant volume to your installation.

Locate the volumes using:

BASH

podman volume list

Example (name can change but should contain (cf-server):

BASH

> podman volume list
local       systemd-cf-server-certs
local       systemd-cf-server-config
local       systemd-cf-server-bin
local       systemd-pgw-config-tomcat
local       systemd-pgw-config-gw

Locate the Mountpoint on host:

CODE

podman volume inspect <volume-name>

Example:

BASH

> podman volume inspect systemd-cf-server-certs
[
     {
          "Name": "systemd-cf-server-certs",
          "Driver": "local",
          "Mountpoint": "/home/azureuser/.local/share/containers/storage/volumes/systemd-cf-server-certs/_data",
          "CreatedAt": "2024-06-20T07:25:01.386573664Z",
          "Labels": {},
          "Scope": "local",
          "Options": {},
          "MountCount": 0,
          "LockNumber": 4
     }
]

Back up the Mountpoint.

Unzip the distribution.

2. Load new container images

Import new images
Load the new container images using:

BASH

podman image load --input certificate-manager-<version>/images/cf-server-image-<version>.tar
podman image load --input certificate-manager-<version>/images/pgw-image-<version>.tar

Verify the images
Ensure the new images are loaded correctly:
BASH
```
podman images
```
Confirm the tags and versions of the new image.

3. Update Quadlet configuration

Locate the cf-server.container and pgw.container used for the previous installation.
The following command displays the path of the files .container under the Loaded attribute.

CODE

 systemctl --user status cf-server
 systemctl --user status pgw

Example

CODE

> systemctl --user status cf-server
× cf-server.service
     Loaded: loaded (/home/azureuser/.config/containers/systemd/cf-server.container; generated)
...

Modify configuration
Modify the Image Name and tag to the new Image in the cf-server.container and pgw.container files.
Reload systemd daemon
Reload the systemd daemon to apply the changes:
BASH
```
systemctl --user daemon-reload
```

4. Upgrade database and configuration files

Example

To upgrade from 8.10.0 to 8.13.0, use the files in the following folders in sequence:

Upgrade from CM 8.10.x to 8.11.0
Upgrade from CM 8.11.x to 8.12.0
Upgrade from CM 8.12.x to 8.13.0

Run database scripts (only necessary for cf-server)
The upgrade scripts are stored in the upgrade directory under the distributable.
Execute the database upgrade script towards the database server.
Ensure the script ran without errors.
Example for postgresql:
CODE
```
psql -U lcmreq -d cmdb -f "CMDBUpgrade_PostgreSQL_8_11_0.sql"
```
Update the configuration files
Locate the .conf files under the upgrade folder inside the release distribution.
Open the .conf files and update the configuration files according to the instructions specified by the .conf file. Upgrade should be done towards the volume Mountpoint.
If the configuration path specified in the .conf file do not map to a volume, changes will automatically be applied hence these files can be ignored.

5. Upgrade Protocol Gateway

Version specific actions for upgrading Protocol Gateway can be found in step 3 of the Upgrade Protocol Gateway page.
Only update configuration files.

6. Restart the containers

Start the updated containers
Start the new containers using:
BASH
```
systemctl --user start cf-server
systemctl --user start pgw
```
Following the logs from startup can be done as follows:
CODE
```
systemctl --user start cf-server;podman logs -f cf-server
```
Monitor logs
Check the logs to ensure the containers started successfully:
BASH
```
podman logs -f cf-server
podman logs -f pgw
```

7. Post-upgrade validation

Verify connectivity
Test connectivity using Administrator's Workbench and/or Registration Authority clients.
Verify configuration
Access the container volumes to confirm that the configuration files are correctly applied:
BASH
```
podman exec -ti cf-server bash
ls /opt/cm/server/
```
Check database connection
Confirm that the container is successfully connected to the upgraded database.
Health checks
Enable and test the health check for the Protocol Gateway (pgw):
BASH
```
systemctl --user start pgw
podman logs -f pgw
```

8. Rollback procedure (if needed)

If any issue occurs during the upgrade, you can roll back using the following steps:

Stop the new containers

BASH

systemctl --user stop cf-server
systemctl --user stop pgw

Restore volumes
Restore the backed-up volumes.
Point to the previous images
Point to the old images in cf-server.container and pgw.container.
Restore database

Start the old containers
Restart the containers using the old quadlets:

BASH

systemctl --user daemon-reload
systemctl --user start cf-server
systemctl --user start pgw

9. Troubleshooting

View logs

Use the following command to view container logs for debugging:

BASH

podman logs <container-name>

Inspect container configuration

Inspect the container configuration for issues:

BASH

podman inspect <container-name>

Access shell in container

Start a shell inside the container for manual troubleshooting:

BASH

podman exec -ti cf-server bash

Check systemd status

Check the status of the systemd service:

BASH

systemctl --user status cf-server