Skip to content
Snippets Groups Projects
Commit 681d465e authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'doc-focus-on-variables' into 'master' 

reword to focus on variables used to configure the sync job

See merge request to-be-continuous/tools/gitlab-sync!38
parents 9eb26ef8 386714bd
No related branches found
No related tags found
No related merge requests found
Expand all Show whitespace changes
Inline Side-by-side
.gitlab-ci.yml
+ 0
− 96
View file @ 681d465e
include:
# $TBC_NAMESPACE is a group variable; can be globally overridden
- project: '$TBC_NAMESPACE/bash'
ref: '3.3.0'
file: '/templates/gitlab-ci-bash.yml'
variables:
BASH_SHELLCHECK_FILES: "*.sh"
GITLAB_SYNC_IMAGE: "registry.hub.docker.com/library/alpine:latest"
# variabilized gitlab-cp image
GITLABCP_IMAGE: registry.gitlab.com/to-be-continuous/tools/gitlab-cp:latest
# variabilized sync tool to use (either 'legacy' or 'gitlab-cp')
SYNC_TOOL: gitlab-cp
stages:
- build
- test
sync-tbc-legacy:
image:
name: "${GITLAB_SYNC_IMAGE}"
entrypoint: [""]
stage: build
variables:
# exclude samples group from sync
EXCLUDE: samples,custom
before_script:
- |
function install_ca_certs() {
certs=$1
if [[ -z "$certs" ]]
then
return
fi
if echo "$certs" >> /etc/ssl/certs/ca-certificates.crt
then
echo -e "CA certificates imported in \\e[33;1m/etc/ssl/certs/ca-certificates.crt\\e[0m"
fi
if echo "$certs" >> /etc/ssl/cert.pem
then
echo -e "CA certificates imported in \\e[33;1m/etc/ssl/cert.pem\\e[0m"
fi
}
- install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
- |
function maybe_install_pkg() {
pkg_name="$1"
cmd_name="${2:-$pkg_name}"
if ! command -v "$cmd_name" > /dev/null
then
echo "[INFO] $pkg_name not found: install..."
if command -v apt-get > /dev/null
then
apt-get install -yq "$pkg_name"
elif command -v apk > /dev/null
then
apk add --no-cache "$pkg_name"
elif command -v yum > /dev/null
then
yum install -yq "$pkg_name"
elif command -v dnf > /dev/null
then
dnf install -yq "$pkg_name"
elif command -v zypper > /dev/null
then
zypper -nq install "$pkg_name"
else
echo "[ERROR] ... didn't find any known package manager to install $pkg_name!"
exit 1
fi
fi
}
- maybe_install_pkg git
- maybe_install_pkg curl
- maybe_install_pkg jq
- maybe_install_pkg bash
# done twice because /etc/ssl/certs/ca-certificates.crt is altered during apk add
# we can't use update-ca-certificates --fresh because $CUSTOM_CA_CERTS can contains several certificates
- install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
script:
- bash ${CI_DEBUG_TRACE:+-x} ./gitlab-sync.sh
rules:
# disable on gitlab.com
- if: '$CI_SERVER_HOST == "gitlab.com"'
when: never
# disable if $SYNC_TOOL != 'legacy'
- if: '$SYNC_TOOL != "legacy"'
when: never
# allow manual source
- if: '$CI_PIPELINE_SOURCE == "web"'
# allow scheduled source
- if: '$CI_PIPELINE_SOURCE == "schedule"'
sync-tbc-legacy-dryrun:
extends: sync-tbc-legacy
script:
# unset dest GitLab (dryrun)
- bash ${CI_DEBUG_TRACE:+-x} ./gitlab-sync.sh --dest-api ""
rules:
# run only on gitlab.com (to test at least the script correctly crawls tbc projects)
- if: '$CI_SERVER_HOST == "gitlab.com"'
sync-tbc:
image:
name: "$GITLABCP_IMAGE"
......@@ -147,9 +54,6 @@ sync-tbc:
# disable on gitlab.com
- if: '$CI_SERVER_HOST == "gitlab.com"'
when: never
# disable if $SYNC_TOOL != 'gitlab-cp'
- if: '$SYNC_TOOL != "gitlab-cp"'
when: never
# allow manual source
- if: '$CI_PIPELINE_SOURCE == "web"'
# allow scheduled source
......
README.md
+ 41
− 24
View file @ 681d465e
......@@ -19,23 +19,47 @@ Once copied _to be continuous_ to your GitLab server, you shall then schedule a
The job only requires a GitLab token, that shall be configured declaring a `$GITLAB_TOKEN` CI/CD project variable.
All the other parameters/variables are automatically determined by the job but might be overwritten with the appropriate project variables if need be:
| CLI option | Env. Variable | Description | Default Value |
| -------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------- |
| `--src-api` | `$SRC_GITLAB_API` | GitLab source API url | `https://gitlab.com/api/v4` |
| `--src-sync-path` | `$SRC_SYNC_PATH` | GitLab source root group path to synchronize | `to-be-continuous` |
| `--dest-api` | `$DEST_GITLAB_API` | GitLab destination API url | `$CI_API_V4_URL` |
| `--dest-token` | `$DEST_TOKEN` or `$GITLAB_TOKEN` | GitLab destination token with at least scopes `api,read_repository,write_repository` and `Owner` role | _none_ (**mandatory**) |
| `--dest-sync-path` | `$DEST_SYNC_PATH` | GitLab destination root group path to synchronize (defaults to `--src-sync-path`) | determined from `$CI_PROJECT_NAMESPACE` |
| `--max-visibility` | `$MAX_VISIBILITY` | maximum visibility of projects in destination group (defaults to `public`) | `$CI_PROJECT_VISIBILITY` |
| `--exclude` | `$EXCLUDE` | project/group path(s) to exclude (multiple CLI option; env. variable is a coma separated list) | `samples,custom` |
| `--insecure` | `$INSECURE` | skip SSL verification | `false` |
| `--update-release` | `$UPDATE_RELEASE` | set to force the update of the latest release (in order to trigger GitLab CI/CD catalog publication) | `false` |
| `--update-avatar` | `$UPDATE_AVATAR` | force update the avatar images even when they exist and look the same | `false` |
| `--no-group-description` | `$GROUP_DESCRIPTION_DISABLED` | don't synchronize group description | `false` |
| `--no-project-description` | `$PROJECT_DESCRIPTION_DISABLED` | don't synchronize project description | `false` |
| `--dry-run` | _none_ | dry run (don't execute any write action) | `false` |
| `--halt-on-error` | _none_ | halt synchronizing when an error occurs | `false` |
| `--cache-dir` | `$CACHE_DIR` | cache directory (used to download resources such as images and Git repositories) (defaults to `.work`) | `.work` |
| Env. Variable | Description | Default Value |
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------- | --------------------------------------- |
| `$SRC_GITLAB_API` | GitLab source API url | `https://gitlab.com/api/v4` |
| `$SRC_SYNC_PATH` | GitLab source root group path to synchronize | `to-be-continuous` |
| `$DEST_GITLAB_API` | GitLab destination API url | `$CI_API_V4_URL` |
| `$DEST_TOKEN` or `$GITLAB_TOKEN` | GitLab destination token with at least scopes `api,read_repository,write_repository` and `Owner` role | _none_ (**mandatory**) |
| `$DEST_SYNC_PATH` | GitLab destination root group path to synchronize (defaults to `--src-sync-path`) | determined from `$CI_PROJECT_NAMESPACE` |
| `$MAX_VISIBILITY` | maximum visibility of projects in destination group (defaults to `public`) | `$CI_PROJECT_VISIBILITY` |
| `$SKIP_VISIBILITY` | skip updating the destination group or project visibility (when it exists already) | `false` |
| `$EXCLUDE` | project/group path(s) to exclude (multiple CLI option; env. variable is a coma separated list) | `samples,custom` |
| `$INCLUDE` | project/group path(s) to include (multiple CLI option; env. variable is a coma separated list) | `tools,docker,ansible` |
| `$INSECURE` | skip SSL verification | `false` |
| `$UPDATE_RELEASE` | set to force the update of the latest release (in order to trigger GitLab CI/CD catalog publication) | `false` |
| `$UPDATE_AVATAR` | force update the avatar images even when they exist and look the same | `false` |
| `$GROUP_DESCRIPTION_DISABLED` | don't synchronize group description | `false` |
| `$PROJECT_DESCRIPTION_DISABLED` | don't synchronize project description | `false` |
| `$NEW_GROUP_OPTIONS` | a JSON string with [extra options for groups creation](https://docs.gitlab.com/api/groups/#create-a-group) | _none_ |
| `$NEW_PROJECT_OPTIONS` | a JSON string with [extra options for projects creation](https://docs.gitlab.com/api/projects/#create-a-project) | _none_ |
| `$CACHE_DIR` | cache directory (used to download resources such as images and Git repositories) (defaults to `.work`) | `.work` |
### Customizing copied project options
Projects are copied with the following default options:
```json
{
"issues_access_level": "disabled",
"merge_requests_access_level": "disabled"
}
```
You can customize those default options with your own ones with the `$NEW_PROJECT_OPTIONS` variable.
For instance if you want to globally disable CI/CD pipelines in copied projects:
```json
{
"builds_access_level": "disabled"
}
```
Similarly, you can customize default group options with the `$NEW_GROUP_OPTIONS` variable.
## CI/CD Catalog
......@@ -43,10 +67,3 @@ To use _to be continuous_ templates as CI/CD Catalog resources in your GitLab se
Then you'll have to re-publish a release to make the template appear in the CI/CD Catalog.
:information_source: You may re-run the GitLab Copy CLI tool with the `--update-release` (or `$UPDATE_RELEASE` variable) set to force re-publishig the latest release of each template.
## Legacy `gitlab-sync.sh` script
This project was previously using the `gitlab-sync.sh` script to perform the recursive copy/sync.
It has now been replaced by the [GitLab Copy CLI](https://gitlab.com/to-be-continuous/tools/gitlab-cp) tool that has numerous advantages over it.
Anyhow, the legacy script can still be reactivated by setting the `SYNC_TOOL` CI/CD project variable to `legacy`.
gitlab-sync.sh deleted 100755 → 0
+ 0
− 535
View file @ 9eb26ef8
This diff is collapsed.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment