Skip to content
Snippets Groups Projects
Commit 7f724539 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

feat: switch to gitlab-cp impl by default

parent e6ee2b61
No related branches found
No related tags found
No related merge requests found
...@@ -10,7 +10,7 @@ variables: ...@@ -10,7 +10,7 @@ variables:
# variabilized gitlab-cp image # variabilized gitlab-cp image
GITLABCP_IMAGE: registry.gitlab.com/to-be-continuous/tools/gitlab-cp:latest GITLABCP_IMAGE: registry.gitlab.com/to-be-continuous/tools/gitlab-cp:latest
# variabilized sync tool to use (either 'legacy' or 'gitlab-cp') # variabilized sync tool to use (either 'legacy' or 'gitlab-cp')
SYNC_TOOL: legacy SYNC_TOOL: gitlab-cp
stages: stages:
- build - build
......
# GitLab Synchronization Script # to-be-continuous synchronization
This project provides a script able to recursively copy/synchronize a GitLab group from one GitLab server to another. This project allows to recursively copy/synchronize the [to be continuous](https://gitlab.com/to-be-continuous) project from gitlab.com to your self-managed GitLab instance.
It can be run manually (command line) and also as scheduled CI/CD job to regularly synchronize a GitLab group mirror. It can be run manually (manual pipeline) and also as scheduled CI/CD pipeline to regularly retrieve upstream changes.
## Pre-requisites Under the hood, it uses the [GitLab Copy CLI](https://gitlab.com/to-be-continuous/tools/gitlab-cp) tool as a Docker image, [directly pulled from gitlab.com](https://gitlab.com/to-be-continuous/tools/gitlab-cp/container_registry/6261230).
The GitLab Synchronization Script has the following requirements: ## Usage: CLI (first-time copy)
* Bash interpreter <br/>_Trivial on Linux or MacOS, tested with [Git Bash](https://www.atlassian.com/git/tutorials/git-bash) on Windows (available in [Git for Windows](https://gitforwindows.org/))_ The GitLab Copy CLI tool can be used directly from your computer (this is the recommended way to clone _to be continuous_ for the first time).
* [curl tool](https://curl.se/) installed and accessible as `curl` command from the Bash interpreter
* [jq tool](https://stedolan.github.io/jq/download/) installed and accessible as `jq` command from the Bash interpreter
## Usage: script More info is available in [_to be continuous_ documentation](https://to-be-continuous.gitlab.io/doc/self-managed/basic/#copy-tbc-to-your-gitlab).
```bash ## Usage: CI/CD
gitlab-sync.sh \
[--src-api {GitLab source API url}] \ Once copied _to be continuous_ to your GitLab server, you shall then schedule a pipeline in this project (`to-be-continuous/tools/gitlab-sync`) - for instance every night - to keep synchronized with the upstream project.
[--src-token {GitLab source token}] \
[--src-sync-path {GitLab source root group path to synchronize}] \ The job only requires a GitLab token, that shall be configured declaring a `$GITLAB_TOKEN` CI/CD project variable.
--dest-api {GitLab destination API url} \ All the other parameters/variables are automatically determined by the job but might be overwritten with the appropriate project variables if need be:
--dest-token {GitLab destination token} \
[--dest-sync-path {GitLab destination root group path to synchronize}] \
[--max-visibility {max visibility}] \
[--exclude {coma separated list of project/group path(s) to exclude}] \
[--insecure] \
[--update-release] \
[--no-group-description {do not synchronise group description}] \
[--no-project-description {do not synchronise project description}]
```
| CLI option | Env. Variable | Description | Default Value | | CLI option | Env. Variable | Description | Default Value |
|----------------------------|----------------------------------|-----------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------| | -------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------- |
| `--src-api` | `$SRC_GITLAB_API` | GitLab source API url | `https://gitlab.com/api/v4` | | `--src-api` | `$SRC_GITLAB_API` | GitLab source API url | `https://gitlab.com/api/v4` |
| `--src-token` | `$SRC_TOKEN` | GitLab source token (_optional_ if source GitLab group and sub projects have `public` visibility) | _none_ |
| `--src-sync-path` | `$SRC_SYNC_PATH` | GitLab source root group path to synchronize | `to-be-continuous` | | `--src-sync-path` | `$SRC_SYNC_PATH` | GitLab source root group path to synchronize | `to-be-continuous` |
| `--dest-api` | `$DEST_GITLAB_API` | GitLab destination API url (**mandatory**) | `$CI_API_V4_URL` (defined when running in GitLab CI) | | `--dest-api` | `$DEST_GITLAB_API` | GitLab destination API url | `$CI_API_V4_URL` |
| `--dest-token` | `$DEST_TOKEN` or `$GITLAB_TOKEN` | GitLab destination token with at least scopes `api,read_repository,write_repository` and `Owner` role (**mandatory**) | _none_ | | `--dest-token` | `$DEST_TOKEN` or `$GITLAB_TOKEN` | GitLab destination token with at least scopes `api,read_repository,write_repository` and `Owner` role | _none_ (**mandatory**) |
| `--dest-sync-path` | `$DEST_SYNC_PATH` | GitLab destination root group path to synchronize | guessed from GitLab CI env or `to-be-continuous` | | `--dest-sync-path` | `$DEST_SYNC_PATH` | GitLab destination root group path to synchronize (defaults to `--src-sync-path`) | determined from `$CI_PROJECT_NAMESPACE` |
| `--max-visibility` | `$MAX_VISIBILITY` | maximum visibility of projects in destination group | `$CI_PROJECT_VISIBILITY` (when running in GitLab CI) or `public` | | `--max-visibility` | `$MAX_VISIBILITY` | maximum visibility of projects in destination group (defaults to `public`) | `$CI_PROJECT_VISIBILITY` |
| `--exclude` | `$EXCLUDE` | coma separated list of project/group path(s) to exclude | _none_ | | `--exclude` | `$EXCLUDE` | project/group path(s) to exclude (multiple CLI option; env. variable is a coma separated list) | `samples,custom` |
| `--insecure` | `$INSECURE` | set to skip TLS check on curl requests | `false` | | `--insecure` | `$INSECURE` | skip SSL verification | `false` |
| `--update-release` | `$UPDATE_RELEASE` | set to update the releases even if they exists | `false` | | `--update-release` | `$UPDATE_RELEASE` | set to force the update of the latest release (in order to trigger GitLab CI/CD catalog publication) | `false` |
| `--no-group-description` | `$GROUP_DESCRIPTION_DISABLED` | do not synchronise group description | _none_ | | `--update-avatar` | `$UPDATE_AVATAR` | force update the avatar images even when they exist and look the same | `false` |
| `--no-project-description` | `$PROJECT_DESCRIPTION_DISABLED` | do not synchronise project description | _none_ | | `--no-group-description` | `$GROUP_DESCRIPTION_DISABLED` | don't synchronize group description | `false` |
| `--no-project-description` | `$PROJECT_DESCRIPTION_DISABLED` | don't synchronize project description | `false` |
You shall use this script to copy the _to be continuous_ project to your own GitLab server for the first time with the following command: | `--dry-run` | _none_ | dry run (don't execute any write action) | `false` |
| `--halt-on-error` | _none_ | halt synchronizing when an error occurs | `false` |
```bash | `--cache-dir` | `$CACHE_DIR` | cache directory (used to download resources such as images and Git repositories) (defaults to `.work`) | `.work` |
curl -s https://gitlab.com/to-be-continuous/tools/gitlab-sync/-/raw/master/gitlab-sync.sh | bash /dev/stdin --dest-api {your GitLab server API url} --dest-token {your GitLab token} --exclude samples,custom
```
:warning: Each CLI option may alternately be specified with an environment variable (see in the table above). This might be useful to configure the CI/CD job. ## CI/CD Catalog
## Usage: CI/CD
Once copied _to be continuous_ to your GitLab server, you shall then schedule a pipeline in this project (`to-be-continuous/tools/gitlab-sync`) - for instance every night - to keep synchronized with source project. To use _to be continuous_ templates as CI/CD Catalog resources in your GitLab server, you'll have to manually activate the _CI/CD Catalog resource_ option (`Settings > General > Visibility > Project Features > Permissions`) in each template project.
Then you'll have to re-publish a release to make the template appear in the CI/CD Catalog.
The script will only require a GitLab token, that shall be configured declaring a `$GITLAB_TOKEN` CI/CD project variable. (`--dest-api` will be implicitly retrieved using predefined `$CI_API_V4_URL`). :information_source: You may re-run the GitLab Copy CLI tool with the `--update-release` (or `$UPDATE_RELEASE` variable) set to force re-publishig the latest release of each template.
You can define a custom alpine image to use by setting `$GITLAB_SYNC_IMAGE` (default to `registry.hub.docker.com/alpine:latest`) ## Legacy `gitlab-sync.sh` script
## CI/CD Catalog This project was previously using the `gitlab-sync.sh` script to perform the recursive copy/sync.
It has now been replaced by the [GitLab Copy CLI](https://gitlab.com/to-be-continuous/tools/gitlab-cp) tool that has numerous advantages over it.
To use _to be continuous_ templates as CI/CD Catalog resources on a your GitLab Server, you have to manually activate the _CI/CD Catalog resource_ option in `Settings`/`General`/`Visibility, project features, permissions`. Anyhow, the legacy script can still be reactivated by setting the `SYNC_TOOL` CI/CD project variable to `legacy`.
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment