VCS Providers

What are VCS providers?

Version Control System (VCS) providers are a feature in Phobos that provide pipeline access to files in providers such as GitLab and GitHub. When a pipeline task is executed, it can mount directories from a VCS provider. This enables pipeline plugins to easily access files stored in a VCS provider without having to upload the files as release assets.

VCS providers can be created at the organization or project level, which offers more fine-grained control. An organization-scoped VCS provider can access all pipelines within the organization's projects. A project-scoped VCS provider can only access pipelines within the project where it was created.

Have a question?

Check the FAQ to see if there's already an answer.

Viewing VCS providers

To view VCS providers, select VCS Providers from the sidebar. This will display a list of all VCS providers in the organization or project. Use the search input to find a specific VCS provider by name. Select a VCS provider to view its details page.

VCS providers are inherited

VCS providers are inherited so a project will display a list of its project-scoped VCS providers as well as VCS providers from its parent organization. In a project, VCS providers inherited from the parent organization will be noted with an inherited from org chip.

VCS provider details page

The VCS provider details page displays the following information:

• Type: The type of VCS provider (e.g., GitHub, GitLab).
• URL: The API URL of the VCS provider.
• Authorization Type: The type of authorization used to authenticate Phobos with the VCS provider (e.g., OAuth, Personal Access Token).
• Extra OAuth Scopes: Additional OAuth scopes specified when creating the VCS provider. This field is only displayed for VCS providers that use OAuth for authentication.

Creating a VCS provider

Initial setup

1. Go to the organization or project sidebar and select VCS Providers. Then select New VCS Provider.
2. Select a VCS Provider type. Currently, Phobos supports GitHub and GitLab.
3. Enter a name for the VCS provider.
4. Enter a description (optional).
5. Enter the VCS provider's API URL (optional). If not provided, Phobos will use the default public API URL for the selected VCS provider.

Provider setup

Select an Authorization Type to authenticate and authorize Phobos to access the VCS provider. The following two authorization types are available:

• OAuth: Authenticate by creating an OAuth application with the VCS provider.
• Personal Access Token: Authenticate using a personal access token.

OAuth

• In a separate window, go to your host provider and start the process to create an OAuth application. Phobos will provide the following information:

  • GitLab:

    • Redirect URI

    • On your OAuth application, enable the Confidential setting and enable the following two scopes:

      • read_api
      • read_user

    

  • GitHub:

    • Homepage URL

    • Callback URL

    • Essential scopes like repo and read:user are always included by default to ensure basic functionality and are displayed in the UI. Note that GitHub does not support a read-only scope for repositories in classic OAuth applications, so the repo scope is necessary to access private repositories.

    

• You can add extra OAuth scopes as needed. For example, to use the GitLab plugin to start a pipeline, you might need to enable the api scope. These additional scopes must be included in the Extra OAuth Scopes field in the Provider setup form to prevent OAuth failures. Essential scopes are always added by default to ensure basic functionality like VCS volume mounting and are always displayed in the UI.

• When you have created the OAuth application in your host provider, you will be provided with an ID and a secret value. Copy and paste these values into the ID and secret fields on the New VCS Provider form.
• Click on Create VCS Provider. If creation is successful, Phobos will immediately generate a new authorization URL and redirect the browser to an approval page where you must authorize your newly created VCS provider to use your OAuth application.

• After you authorize the application, you will be redirected to the host provider's authorization confirmation page. If the authorization is successful, you will be redirected to a success page indicating that the VCS provider was successfully authorized. You can then close the browser window and return to the Phobos UI.

Personal Access Token

• In a separate window, go to your host provider and start the process to generate a personal access token.

  • GitLab:

    • Enable the following two scopes:

      • read_api
      • read_user

  • GitHub:

    GitHub supports two types of personal access tokens: Fine-grained token and Classic token. Phobos supports both types of tokens.

    • Fine grained token: This token is created with specific scopes. When creating a fine-grained token, ensure that the following scopes are enabled:

      • Contents (read-only)
      • Repository metadata (read-only)

    • Classic token: This token is created with full access to all repositories. When creating a classic token, ensure that the following scopes are enabled:

      • repo
      • read:user

• When you have created the personal access token in your host provider, copy and paste the token into the Personal Access Token field on the New VCS Provider form.

• Click on Create VCS Provider.

• If the VCS provider was successfully created, you will be navigated to the VCS provider's details page where you can view the details of the newly created VCS provider.

Updating a VCS provider

1. Navigate to the organization or project sidebar and select VCS Providers.
2. Click on the VCS provider you want to update.
3. Select Edit.
4. Update the description or API URL.
5. Select Update VCS Provider.

Edit Auth settings

OAuth

1. Navigate to the organization or project sidebar and select VCS Providers.
2. Click on the upside-down caret next to the Edit button and select Edit Auth Settings.
3. To change or clear the OAuth Application ID and Secret, tick the Change Application ID or Secret checkbox. Enter the new ID and Secret values in the respective fields.
4. To add or remove extra OAuth scopes, use the input on the Extra OAuth Scopes field.
5. Select Update VCS Provider.

Personal Access Token

1. Navigate to the organization or project sidebar and select VCS Providers.
2. Click on the upside-down caret next to the Edit button and select Edit Auth Settings.
3. Input the new personal access token in the Personal Access Token field.
4. Select Update VCS Provider.

Reset OAuth token

note

This option is only available for VCS providers that use OAuth for authentication.

1. Navigate to the organization or project sidebar and select VCS Providers.
2. Click on the upside-down caret next to Edit and select Reset OAuth Token.
3. In the confirmation dialog, select Reset OAuth Token to proceed. Click Cancel to close the dialog without resetting the OAuth token.

Deleting a VCS provider

1. Navigate to the organization or project sidebar and select VCS Providers.
2. Click on the VCS provider you want to delete.
3. Click on the upside-down caret next to Edit and select Delete VCS Provider.
4. In the confirmation dialog, select Delete to proceed. Click No to close the dialog without deleting the VCS provider.

Deletion is dangerous

Deleting a VCS provider is an irreversible operation.

Proceed with caution as deletion will sever your connection with your host provider. If unsure, do not proceed.

Testing your VCS provider

Below is a method to test your VCS provider connection. It also demonstrates a simple way to integrate your VCS provider with a pipeline and perform actions on it. You just need a private repository in your host provider to test this.

1. Create a new pipeline template:

pipeline.hcl

plugin exec {}

volume "demo_project" {
  type = "vcs"
  vcs_options {
    provider_id     = "<prn/vcs-provider>"
    ref             = "main"
    repository_path = "<username/repository-name>"
  }
}

stage "main" {
  task "list_files" {
    mount_point {
      volume = "demo_project"
    }
    action "exec_command" {
      command = <<EOF
        echo "Listing files mounted in VCS provider volume..."
        ls -la
      EOF
    }
  }
}

• The template demonstrates a task that lists files mounted in a VCS provider volume. It uses the built-in exec plugin for executing commands, mounts a version control system (VCS) repository as a volume, demo_project, and specifies a stage main containing a task list_files. Within this task, the template mounts the VCS volume and executes the exec_command, which prints a statement and lists the files mounted in the VCS provider volume.

• Before using this template to create a pipeline, in the vcs_options block, replace <prn/vcs-provider> with the PRN of your newly created VCS provider. Replace <username/repository-name> with the repository path you want to access, for example, if the repository is https://gitlab.com/user-name/vcs-provider-phobos-test, then repository_path would be user-name/vcs-provider-phobos-test.

2. cd into the directory where the pipeline template is saved (saved as pipeline.hcl) and run the following command:

   phobos pipeline validate

3. If the pipeline is valid, run the following command to execute the pipeline:

   phobos pipeline create --org-name "<organization name>" --type "runbook" <project-name>
   • Replace <organization name> with the name of the organization where the VCS provider is created. Replace <project-name> with the name of the project where the pipeline will run.

4. Navigate to the Phobos UI. Go to the target project and select Pipelines from the project sidebar. You should see a pipeline of type RUNBOOK. Going to its details page, should should see the main stage and the list_files task. Navigate to the task details page to see the output of the task.

   

Using a token from a VCS provider in a pipeline

Phobos allows you to securely use tokens from VCS providers directly within pipeline tasks, eliminating the need to pass VCS tokens as variables in pipeline templates. This feature supports both OAuth and personal access tokens.

Prerequisites

• A VCS provider must be created and authorized in Phobos. See Creating a VCS provider for more information.
• Optionally, depending on the use case, the OAuth scopes and access token permissions in the host provider and the VCS provider must be set correctly. See Provider setup for more information.

Tokens are sensitive

When using Personal Access Token (PAT) authentication, the token used in the pipeline is the same as the one provided during the VCS provider setup. Phobos does not generate a new token. Ensure the token's security and avoid exposing it in your pipeline.

When using OAuth authentication, the token used in the pipeline is dynamically generated and managed by Phobos. Ensure the OAuth application's security and avoid exposing sensitive information in your pipeline.

A feature to mask sensitive data in the logs may be available in future releases.

Using a VCS token in a pipeline template

See here for more information.

Using a VCS token in a lifecycle template

See here for more information.

Frequently Asked Questions (FAQ)

After creating a VCS provider with OAuth, why is the authorization link not working or showing an error page?

Check that your API URL is valid. Try updating your OAuth credentials in your host provider by reobtaining an ID and secret value. Also, ensure that the Extra OAuth Scopes field is correctly set with the necessary scopes and matches the scopes in your host provider. When you have these values, you can Edit Auth Credentials and then Reset OAuth Token.

I've tried the example from Testing your VCS provider, but the pipeline is failing or just not running. What could be the issue?

Here are a few things to check:

vcs_options block in your pipeline template

• Verify that the provider_id is using a valid VCS provider GID or PRN and that the repository_path is correct. Ensure that ref is set to the appropriate branch or tag name (note: ref is optional).

Scopes and permissions in your host provider

• Ensure that the minimum scopes and permissions suggested in the Provider setup section are correctly configured in your host provider. These settings should allow connections to both public and private repositories. If using OAuth, verify that the Extra OAuth Scopes field matches the necessary scopes in your host provider.

Confirm the scope of your VCS provider.

• Verify that the VCS provider is scoped correctly. If it is project-scoped, ensure the pipeline runs in the appropriate project. If it is organization-scoped, ensure the pipeline runs in a project within that organization.

If you are having trouble running the pipeline create command, check the CLI documentation for more information.

Can organization-scoped VCS providers be inherited by projects?

Yes, organization-scoped VCS providers are inherited by projects within the organization. When you create a project-scoped VCS provider, it will only be accessible within that project.

Can I change the authorization type after creating a VCS provider?

No, you cannot change the authorization type directly. You will need to delete the existing VCS provider and create a new one with the desired authorization type.