What is Digger?

Digger is an open-source CI/CD orchestrator for Terraform which helps you run Terraform in your own CI setup. It seamlessly integrates with our existing CI setup, automating Terraform tasks within our pipeline. With enhanced security features like RBAC via OPA, Digger ensures sensitive data remains within your CI environment. Its cost-effective approach utilizes existing infrastructure, making it an efficient solution for managing Terraform deployments.

How Digger works

As mentioned earlier, Digger helps you run Terraform in your own CI setup. To power this Digger operates through two main components:

1. CLI Agent: The CLI Agent is a core part of Digger that lives inside our CI setup. It's like having a special assistant dedicated to handling Terraform tasks. This agent directly interacts with the Terraform Command-Line Interface (CLI), making it easy to give commands and get things done without any hassle.

2. Orchestrator Backend: The Orchestrator Backend acts as the brain of Digger. It keeps track of what's happening in your GitHub repository, such as when pull requests are opened. When it detects these events, it kicks off the necessary tasks to handle them, ensuring everything runs smoothly behind the scenes.

   Note: Digger can also run as a standalone GitHub Action without a backend but there are some drawbacks in it which are as follows:

   Drawbacks:
   Delayed Updates: Updates may be delayed because of the independent execution mode for comments and status checks.
   Sequential Execution: In a single task, all applications execute one after the other without concurrency, which could result in lengthier execution durations for complicated deployments.
   Handling Clashing : If clashing applies from other tasks are not queued, they will not succeed, which could lead to disputes and disturbances in the deployment procedures.
   Manual Configuration: In order to enable PR-level locks in their cloud account, users must manually create buckets or tables, which adds time and effort to the setup procedure.

   Digger's workflow:

3. PR Event Trigger: Digger's orchestrator backend recognizes the event when a pull request is opened in the GitHub repository.

4. CI Job Initiation: Using Digger's CLI agent, the orchestrator backend starts a CI job inside the current CI environment.

5. Terraform Plan Execution: To create an execution plan for the suggested changes, the CI task runs the "terraform plan" command.

6. Plan Output Posting: Digger posts the plan output as a comment on the PR, allowing team members to review the proposed changes.

7. Apply Command: The CI job runs the terraform apply command to apply the modifications after receiving confirmation from a team member, usually by writing "digger apply" on the PR.

What Digger solves

Those who are familiar with the Terraform knows that there is a fundamental difference between application code and terraform code which is the terraform state which makes CI/CD for it difficult to manage. So let's first see the problem that might arise while using Terraform in CI/CD and how Digger tackles these problems:

Problem 1:

Infrastructure Redundancy: Duplicating CI/CD infrastructure due to the use of specialized CI solutions like Terraform Cloud or Atlantis (TACOs) results in needless overheads such as redundant computing resources, complicated job management, and higher resource expenses.

Digger's Solution: Digger avoids duplication by utilizing your existing continuous integration (CI) system to remove the requirement for additional infrastructure configuration. This guarantees effective use of resources and reduces the overhead expenses related to running different continuous integration systems.

Problem 2:

Security Issues: Giving admin access to cloud accounts for CI/CD tools exposes critical resources and sensitive data to potential vulnerabilities, posing a serious security risk. Unauthorized alterations, data breaches, and other issues can result from unauthorized access to cloud resources.

Digger's Solution: Digger ensures that secrets and sensitive data are not shared with third-party services, maintaining confidentiality and mitigating the risk of unauthorized access. Additionally, Digger offers Role-Based Access Control (RBAC) through Open Policy Agent (OPA), allowing organizations to define granular roles and permissions. This RBAC feature enables precise control over access to sensitive resources and actions within the CI/CD environment.
Note: RBAC via OPA is an enterprise feature.

Digger Features:

1. Comprehensive Dashboard: The Digger Dashboard serves as a user interface (UI) that facilitates the management of projects, connected repositories, plan/apply runs, and OPA policies.

2. OPA Policies: Digger uses Open Policy Agent (OPA) to facilitate granular policy-as-code governance. Policies can be specified both at the project and organizational levels.
   Digger offers two different kinds of policies: Plan Access & Access policies

3. Concurrency: Using Digger plans and applications, jobs that are independent of one another operate concurrently. This significantly speeds up execution of several state files.
   Instead of operating on its own, Digger is an orchestrator that launches tasks within your CI. Concurrency is therefore inherent to this architecture.

4. PR-Level Locks: Digger's PR-level locks feature makes sure that modifications contributed to a pull request (PR) don't conflict with one another while being handled. It keeps plan previews accurate by averting interruptions until the PR is integrated. This feature improves the dependability of your development workflow by providing flexibility in configuration, having an easy-to-use interface, and handling many projects with ease.

5. RBAC: Digger offers robust role-based access control (RBAC), tightly integrated with GitHub users, teams, and organizations. This allows for granular control over permissions and access levels within your projects.

Getting Started with Digger:

• Digger official documentation has cloud specific starting guide which you can follow step-by-step.

• Github Actions+AWS

• Github Actions+GCP

• Github Actions+Azure

• 

Digger Pricing:

While Digger is an open-source platform, it offers two tiers: Community Edition and Enterprise. These tiers are specifically tailored to cater to a diverse range of user requirements. You can check the official pricing document from here

Conclusion:

Digger is a very attractive option that offers cost-effectiveness, improved security features, and smooth integration for managing Terraform activities in CI/CD pipelines. However, it's important to take into account a number of aspects, such as deployment simplicity, learning possibilities, and community support, while deciding whether to use Digger.