Skip to content

Commit 8cca41e

Browse files
motatoesmotatoes
authored
local dev support (#2088)
* support local dev command
1 parent 413a1d4 commit 8cca41e

File tree

6 files changed

+222
-5
lines changed

6 files changed

+222
-5
lines changed

‎CONTRIBUTING.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ and we would be happy to set you up.
2020

2121
## Introduction
2222

23-
Digger is an open source terraform cloud alternative. We believe that open source software is important, and we welcome contributions from anyone who is interested in making our tool better.
23+
Digger is an open source terraform cloud alternative. We believe that open source software is important, and we welcome contributions from anyone who is interested in making our platform better.
2424
This document is intended to be a guide for people who want to contribute to Digger. We appreciate all contributions, no matter how big or small.
2525

2626
## How to contribute
@@ -51,6 +51,7 @@ libs/ # contains libraries that are common between digger cli and digger cloud b
5151
cli/cmd/ # contains the main cli files
5252
cli/pkg/ # contains packages that are used by the cli code, can import from libs/
5353
backend/ # contains the backend code, can import from libs/
54+
ee/ # contains the ee code, can import from libs/ and the other ce packages (but ce must not import from ee)
5455
docs/ # contains documentation pages
5556
```
5657

@@ -64,7 +65,7 @@ When you have made changes to the codebase that you would like to contribute bac
6465
4. Test out your changes in a demo GitHub repository using Github Actions.
6566

6667
- You can test out the changes from your fork by referencing the Action within a Github workflow file: `uses: <github-username>/digger@your-branch`.
67-
- Fork [this demo repository](https://github.com/diggerhq/digger_demo_multienv) to setup and assert your tests.
68+
- Fork [this demo repository](https://github.com/diggerhq/demo-opentofu) to setup and assert your tests.
6869
- If you're adding new app level inputs that implies new environment variables, make sure to reference them **both within the `build digger` and `run digger` steps** in [`action.yml`](./action.yml).
6970

7071
5. Update the documentation to reflect your changes, if applicable.
@@ -81,6 +82,11 @@ When you have made changes to the codebase that you would like to contribute bac
8182
- Once we are ready we tag the head of our release branch and perform a release on it.
8283
- Tagged releases are published as actions and they are the most suitable to be used in production.
8384

85+
86+
## How to test locally
87+
88+
The easiest way to test locally is to follow our [Testing locally](https://docs.digger.dev/ce/contributing/setup-dev-environment) instructions.
89+
8490
## Code of Conduct
8591

8692
We expect all contributors to follow our [Code of Conduct](https://www.contributor-covenant.org/version/2/1/code_of_conduct/) when participating in our community. Please read it carefully before contributing.

‎action.yml‎

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,14 @@ description: Manage terraform collaboration
44
author: Digger
55

66
inputs:
7+
local-dev-mode:
8+
description: run digger for local development?
9+
required: false
10+
default: false
11+
local-dev-cli-path:
12+
description: The path to where the compiled digger cli on the self-hosted runner exists (absolute path)
13+
required: false
14+
default: './digger'
715
ee:
816
description: use ee cli?
917
required: false
@@ -365,7 +373,9 @@ runs:
365373

366374
- name: Copy Digger CLI go.sum for cache key
367375
run: |
368-
if [[ ${{ inputs.ee }} == "true" ]]; then
376+
if [[ "${{ inputs.local-dev-mode }}" == "true" ]]; then
377+
echo "Digger CLI is in local dev mode, skipping go.sum copy"
378+
elif [[ ${{ inputs.ee }} == "true" ]]; then
369379
cp "$GITHUB_ACTION_PATH/ee/cli/go.sum" "$GITHUB_WORKSPACE/.digger.go.sum"
370380
else
371381
cp "$GITHUB_ACTION_PATH/cli/go.sum" "$GITHUB_WORKSPACE/.digger.go.sum"
@@ -399,7 +409,7 @@ runs:
399409

400410

401411
- name: build and run digger
402-
if: ${{ !startsWith(github.action_ref, 'v') }}
412+
if: ${{ !startsWith(github.action_ref, 'v') && inputs.local-dev-mode == 'false' }}
403413
shell: bash
404414
env:
405415
PLAN_UPLOAD_DESTINATION: ${{ inputs.upload-plan-destination }}
@@ -448,7 +458,7 @@ runs:
448458
digger
449459
450460
- name: run digger
451-
if: ${{ startsWith(github.action_ref, 'v') }}
461+
if: ${{ startsWith(github.action_ref, 'v') && inputs.local-dev-mode == 'false' }}
452462
env:
453463
actionref: ${{ github.action_ref }}
454464
PLAN_UPLOAD_DESTINATION: ${{ inputs.upload-plan-destination }}
@@ -495,6 +505,70 @@ runs:
495505
cd $GITHUB_WORKSPACE
496506
digger
497507
508+
- name: run digger in local dev mode
509+
if: ${{ inputs.local-dev-mode == 'true' }}
510+
env:
511+
actionref: ${{ github.action_ref }}
512+
PLAN_UPLOAD_DESTINATION: ${{ inputs.upload-plan-destination }}
513+
PLAN_UPLOAD_S3_ENCRYPTION_ENABLED: ${{ inputs.upload-plan-destination-s3-encryption-enabled }}
514+
PLAN_UPLOAD_S3_ENCRYPTION_TYPE: ${{ inputs.upload-plan-destination-s3-encryption-type }}
515+
PLAN_UPLOAD_S3_ENCRYPTION_KMS_ID: ${{ inputs.upload-plan-destination-s3-encryption-kms-key-id }}
516+
PLAN_UPLOAD_AZURE_STORAGE_CONTAINER_NAME: ${{ inputs.upload-plan-destination-azure-container }}
517+
PLAN_UPLOAD_AZURE_STORAGE_ACCOUNT_NAME: ${{ inputs.upload-plan-destination-azure-storage-account }}
518+
GOOGLE_STORAGE_LOCK_BUCKET: ${{ inputs.google-lock-bucket }}
519+
GOOGLE_STORAGE_PLAN_ARTEFACT_BUCKET: ${{ inputs.upload-plan-destination-gcp-bucket }}
520+
AWS_S3_BUCKET: ${{ inputs.upload-plan-destination-s3-bucket }}
521+
ACTIVATE_VENV: ${{ inputs.setup-checkov == 'true' }}
522+
DISABLE_LOCKING: ${{ inputs.disable-locking == 'true' }}
523+
DIGGER_PRIVATE_KEY: ${{ inputs.digger-private-key }}
524+
DIGGER_TOKEN: ${{ inputs.digger-token }}
525+
DIGGER_ORGANISATION: ${{ inputs.digger-organisation }}
526+
DIGGER_HOSTNAME: ${{ inputs.digger-hostname }}
527+
DIGGER_FILENAME: ${{ inputs.digger-filename }}
528+
ACCUMULATE_PLANS: ${{ inputs.post-plans-as-one-comment == 'true' }}
529+
REPORTING_STRATEGY: ${{ inputs.reporting-strategy }}
530+
INPUT_DIGGER_PROJECT: ${{ inputs.project }}
531+
INPUT_DIGGER_MODE: ${{ inputs.mode }}
532+
INPUT_DIGGER_COMMAND: ${{ inputs.command }}
533+
INPUT_DRIFT_DETECTION_SLACK_NOTIFICATION_URL: ${{ inputs.drift-detection-slack-notification-url }}
534+
INPUT_DRIFT_DETECTION_ADVANCED_SLACK_NOTIFICATION_URL: ${{ inputs.drift-detection-advanced-slack-notification-url }}
535+
NO_BACKEND: ${{ inputs.no-backend }}
536+
TG_PROVIDER_CACHE: ${{ inputs.cache-dependencies == 'true' && 1 || 0 }}
537+
TERRAGRUNT_PROVIDER_CACHE: ${{ inputs.cache-dependencies == 'true' && 1 || 0 }}
538+
DIGGER_RUN_SPEC: ${{inputs.digger-spec}}
539+
id: digger-local-run
540+
shell: bash
541+
run: |
542+
set -euo pipefail
543+
544+
cd $GITHUB_WORKSPACE
545+
546+
echo "🚀 Running digger..."
547+
RAW="${{ inputs.local-dev-cli-path }}"
548+
549+
# Validate path to prevent command injection
550+
if [[ "$RAW" =~ [^a-zA-Z0-9_./-] ]]; then
551+
echo "❌ Invalid characters in local-dev-cli-path"
552+
exit 1
553+
fi
554+
555+
if "$RAW" in [!A-Za-z0-9_./-]; then
556+
echo "❌ bad chars" ; exit 1
557+
fi
558+
559+
if [[ "$RAW" =~ \.\. || "$RAW" == : || "$RAW" != /* ]]; then
560+
echo "❌ traversal/colon/relative not allowed"; exit 1
561+
fi
562+
563+
DIR=$(realpath -- "$RAW") || { echo "❌ not found"; exit 1; }
564+
[[ -d "$DIR" ]] || { echo "❌ not a dir"; exit 1; }
565+
566+
BIN="$DIR/digger"
567+
[[ -x "$BIN" ]] || { echo "❌ digger not executable at $BIN"; exit 1; }
568+
569+
$BIN
570+
echo "✅ digger completed"
571+
498572
- uses: actions/cache/save@v4
499573
name: cache-save
500574
if: ${{ always() && inputs.cache-dependencies == 'true' && steps.restore_cache.outputs.cache-hit != 'true' }}

‎cli/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
**/digger
22
!/pkg/digger
33
!/cmd/digger
4+
actions-runner/

‎docs/ce/contributing/setup-dev-environment.mdx‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
---
2+
title: "Setting up a local development environment"
3+
---
4+
5+
6+
## Setting up the backend
7+
8+
To get started running the backend service you will need:
9+
10+
- digger’s repo cloned https://github.com/diggerhq/digger
11+
- some kind of ngrok to tunnel connections to port 3000 - this is needed to receive github webhooks to the service
12+
- go 1.24 running on your machine
13+
14+
<Steps>
15+
<Step title="Install ngrok">
16+
You can use ngrok to tunnel connections to port 3000. You can find installation instructions [here](https://ngrok.com/docs/getting-started).
17+
The reason we need this is so that we can receive github webhooks to the service
18+
</Step>
19+
20+
<Step title="Create a local database">
21+
I usually start one with docker:
22+
23+
```
24+
docker run --name digger-pg \
25+
-e POSTGRES_PASSWORD='SuperSecurePasswordWithAtLeast14Characters' \
26+
-e POSTGRES_USER=postgres \
27+
-e POSTGRES_DB=digger \
28+
-p 54312:5432 \
29+
-d postgres:15
30+
```
31+
32+
which will create this connections tring:
33+
34+
```
35+
export DATABASE_URL=postgres://postgres:SuperSecurePasswordWithAtLeast14Characters@localhost:54312/digger?sslmode=disable
36+
```
37+
</Step>
38+
39+
<Step title="Clone the repo">
40+
If you are planning to contribute you would need to fork the repo and clone it locally:
41+
42+
```
43+
# replace diggerhq with your username
44+
git clone https://github.com/diggerhq/digger.git
45+
```
46+
</Step>
47+
48+
<Step title="Creating environment variables">
49+
Create a `.env` file in the folder backend/ of the repo and add the following:
50+
```
51+
GITHUB_ORG=your_github_org # this is your github org where an app will be installed, leave it out to install in your personal account
52+
HOSTNAME=your_public_digger_hostname # the ngrok hostname (include https://)
53+
DATABASE_URL=postgres://.....
54+
HTTP_BASIC_AUTH=1
55+
HTTP_BASIC_AUTH_USERNAME=mydiggerorg
56+
HTTP_BASIC_AUTH_PASSWORD=$(openssl rand -base64 12)
57+
ALLOW_DIRTY=false # set to true if the database has already a schema configured
58+
```
59+
60+
note that you would need to source this file before running the backend service
61+
62+
```
63+
set -a
64+
source .env
65+
set +a
66+
```
67+
68+
And after that you can run the service with:
69+
70+
```
71+
go run backend/main.go
72+
```
73+
74+
You should see a message that the service is listening on port 3000. Visiting that url should show you a ui screen that digger is up and running.
75+
</Step>
76+
77+
<Step title="Next steps">
78+
From here on you can follow the steps in the self hosting binary guide [here](/ce/self-host/deploy-binary)
79+
</Step>
80+
81+
</Steps>
82+
83+
## Setting up the cli
84+
85+
The cli is mainly meant to be used and invoked in CI systems. So in order to test cli changes the easiest way is to push it to your digger branch and then update the step to point to the branch directly:
86+
87+
```
88+
- name: digger
89+
uses: diggerhq/digger@feat/my-test-branch
90+
with:
91+
....
92+
```
93+
94+
Digger will then build the cli on your test runs and so you can test your changes from branch. It can be time consuming to build the cli on every run so we also added a way to use it from local
95+
via self-hosted runners on your machine. In order to achieve this you would need to create a self-hosted runner connected to your repo.
96+
97+
Visit the repo with your test terraform and then click on "actions" -> "runners" -> "self-hosted runners" -> "new runner".
98+
Follow the instructions for your local machine and to set it up. Once set up you should see it in the list of runners.
99+
100+
From here you will need to build a version of your cli locally.
101+
102+
```
103+
cd cli
104+
go build -o digger ./cmd/digger
105+
chmod +x digger
106+
```
107+
108+
Test run the local cli - it should display a message like "no CI detected". Now we can modify our workflow file to use our self-hosted runner and to invoke the local cli during processing:
109+
110+
```
111+
- name: digger
112+
uses: diggerhq/digger@vLatest # this version doesn't matter when using local dev
113+
with:
114+
# ...
115+
local-dev-mode: "true"
116+
local-dev-cli-path: "/Users/myname/dev/digger/digger/cli"
117+
```
118+
119+
The value of "local-dev-cli-path" should be an absolute path to the location compiled binary from the previous step (excluding the binary name). You should see this line when you try to run it:
120+
121+
![](/images/contributing/self-hosted-cli-local-dev.png)
122+
123+
> **Security Warning**: Self-hosted runners have significant security implications. When using self-hosted runners, be aware that:
124+
> - Code from pull requests can run on your self-hosted runner, potentially executing malicious code on your machine
125+
> - If your repository is public, anyone can fork it and submit a pull request that could run code on your runner
126+
> - Only use self-hosted runners in private repositories where you trust all contributors, or implement proper security controls
127+
> - For more information, see [GitHub's security considerations for self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners#self-hosted-runner-security)
128+
129+
From here onwards the cycle is to make a local change, rebuild the cli and then trigger in github to test that change. Its a much faster iterative cycle in comparison to building from a branch each time.

‎docs/images/contributing/self-hosted-cli-local-dev.png‎

168 KB
Loading

‎docs/mint.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,13 @@
155155
"ce/azure-specific/azure-devops-locking-connection-methods"
156156
]
157157
},
158+
{
159+
"group": "Contributing",
160+
"pages": [
161+
"ce/contributing/setup-dev-environment"
162+
]
163+
},
164+
158165
{
159166
"group": "Troubleshooting",
160167
"pages": [

0 commit comments

Comments
 (0)