Some time ago I became part of a team that, after months of working with third-party software, decided to go with their own tailored solution. The third-party software supported custom code, and various required features were implemented by the developers to somehow fulfill the requirements. However, such setups tend not to be sustainable or reliable.
As soon as the decision to start developing was made, I recommended implementing a lean and simple CI/CD Pipeline, even if, in the beginning, the code was only a few thousand lines long.
Such decisions in the early stages will pay off later on as the project itself, the requirements, and the lines of code grow.
Technical Stack
Since I do not want to reveal too much information, I will only talk about the two most important components. The programming language in question was Go, and Git was the chosen VCS (Version Control System).
Lean pipeline with a focus on security
My goal was to create a CI/CD Pipeline with a focus on security as well – not only automating the build and release.
I split my pipeline into six different parts:
- Linting with
golangci-lint - Creating a Go-specific SCA with
govulncheck - Scanning with gitleaks
- Generating an SBOM and scanning it with
grype - Performing SAST with
gosec - Building and releasing
A single main.yml orchestrates the task order and dependencies to ensure that each stage gates the next.
jobs:
linting:
name: Linting & govulncheck
uses: ./.gitea/workflows/linting.yml
secret_scanner:
name: Scanning repository for secrets
uses: ./.gitea/workflows/secret_scanner.yml
secrets: inherit
sbom:
name: Creating Software Bill of Materials (SBOM)
uses: ./.gitea/workflows/sbom.yml
sast:
name: Running a Static Application Security Test with gosec
uses: ./.gitea/workflows/sast.yml
build:
name: Building & releasing the binary
needs: [linting, secret_scanner, sast, sbom]
uses: ./.gitea/workflows/build.yml
secrets: inheritLinting
Given that Go was the language in question, I used the official golangci-lint GitHub Action. The result is then saved to a file, which the developers can view in order to fix linting issues.
This had a massive effect. The pipeline was introduced after about 4.000 lines of code were already written, the linting job found minor issues which were immediately addressed by the developers.
- name: Run golangci-lint (latest)
uses: golangci/golangci-lint-action@v8Scanning with gitleaks
There are hundreds, if not thousands, of stories where credentials were leaked in source code or commits and nobody noticed. This is exactly why I introduced the gitleaks GitHub Action into this pipeline.
Note that to use gitleaks you need to obtain a license key, which should then be saved as a secret for your repository or organization.
To be certain that gitleaks was working, I committed a dummy string with the prefix OPENAI_API_KEY=. Gitleaks immediately stopped the pipeline.
Keys and passwords that are accidentally committed must be removed from the code and rotated.
- name: Scan with Gitleaks
uses: gitleaks/gitleaks-action@v2govulncheck
govulncheck performs a Go-specific SCA (Software Composition Analysis). It looks up the build list in go.mod and go.sum. Afterwards, it checks the Go Vulnerability Database to verify whether any used module has a known vulnerability.
The key feature, however, is that it analyzes the codes call graph to verify whether the application is actually calling a vulnerable function. If that is the case, govulncheck exits with a non-zero status code. If any job fails with a non-zero status code, the pipeline comes to a stop.
As with nearly all of the other tasks, the results are uploaded so they can be verified by the developers if any problems occur.
- name: Run govulncheck
uses: golang/govulncheck-action@v1SAST – Static Application Security Testing
Here, another GitHub Action comes into play. On every run gosec inspects the source code and reports any vulnerable code found. Vulnerabilities can range from the use of insecure hashing algorithms to potential integer overflows or injection points.
As soon as any vulnerability is found, the pipeline stops. Developers must verify and fix the issue before the pipeline can be successfully executed again.
- name: Run Gosec Security Scanner
uses: securego/gosec@2SBOM and scanning with grype
In order to also cover any possible container vulnerabilities, I implemented two Anchore GitHub Actions:
The SBOM job creates a container with the application running inside. This is done to represent the application running in a production environment. The SBOM GitHub Action first creates the software bill of materials – for both the application and the running container.
Grype takes this SBOM and compares every single item against multiple public vulnerability databases in order to verify whether there are known vulnerabilities inside the software, libraries, or modules used.
If this task exits with a non-zero status code, the pipeline comes to a stop. Results are uploaded so they can be reviewed if any issues occur.
- name: Generate SBOM
uses: anchore/sbom-action@v0
[SNIP..]
- name: Scan SBOM for vulnerabilities
uses: anchore/scan-action@v7Build and Release
Only after every job exits successfully is the final binary built and then released to the dedicated environment.
Conclusion
While shipping fast is great, shipping fast and safely is better. From the beginning, I wanted a lean and secure pipeline that not only performs some tasks but also really helps every developer.
Skipping the basics in the beginning will come back later as technical debt which itself comes with many different issues. I personally think that every software project does need a lean pipeline which ensures that at least the fundamentals are implemented correctly.
This pipeline is, of course, not the end. There are many different things that will have to be integrated into the pipeline in the future. One of those things is, for example, DAST (Dynamic Application Security Testing).
However, when I started this pipeline from scratch, the main goal was a lean and secure pipeline. As of now, this goal is achieved, and it already pays off. The pipeline initially found linting issues. The Grype scan also found vulnerabilities which stopped the pipeline:
[SNIP..]
[0036] ERROR discovered vulnerabilities at or above the severity threshold
{
"package": "golang.org/x/xyz",
"vuln": "xyz",
"severity": "Medium"
}
{
"package": "golang.org/x/xyz",
"vuln": "xyz",
"severity": "Medium"
}
[SNIP..]Additionally, Gosec has also helped to improve the code base:
[SNIP..]
"message":
{
"text": "integer overflow conversion int -\u003e uint32"
},
[SNIP..]In the future, this pipeline will grow and improve the Secure Software Development Lifecycle (SSDLC) from which each involved party will benefit.