28 - Secret Manager & Cloud Build CI-CD

Welcome to Day 28 of Learn GCP in 30 Days!

Yesterday, you conquered BigQuery and learned how to query terabytes of analytical data in seconds. Today, we step into the critical worlds of Cloud Security and DevOps Automation: Google Cloud Secret Manager and Google Cloud Build.

😱 The "Aha!" Moment: The $10,000 2:00 AM GitHub Disaster

Imagine a junior developer named Sam building a weather app late at night:

Why did this happen? Because automated scraper bots continuously scan millions of GitHub commits every single second. Even if you realize your mistake and delete the commit 60 seconds later, it's already too late—the secret was cached by bots the moment it touched GitHub!

To solve this, modern engineering teams follow one absolute rule: Never store secrets in source code, .env files, or Dockerfiles.

🛑 The Core Problem: Manual Deployments & Leaked Secrets

What Existed Previously:

1. Hardcoded Secrets: Passwords and API keys were pasted directly into configuration files or environment variables on virtual machines.
2. Manual Deployments: Developers built Docker images locally on their laptops, manually ran SSH commands to log into production servers, and restarted containers by hand.

Problems Faced:

• 🚨 Catastrophic Security Leaks: Secrets ended up in git logs, Docker image layers, and bash terminal histories.
• 🐌 "It Works on My Machine" Syndrome: If developer A had Node 18 on their MacBook and developer B had Node 20 on Windows, builds broke unpredictably.
• ⏳ Slow, Error-Prone Deployments: Releasing a 1-line bug fix required 20 manual terminal steps.
• 🏗️ Heavy Jenkins Server Maintenance: Teams spent days managing, patching, and paying for dedicated 24/7 CI/CD virtual machines that sat idle most of the day.

How Present Technology Solves It:

Google created a seamless, serverless duo:

1. Google Cloud Secret Manager: An ultra-secure, encrypted vault that stores passwords, certificates, and API keys with versioning and granular IAM access controls.
2. Google Cloud Build: A 100% serverless CI/CD automation platform that executes your build, test, and deployment steps inside fast Docker containers on demand ($0.00 when idle, with 120 free build-minutes every single day!).

🎭 What People Believe vs. What Actually Is

🧠 Myth #1: "My GitHub repo is private, so storing API keys in my code is fine!"

• What People Believe: If nobody else can see my repo, my secrets are secure.
• What Actually Happens: Private repos get cloned by interns, contractors, and CI tools. Even worse, secrets get baked permanently into Docker image layers, making them visible to anyone who pulls the container image.

🧠 Myth #2: "Setting up CI/CD requires a full-time DevOps engineer to manage Jenkins."

• What People Believe: You need a dedicated server cluster running 24/7 with 50 plugins just to automate deployments.
• What Actually Happens: Cloud Build requires zero servers. You write a 15-line cloudbuild.yaml file, and Google runs your pipeline on massive, high-speed infrastructure in seconds.

🧠 Myth #3: "Passing plain environment variables is secure."

• What People Believe: If I write API_KEY=12345 in my deployment script, it's safe.
• What Actually Happens: Plain environment variables are displayed in clear text in the GCP Console and logs. Secret Manager encrypts your data with Google Cloud KMS keys, tracks version history (v1, v2), and logs every access attempt in Cloud Audit Logs.

🏛️ Real-World Analogies

🔑 Deep Dive 1: Google Cloud Secret Manager

Secret Manager is Google's centralized, highly available credential store.

🌟 Key Features of Secret Manager

1. First-Class Versioning: You never overwrite a secret. You add a new version (v1 $\rightarrow$ v2). If a key rotation breaks, you can instantly roll back to v1.
2. Granular IAM Control: Grant access to specific applications using the roles/secretmanager.secretAccessor role without giving admin permissions.
3. Automatic Cloud Run Integration: Cloud Run can mount secrets directly as environment variables or volume files in memory—secrets are never written to disk.
4. Generous Free Tier: Google gives you 6 active secret versions for FREE every month!

⚡ Deep Dive 2: Google Cloud Build & cloudbuild.yaml

Cloud Build is Google Cloud's serverless build, test, and release engine.

📜 Anatomy of a cloudbuild.yaml Pipeline

Every Cloud Build pipeline is defined by a simple YAML file containing sequential Steps:

steps:
  # Step 1: Run automated unit tests
  - name: 'python:3.11-slim'
    entrypoint: 'pytest'
    args: ['tests/']

  # Step 2: Build the Docker container image
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$COMMIT_SHA', '.']

  # Step 3: Push image to Artifact Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$COMMIT_SHA']

  # Step 4: Deploy to Cloud Run
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: 'gcloud'
    args:
      - 'run'
      - 'deploy'
      - 'my-app'
      - '--image=us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$COMMIT_SHA'
      - '--region=us-central1'
      - '--allow-unauthenticated'

# Store images in Artifact Registry
images:
  - 'us-central1-docker.pkg.dev/$PROJECT_ID/my-repo/my-app:$COMMIT_SHA'

🛠️ Step-by-Step Hands-On Lab: Build an Automated Secure Pipeline

Let's build a real secure CI/CD pipeline that creates a secret, injects it into a Python app, and deploys it automatically using Cloud Build!

Step 1: Initialize Project & Enable APIs

Open Google Cloud Shell and run:

# 1. Set environment variables
export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_NUMBER=$(gcloud projects describe ${PROJECT_ID} --format='value(projectNumber)')
export REGION="us-central1"

# 2. Enable Secret Manager, Cloud Build & Cloud Run APIs
gcloud services enable \
    secretmanager.googleapis.com \
    cloudbuild.googleapis.com \
    run.googleapis.com

🔍 Under the Hood & Parameter Breakdown:

• PROJECT_ID=$(gcloud config get-value project): Extracts the human-readable project ID string (e.g. my-gcp-project-12345).
• PROJECT_NUMBER: Queries Google Resource Manager to fetch the unique numeric project number (e.g. 834719283741). This is required because GCP creates default service accounts using the project number format (e.g. <PROJECT_NUMBER>-compute@...).
• gcloud services enable: Activates the underlying Google Cloud REST APIs for Secret Manager, Cloud Build, and Cloud Run.

Step 2: Create a Secret in Secret Manager

Let's create a secret named my-api-key containing a mock high-security payment token:

# 1. Create the secret metadata container
gcloud secrets create my-api-key \
    --replication-policy="automatic"

# 2. Add the first secret version (the actual payload)
echo -n "super-secret-stripe-token-xyz-2026" | \
    gcloud secrets versions add my-api-key --data-file=-

🔍 Under the Hood & Parameter Breakdown:

Step 3: Grant IAM Permissions to Cloud Run & Cloud Build

For Cloud Run to read the secret, and for Cloud Build to deploy the app, we configure least-privilege IAM bindings:

# 1. Grant Secret Accessor role to Compute Engine default service account (used by Cloud Run)
gcloud secrets add-iam-policy-binding my-api-key \
    --member="serviceAccount:${PROJECT_NUMBER}-compute@developer.gserviceaccount.com" \
    --role="roles/secretmanager.secretAccessor"

# 2. Grant Cloud Run Admin role to Cloud Build service account so it can deploy
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
    --member="serviceAccount:${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com" \
    --role="roles/run.admin"

# 3. Grant Service Account User role to Cloud Build
gcloud projects add-iam-policy-binding ${PROJECT_ID} \
    --member="serviceAccount:${PROJECT_NUMBER}@cloudbuild.gserviceaccount.com" \
    --role="roles/iam.serviceAccountUser"

🔍 Under the Hood & Parameter Breakdown:

• add-iam-policy-binding my-api-key: Attaches an IAM policy directly to the secret resource itself (Resource-level Least Privilege!).
• --member="serviceAccount:...": Identifies the exact robot identity receiving access.
• --role="roles/secretmanager.secretAccessor": Grants read-only decryption permission for secret payloads. It does not allow deleting, modifying, or managing secret permissions.
• roles/run.admin & roles/iam.serviceAccountUser: Allows Cloud Build's automated runner to deploy container revisions to Cloud Run and attach the runtime identity.

Step 4: Write the Application Code & Pipeline

Let's create our project directory and files:

mkdir -p ~/secure-cicd-app
cd ~/secure-cicd-app

1. Author main.py (Reads Secret from Environment)

⚡ Option A: Fast 1-Click Way (Copy & Paste):

cat << 'EOF' > main.py
import os
from flask import Flask, jsonify

app = Flask(__name__)

@app.route("/", methods=["GET"])
def home():
    # Fetch secret injected by Cloud Run from Secret Manager
    api_key = os.environ.get("MY_API_KEY", "NOT_FOUND")
    
    # Mask key for security: show only first 4 and last 4 characters
    masked_key = f"{api_key[:4]}****{api_key[-4:]}" if len(api_key) > 8 else "HIDDEN"
    
    return jsonify({
        "status": "success",
        "message": "🚀 App deployed securely via Cloud Build CI/CD!",
        "secret_injected": "YES" if api_key != "NOT_FOUND" else "NO",
        "masked_secret": masked_key
    })

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))
EOF

📝 Option B: Manual Way with Nano:

nano main.py

(Paste the code above, press Ctrl+O $\rightarrow$ Enter to save, then Ctrl+X to exit).

2. Author requirements.txt & Dockerfile

cat << 'EOF' > requirements.txt
Flask==3.0.3
gunicorn==22.0.0
EOF

cat << 'EOF' > Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
ENV PORT=8080
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
EOF

3. Author the CI/CD Pipeline (cloudbuild.yaml)

This pipeline will build the container, push it, and deploy it to Cloud Run with the Secret Manager secret securely mounted!

cat << 'EOF' > cloudbuild.yaml
steps:
  # Step 1: Build Docker Container
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/secure-app:$BUILD_ID', '.']

  # Step 2: Push Container to Registry
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/secure-app:$BUILD_ID']

  # Step 3: Deploy to Cloud Run & Inject Secret Manager Secret
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: 'gcloud'
    args:
      - 'run'
      - 'deploy'
      - 'secure-app'
      - '--image=gcr.io/$PROJECT_ID/secure-app:$BUILD_ID'
      - '--region=us-central1'
      - '--allow-unauthenticated'
      - '--set-secrets=MY_API_KEY=my-api-key:latest'

images:
  - 'gcr.io/$PROJECT_ID/secure-app:$BUILD_ID'
EOF

🔍 cloudbuild.yaml Parameter & Substitution Breakdown:

Step 5: Execute the Pipeline with Cloud Build

Trigger the automated build pipeline directly from Cloud Shell:

gcloud builds submit --config=cloudbuild.yaml .

🔍 Under the Hood:

1. gcloud builds submit: Zips your local directory and streams it to a private temporary Cloud Storage staging bucket.
2. Cloud Build spins up a dedicated serverless virtual machine runner.
3. The runner executes Step 1 (Docker build), Step 2 (push to Container Registry), and Step 3 (deploy to Cloud Run with Secret Manager binding).
4. Cloud Build streams logs in real-time to your terminal and tears down the build VM ($0.00 idle cost).

Step 6: Test Your Secure Production App

1. Retrieve the Live Web URL

gcloud run services describe secure-app \
    --region=${REGION} \
    --format='value(status.url)'

2. Test the Endpoint with curl or in your Browser

curl -s $(gcloud run services describe secure-app --region=${REGION} --format='value(status.url)')

Expected JSON Output:

{
  "status": "success",
  "message": "🚀 App deployed securely via Cloud Build CI/CD!",
  "secret_injected": "YES",
  "masked_secret": "supe****2026"
}

gcloud run services add-iam-policy-binding secure-app \
    --region=${REGION} \
    --member="allUsers" \
    --role="roles/run.invoker"

curl -s -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
    $(gcloud run services describe secure-app --region=${REGION} --format='value(status.url)')

(Notice how your app successfully loaded the secret without hardcoding a single password anywhere in the source code!)

🛡️ Step 7: Credit Safety & Resource Teardown ($0.00 Guarantee)

To ensure zero ongoing charges against your $300 Free Trial credits:

1. Delete the Cloud Run Service

gcloud run services delete secure-app --region=${REGION} --quiet

2. Delete the Secret in Secret Manager

gcloud secrets delete my-api-key --quiet

3. Delete the Container Images

Because the image was tagged with a dynamic $BUILD_ID instead of :latest, delete the tagged image with this 1-line command:

gcloud container images list-tags gcr.io/${PROJECT_ID}/secure-app --format="value(tags)" | \
    xargs -I {} gcloud container images delete -q --force-delete-tags "gcr.io/${PROJECT_ID}/secure-app:{}"

🧠 Daily Practice Drill & Knowledge Check

Test your understanding of Secret Manager and CI/CD pipelines:

// Try answering these:
1. Why should API keys and database passwords NEVER be committed to a git repository, even if the repository is private?A) Git cannot store letters and numbersB) Automated scraper bots scan repositories for leaked credentials, and secrets can be baked permanently into Docker image layersC) Python crashes if git has secretsD) Git automatically deletes private repos with passwords
2. How does Google Cloud Run access secrets stored in Secret Manager securely?A) By hardcoding the password in the URLB) By mounting the secret directly into memory as an environment variable or volume file via IAM service account permissionsC) By downloading a public ZIP file from Google DriveD) By disabling firewall rules
3. What is the primary benefit of defining your build and deployment process in a cloudbuild.yaml file?A) It makes Python run 100x fasterB) It provides automated, consistent, reproducible deployments with zero manual steps, running on serverless infrastructureC) It eliminates the need for a web browserD) It replaces the Linux operating system
4. How many free build-minutes does Google Cloud Build provide every single day?A) 10 minutesB) 120 free build-minutes per day (across all projects in a billing account)C) 0 minutesD) 5,000 minutes

📋 Day 28 Cheat Sheet Summary

Tomorrow, in Day 29, we step into the cutting-edge future with Vertex AI Studio & Gemini API: learning how to build Generative AI applications and integrate foundation models into Google Cloud! 🤖🚀

← 27 - BigQuery 101 - SQL Data Warehouse | Next Topic → 29 - Vertex AI Studio and Gemini API