18 - Eventarc and Pub/Sub Integration

Welcome to Day 18 of Learn GCP in 30 Days! Yesterday, you built event-driven microservices using Cloud Functions. Today, we learn how modern cloud applications communicate with each other asynchronously without crashing: Google Cloud Pub/Sub and Eventarc.

🎯

Today's Goal Today, you will understand the difference between Synchronous and Asynchronous cloud communication in plain English, why tight coupling crashes apps during traffic spikes, how Pub/Sub uses Topics & Subscriptions as a shock absorber, how Eventarc routes 130+ Google Cloud events, and how to build your first live messaging pipeline!

🛑 The Core Problem: Coupled vs. Decoupled Systems

Before looking at Google Cloud tools, let's understand why direct connections cause apps to crash:

mermaid

🏃 Real-World Analogy: Three-Legged Race vs. Post Office Box

Real-World Analogy Mapping:

⛓️ Coupled (Three-Legged Race): Two runners tied together by the leg. If Runner B trips, Runner A immediately crashes too! When services call each other directly, one slow service freezes the entire application.
🎈 Decoupled (Post Office Mailbox): You drop a letter in a Mailbox (Pub/Sub) and walk away in 5 seconds. Your friend reads it whenever they are free. Even if your friend is asleep, your day was never stopped!

⚖️ Quick Cheat Sheet: Coupled vs. Decoupled

Feature	⛓️ Tightly Coupled (Direct Calls)	🎈 Decoupled (Pub/Sub)
How they connect	App A calls App B directly (waits on hold)	Both talk only to a neutral queue
If Service B crashes	App A crashes too (User gets an error)	App A is unaffected (Messages wait safely in queue)
During traffic spikes	Downstream server gets crushed and dies	Queue acts as a shock absorber

How Present Technology Solves It (Asynchronous Pub/Sub):

The Web App processes the checkout and drops an "Order Placed" message into Pub/Sub in 5 milliseconds.
The Web App immediately replies to the customer: "Order Confirmed!" (Done in 200ms).
Background workers (Invoices, SMS, Analytics) read from Pub/Sub and process at their own pace safely.

📬 Real-World Analogy: Phone Call vs. WhatsApp / YouTube Channel

mermaid

Synchronous Communication = A Live Phone Call:
- Both parties must be available simultaneously. If the receiver doesn't answer or drops the call, communication breaks.
Asynchronous Pub/Sub = A YouTube Channel:
- Publisher (Content Creator): Uploads 1 video to their channel (The Topic). The creator doesn't need to know who the subscribers are or phone them individually.
- Subscribers (Followers): Each subscriber gets a copy in their personal feed (The Subscription) and watches the video at their own convenient time!

🔑 Everyday Cloud Terms Explained in Plain English

1. The 4 Pillars of Google Cloud Pub/Sub

mermaid

👨‍💻 Publisher: The application that creates and sends a message (e.g., your Web App).
📨 Topic: A named channel or drop-box where messages are sent (e.g., orders-topic).
📦 Message: The actual data payload (usually a JSON string like {"order_id": 492, "amount": 99.0}).
📬 Subscription: A dedicated inbox attached to a topic. Pub/Sub stores messages in the subscription buffer until the consumer is ready to process them.

2. Pull Subscriptions vs. Push Subscriptions

How does the consumer receive the message from the subscription inbox?

Feature	📥 Pull Subscription	📤 Push Subscription
How it works	The worker asks Pub/Sub: "Do you have any new messages for me?"	Pub/Sub automatically makes an HTTP POST request to your worker's URL.
Control	Worker controls the pace (can process 10 messages at a time).	Pub/Sub pushes traffic as fast as it arrives.
Best For	Heavy batch jobs, Compute Engine VMs, long background workers.	Cloud Run & Cloud Functions (serverless HTTP webhooks).

3. Acknowledgment (ACK) & At-Least-Once Delivery

ACK (Acknowledgment): When your worker finishes processing a message, it sends an ACK signal back to Pub/Sub. Pub/Sub then deletes the message from the queue.
NACK / Timeout: If your worker crashes halfway through processing, it never sends an ACK. After a deadline (e.g., 10 seconds), Pub/Sub automatically redelivers the message to another healthy worker so no data is ever lost!

4. What is Google Cloud Eventarc?

While Pub/Sub is a general-purpose message queue, Eventarc is Google Cloud’s universal event routing hub:

mermaid

Eventarc connects 130+ Google Cloud services directly to Cloud Run and Cloud Functions.
It normalizes all cloud events into a standardized open format called CloudEvents.
You don't write custom plumbing code—you just configure an Eventarc Trigger ("When a PDF is added to bucket X, invoke Cloud Run Y").

🧪 Hands-on Lab Activity: Build a Pub/Sub Messaging Pipeline

In this hands-on lab, we will:

Enable the Cloud Pub/Sub API.
Create a Pub/Sub Topic (order-events-topic).
Create a Pull Subscription (order-processor-sub).
Publish test messages into the Topic.
Pull and acknowledge (ACK) messages from the Subscription!

Step 1: Enable the Cloud Pub/Sub API

Option A: Web Console UI

Open console.cloud.google.com.
Press /, type Cloud Pub/Sub API, and click on it.
Click Enable.

Option B: Cloud Shell CLI

bash

gcloud services enable pubsub.googleapis.com eventarc.googleapis.com

Step 2: Create a Pub/Sub Topic

Option A: Web Console UI (Click-by-Click)

In the GCP Console, press / $\rightarrow$ type Pub/Sub $\rightarrow$ select Topics.
Click Create Topic at the top.
Set Topic ID: order-events-topic.
Leave other default settings checked $\rightarrow$ Click Create.

Option B: Cloud Shell CLI

bash

gcloud pubsub topics create order-events-topic

Anatomy Breakdown: What does gcloud pubsub topics create do?

Command Part	What It Does In Plain English
`gcloud pubsub topics create`	Instructs GCP to allocate a new message broadcast channel.
`order-events-topic`	The unique name of your topic within your GCP project.

Step 3: Create a Subscription Attached to the Topic

Now, create a subscriber queue that listens to order-events-topic.

Option A: Web Console UI (Click-by-Click)

Go to Pub/Sub $\rightarrow$ Click Subscriptions in the left menu.
Click Create Subscription at the top.
Configure the subscription:
- Subscription ID: order-processor-sub
- Select a Cloud Pub/Sub topic: Choose order-events-topic
- Delivery type: Pull
- Message retention duration: 7 days (default)
- Acknowledgment deadline: 10 seconds (default)
Click Create.

Option B: Cloud Shell CLI

bash

gcloud pubsub subscriptions create order-processor-sub \
    --topic=order-events-topic \
    --ack-deadline=10

Anatomy Breakdown: What does gcloud pubsub subscriptions create do?

Parameter	What It Does In Plain English
`order-processor-sub`	The name of the dedicated inbox/queue that holds messages for a worker.
`--topic=order-events-topic`	Tells Pub/Sub to link this subscription inbox directly to our topic.
`--ack-deadline=10`	Gives the worker 10 seconds to process and acknowledge the message before Pub/Sub assumes it crashed and redelivers it.

Step 4: Publish Test Messages into the Topic

Let's act as the publisher and broadcast a JSON order message into order-events-topic:

Option A: Web Console UI

Go to Pub/Sub $\rightarrow$ Topics $\rightarrow$ Click on order-events-topic.
Click the Messages tab $\rightarrow$ Click Publish Message.

In the Message body, paste:

json

{"order_id": "ORD-101", "item": "Wireless Headphones", "amount": 79.99}

Click Publish.

Option B: Cloud Shell CLI

bash

gcloud pubsub topics publish order-events-topic \
    --message='{"order_id": "ORD-101", "item": "Wireless Headphones", "amount": 79.99}'

Step 5: Pull and Acknowledge (ACK) the Message

Now, let's act as the downstream worker service reading from the subscription inbox:

Option A: Web Console UI

Go to Pub/Sub $\rightarrow$ Subscriptions $\rightarrow$ Click on order-processor-sub.
Click the Messages tab $\rightarrow$ Click Pull.
The message payload {"order_id": "ORD-101", ...} appears on your screen!

Option B: Cloud Shell CLI

Run this command to pull the message and automatically send an ACK:

bash

gcloud pubsub subscriptions pull order-processor-sub --auto-ack --limit=1

Output:

text

DATA: {"order_id": "ORD-101", "item": "Wireless Headphones", "amount": 79.99}
MESSAGE_ID: 1029384756
ORDER: ACKNOWLEDGED (Message safely processed and removed from queue)

💡

What happened? Pub/Sub delivered the message to your terminal and because we passed --auto-ack, Pub/Sub marked the message as completed and removed it from the queue!

🧹 Step 6: Clean Up All Resources (Credit Safety Guarantee)

To ensure your GCP trial balance remains at $0.00, delete the subscription and topic:

Option A: Web Console UI

Go to Pub/Sub $\rightarrow$ Subscriptions $\rightarrow$ Select order-processor-sub $\rightarrow$ Click Delete.
Go to Pub/Sub $\rightarrow$ Topics $\rightarrow$ Select order-events-topic $\rightarrow$ Click Delete.

Option B: Cloud Shell CLI

bash

# 1. Delete Subscription
gcloud pubsub subscriptions delete order-processor-sub --quiet

# 2. Delete Topic
gcloud pubsub topics delete order-events-topic --quiet

Signal vs. Noise: Key Concepts & Noise Filter

🧠

Good to Know (Key Concepts)

Pub/Sub: Fully managed, globally scalable asynchronous messaging service for decoupling systems.
Topic vs. Subscription: A topic receives broadcasts; subscriptions store copies for specific listeners.
Pull vs. Push: Pull is worker-initiated (great for batch jobs); Push is Pub/Sub-initiated HTTP POST (great for Cloud Run/Functions).
Eventarc: Unified event router that maps GCP cloud events (GCS, Audit Logs, BigQuery) to serverless targets.
At-Least-Once Delivery: Messages are guaranteed to be delivered at least once, meaning your consumer code should be idempotent (safe against rare duplicates).

ℹ️

Noise Filter (Don't Memorize)

Do NOT memorize advanced schema validation protocols (Avro / Protobuf schemas) when starting out; standard JSON message payloads work for 95% of use cases.
Do NOT worry about partition ordering keys unless your app strictly requires exact sequential FIFO message processing.

Common Doubts & Interview Traps

Q1: What is the difference between Google Cloud Pub/Sub and Apache Kafka / RabbitMQ?

Answer: RabbitMQ and Kafka often require managing server clusters, configuring brokers, managing partition storage, and paying for idle VMs 24/7. Google Cloud Pub/Sub is 100% serverless—it automatically scales from 1 message per day to millions of messages per second with zero server management and zero cluster provisioning!

Q2: What happens if a consumer worker crashes while processing a Pub/Sub message?

Answer: Because the worker crashed, it never sends an ACK (Acknowledgment) back to Pub/Sub. When the acknowledgment deadline timer expires, Pub/Sub automatically redelivers the message to another healthy worker.

Q3: What is a "Dead Letter Topic"?

Answer: If a "poison" message has corrupted data that causes consumer code to crash repeatedly (e.g., 5 times in a row), Pub/Sub can automatically move that problematic message to a Dead Letter Topic so it doesn't block the rest of the queue, allowing engineers to inspect the bug safely.

Daily Practice Drill & Self-Check

Test your understanding of today's lesson:

text

// Try answering these:
1. In Pub/Sub, where does a publisher send messages: directly to a Subscription or to a Topic?
2. Which delivery model does Google Cloud Pub/Sub use when it automatically sends an HTTP POST request to a Cloud Run URL: Pull or Push?
3. What is the name of Google Cloud's universal event router that connects 130+ GCP services to Cloud Run using CloudEvents format?

💡 Click for Solutions

To a Topic!
Push Subscription!
Google Cloud Eventarc!

🎉 Fantastic progress! You have completed Day 18!

You have mastered Asynchronous Event-Driven Architectures, Google Cloud Pub/Sub topics & subscriptions, and Eventarc event routing.

Tomorrow on Day 19, we will explore the world of enterprise container orchestration: GKE Kubernetes Overview for Beginners!

← 17 - Cloud Functions - Event-Driven Code | Next Topic → 19 - GKE Kubernetes Overview for Beginners