23 - NoSQL DBs - Firestore for Realtime Apps

Welcome to Day 23 of Learn GCP in 30 Days! Yesterday, you mastered relational SQL databases with Google Cloud SQL. Today, we step into the flexible, blazing-fast world of NoSQL Document Databases with Google Cloud Firestore.

🛑 The Core Problem: When Rigid SQL Tables Slow Down Modern Apps

In traditional relational databases (like PostgreSQL or MySQL), every row in a table must strictly conform to a predefined schema.

What Existed Previously:

Developers built apps by defining strict SQL column structures. Whenever an application added a new feature (like adding social login handles or user preferences), engineers had to write risky database migrations (ALTER TABLE) and coordinate downtime.

Furthermore, building real-time applications (like chat apps, live scoreboards, or ride-sharing trackers) required building complex WebSocket servers and continuous database polling.

Problems Faced:

• 🧱 Rigid Schemas: Every record in a SQL table must have the exact same columns. Storing semi-structured or varying user profiles requires numerous nullable columns or messy JSON string fields.
• 🔄 Polling Overhead: If 50,000 users open a chat app, each device polling the SQL database every 2 seconds creates 25,000 queries/second of idle traffic.
• 📶 Zero Offline Support: If a mobile phone loses network coverage in a tunnel, standard SQL client libraries immediately throw timeout exceptions and crash the user interface.
• ⚖️ Scaling Bottlenecks: Relational databases scale vertically (bigger VMs). Handling millions of concurrent mobile connections requires complex connection pooling and sharding.

How Present Technology Solves It:

Google created Cloud Firestore (Serverless NoSQL Document Database):

1. Flexible Document Model: Data is stored as JSON-like documents inside collections. Every document can have its own unique fields without database migrations.
2. Realtime Push Listeners: Instead of clients repeatedly polling the server, Firestore uses persistent bi-directional streams. When a document changes in the cloud, Google pushes the update to all connected web and mobile devices in milliseconds.
3. Automatic Offline Caching: The Firestore client SDK automatically caches data locally on iOS, Android, and Web. Users can write data while offline on an airplane, and Firestore seamlessly synchronizes the data when connectivity is restored!
4. Massive Serverless Scalability: Firestore scales automatically to millions of concurrent connections with zero server provisioning and guarantees multi-region 99.999% availability.

🗄️ Real-World Analogy: Strict Spreadsheet vs. Dynamic Filing Folders

• Relational SQL Database = Strict Excel Spreadsheet:
  • If Column D is phone_number (Integer), every row must follow that rule. You cannot insert a row with an extra address object without adding new columns to the entire sheet.
• Cloud Firestore NoSQL = Filing Cabinet with Flexible Index Cards:
  • The filing drawer is a Collection (e.g., customers).
  • Inside the drawer, each index card is an independent Document (e.g., cust_101, cust_102).
  • Card 1 can have name and email.
  • Card 2 can have name, email, preferred_currency, and an array of shipping_addresses. No schema migrations needed!

🧩 Firestore Data Model: Collections, Documents & Subcollections

Firestore organizes data in a hierarchical tree:

1. Collections (Folders)

• A Collection is a container for documents.
• Collections cannot hold raw data directly (no raw strings or numbers); they contain only Documents.
• Collections are lightweight and created automatically as soon as you add the first document.

2. Documents (JSON-like Objects)

• A Document is a record that contains a set of key-value pairs (called Fields).
• Maximum document size is 1 Megabyte (plenty of room for rich user profiles or metadata).
• Each document has a unique identifier (e.g., auto-generated string like al9xK90vQ or custom ID like user_101).

3. Subcollections (Nested Hierarchies)

• Documents can contain nested Subcollections.
• For example, /users/user_881/orders/order_9001 allows you to store an order history that naturally belongs strictly to user_881.

4. Supported Data Types

Firestore supports rich, native data types:

• String, Number (integer or floating point), Boolean
• Timestamp (microsecond-precision cloud time)
• Geopoint (latitude and longitude coordinates)
• Map (nested JSON-like key-value objects)
• Array (lists of values or maps)
• Reference (pointers to other documents in the database)
• Null

⚡ 4 Architectural Superpowers of Firestore

1. Realtime Listeners (onSnapshot)

Instead of making one-off HTTP requests, your client code can attach a listener to a document or query. Whenever data changes in the cloud, your UI callback executes instantly.

2. Shallow Queries (Cost & Bandwidth Optimizer)

Queries in Firestore are shallow:

• If you query the /users collection to list all users, Firestore downloads only the user document fields.
• It does NOT download their nested /orders or /messages subcollections. This prevents massive bandwidth consumption.

3. Automatic Indexing

• Firestore automatically indexes every single field in every document.
• In traditional SQL, query performance slows down as your database grows from 1,000 to 10,000,000 rows unless you manually create indexes.
• In Firestore, query time is proportional to the size of the result set, NOT the size of your database! (Fetching 10 rows from a collection of 100,000,000 documents takes the exact same 15 milliseconds as fetching 10 rows from a collection of 10 documents).

4. ACID Multi-Document Transactions

Firestore is not an "eventually consistent" weak NoSQL database. It supports true ACID multi-document transactions and batch writes. If an operation updating 5 different documents encounters an error, all changes are atomically rolled back.

⚖️ Cloud SQL (Relational) vs. Cloud Firestore (NoSQL)

🎁 The Firestore Perpetual Free Tier ($0 Forever)

One of the biggest advantages of Firestore is its Generous Perpetual Free Tier (part of Google Cloud's Free Tier that never expires, even after your $300 trial ends!):

• 💽 Storage: 1 GB of stored data free forever.
• 📖 Document Reads: 50,000 reads per day free.
• ✍️ Document Writes: 20,000 writes per day free.
• 🗑️ Document Deletes: 20,000 deletes per day free.
• 🌐 Egress Traffic: 10 GB / month free.

🧪 Hands-on Lab: Provisioning Firestore & Python CRUD Operations

In this lab, you will initialize a Cloud Firestore database in Native Mode, create collections and documents via the GCP Console, and run a Python script in Cloud Shell to query and listen to data.

Step 1: Initialize Cloud Firestore in Native Mode

Pathway A: Web Console (Click-by-Click)

1. Open the Google Cloud Console (https://console.cloud.google.com/).
2. In the top search bar, type Firestore and select Firestore Studio (or navigate to Navigation Menu $\rightarrow$ Databases $\rightarrow$ Firestore).
3. If prompted to select a database mode, choose Firestore Native mode (Recommended for modern apps).
4. Configure Database:
   • Database ID: (default) (leave as default).
   • Location type: Select Region.
   • Region: Choose us-central1 (Iowa) (or your closest region).
5. Click CREATE DATABASE. (Creation takes ~15 seconds).

Pathway B: Cloud Shell CLI

Open Google Cloud Shell and run:

# 1. Set environment variables
export PROJECT_ID=$(gcloud config get-value project)
export REGION="us-central1"

# 2. Create the default Firestore database in Native mode
gcloud firestore databases create \
  --location=${REGION} \
  --type=firestore-native

Step 2: Create a Collection and Document in the Web Console

Let's see how intuitive document creation is in the Cloud Console:

1. In Firestore Studio, click + START COLLECTION.
2. Collection ID: products. Click Next.
3. Document ID: Leave as Auto-ID (or enter prod_laptop_01).
4. Add fields to this document:
   • Field: name | Type: string | Value: MacBook Pro 16
   • Click + Add field
   • Field: price | Type: number | Value: 2499
   • Click + Add field
   • Field: in_stock | Type: boolean | Value: true
   • Click + Add field
   • Field: tags | Type: array | Add values: "electronics", "computers"
5. Click SAVE.

You now have your first NoSQL collection and document live in the cloud!

Step 3: Write a Python Script to Insert & Query Documents

Let's interact with Firestore programmatically using the official Google Cloud Python client library.

In Cloud Shell:

# 1. Install the official Google Cloud Firestore library
pip install google-cloud-firestore --quiet

# 2. Create the Python script
cat << 'EOF' > firestore_demo.py
import datetime
from google.cloud import firestore

# Initialize Firestore Client (auto-authenticates with Cloud Shell credentials)
db = firestore.Client()

print("🚀 1. Adding documents to 'customers' collection...")

# Add Document 1 with custom ID
db.collection("customers").document("cust_101").set({
    "name": "Manikanta Kumar",
    "email": "mani@example.com",
    "membership": "premium",
    "points": 450,
    "created_at": datetime.datetime.now(datetime.timezone.utc)
})

# Add Document 2 with auto-generated ID
doc_ref = db.collection("customers").add({
    "name": "Sarah Connor",
    "email": "sarah@skynet.com",
    "membership": "basic",
    "points": 120,
    "created_at": datetime.datetime.now(datetime.timezone.utc)
})

# Add Document 3
db.collection("customers").document("cust_103").set({
    "name": "Bruce Wayne",
    "email": "bruce@wayne-enterprises.com",
    "membership": "premium",
    "points": 9800,
    "created_at": datetime.datetime.now(datetime.timezone.utc)
})

print("✅ Documents successfully written!\n")

print("🔍 2. Fetching all 'premium' customers...")
query = db.collection("customers").where(filter=firestore.FieldFilter("membership", "==", "premium"))
results = query.stream()

for doc in results:
    data = doc.to_dict()
    print(f"  👉 ID: {doc.id} | Name: {data['name']} | Points: {data['points']}")

print("\n✍️ 3. Updating points for cust_101 (+50 points)...")
db.collection("customers").document("cust_101").update({
    "points": firestore.Increment(50)
})

updated_doc = db.collection("customers").document("cust_101").get()
print(f"  🎉 Updated cust_101 Points: {updated_doc.to_dict()['points']}")
EOF

# 3. Run the script
python3 firestore_demo.py

Expected Output:

🚀 1. Adding documents to 'customers' collection...
✅ Documents successfully written!

🔍 2. Fetching all 'premium' customers...
  👉 ID: cust_101 | Name: Manikanta Kumar | Points: 450
  👉 ID: cust_103 | Name: Bruce Wayne | Points: 9800

✍️ 3. Updating points for cust_101 (+50 points)...
  🎉 Updated cust_101 Points: 500

Step 4: Verify in Firestore Studio

1. Return to the Google Cloud Console $\rightarrow$ Firestore Studio.
2. Notice that the new customers collection appeared automatically!
3. Click on cust_101 to view its fields, types, and the atomically incremented points: 500.

🛡️ Step 5: Credit Safety & Resource Teardown

Because Firestore operates on a Serverless Free Tier, storing a few test documents costs $0.00. However, to keep your project completely clean:

Pathway A: Web Console (Click-by-Click)

1. In Firestore Studio, click on the customers collection.
2. Click the three dots ⋮ at the top right of the collection column $\rightarrow$ select Delete collection.
3. Repeat for the products collection.

Pathway B: Cloud Shell CLI

# Delete all documents in collections recursively
gcloud firestore operations list
gcloud firestore databases delete --database='(default)' --quiet

# Clean up local Python demo file
rm -f firestore_demo.py

📝 Day 23 Summary & Quick Reference

Core Architecture Takeaways

1. Document NoSQL Model: Firestore stores semi-structured data as JSON-like documents grouped into collections with zero rigid schema constraints.
2. Realtime Push Sync: Built-in listeners push database changes to web/mobile devices in milliseconds without custom WebSocket servers.
3. Offline First: Automatic local client caching enables uninterrupted mobile app usage even in airplane mode.
4. Shallow Queries: Querying a collection returns document data without downloading heavy nested subcollections.
5. Perpetual Free Tier: 1 GB storage and 50,000 reads/day free forever with zero idle VM costs.

Quick Command Cheat Sheet

Tomorrow, in Day 24, we enter the world of ultra-fast in-memory caching with Google Cloud Memorystore (Redis)—learning how to achieve sub-millisecond query responses and reduce database loads by 90%!

← 22 - Relational DBs - Cloud SQL (Postgres and MySQL) | Next Topic → 24 - Caching - MemoryStore Redis