Associate-Cloud-Engineer Google Cloud Certified - Associate Cloud Engineer Question and Answers

As part of your workload, there might be certain VM instances that are critical to running your application or services, such as an instance running a SQL server, a server used as a license manager, and so on. These VM instances might need to stay running indefinitely so you need a way to protect these VMs from being deleted. By setting the deletionProtection flag, a VM instance can be protected from accidental deletion. If a user attempts to delete a VM instance for which you have set the deletionProtection flag, the request fails. Only a user that has been granted a role with compute.instances.create permission can reset the flag to allow the resource to be deleted.

Ref: https://cloud.google.com/compute/docs/instances/preventing-accidental-vm-deletion

Install the workload in a compute engine VM, start and stop the instance as needed, because as per the question the VM runs for 30 hours, process can be performed offline and should not be interrupted, if interrupted we need to restart the batch process again. Preemptible VMs are cheaper, but they will not be available beyond 24hrs, and if the process gets interrupted the preemptible VM will restart.

Logged information Within Cloud Audit Logs, there are two types of logs: Admin Activity logs: Entries for operations that modify the configuration or metadata of a project, bucket,or object. Data Access logs: Entries for operations that modify objects or read a project, bucket, or object. There are several sub-types of data access logs: ADMIN_READ: Entries for operations that read the configuration or metadata of a project, bucket, or object. DATA_READ: Entries for operations that read an object. DATA_WRITE: Entries for operations that create or modify an object.https://cloud.google.com/storage/docs/audit-logs#types

https://cloud.google.com/bigquery/docs/custom-quotashttps://cloud.google.com/bigquery/pricing#flat_rate_pricing

Using primitive roles The following table lists the primitive roles that you can grant to access a project, the description of what the role does, and the permissions bundled within that role. Avoid using primitive roles except when absolutely necessary. These roles are very powerful, and include a large number of permissions across all Google Cloud services. For more details on when you should use primitive roles, see the Identity and Access Management FAQ. IAMpredefined roles are much more granular, and allow you to carefully manage the set of permissions that your users have access to. See Understanding Roles for a list of roles that can be granted at the project level. Creating custom roles can further increase the control you have over user permissions.https://cloud.google.com/resource-manager/docs/access-control-proj#using_primitive_roles

https://cloud.google.com/iam/docs/understanding-custom-roles

Cloud Storage allows world-wide storage and retrieval of any amount of data at any time. We dont need to set up auto-scaling ourselves. Cloud Storage autoscaling is managed by GCP. Cloud Storage is an object store so it is suitable for storing photos. Cloud Storage allows world-wide storage and retrieval so cater well to our worldwide audience. Cloud storage provides us lifecycle rules that can be configured to automatically delete objects older than 30 days. This also fits our requirements. Finally, Google Cloud Storage offers several storage classes such as Nearline Storage ($0.01 per GB per Month) Coldline Storage ($0.007 per GB per Month) and Archive Storage ($0.004 per GB per month) which are significantly cheaper than any of the options above.

Ref: https://cloud.google.com/storage/docs

Ref: https://cloud.google.com/storage/pricing

Standard persistent disks are efficient and economical for handling sequential read/write operations, but they aren't optimized to handle high rates of random input/output operations per second (IOPS). If your apps require high rates of random IOPS, use SSD persistent disks. SSD persistent disks are designed for single-digit millisecond latencies. Observed latency is application specific.

https://cloud.google.com/bigquery/docs/quickstarts/quickstart-web-ui?authuser=4

Adding an API as a type provider

This page describes how to add an API to Google Cloud Deployment Manager as a type provider. To learn more about types and type providers, read the Types overview documentation.

A type provider exposes all of the resources of a third-party API to Deployment Manager as base types that you can use in your configurations. These types must be directly served by a RESTful API that supports Create, Read, Update, and Delete (CRUD).

If you want to use an API that is not automatically provided by Google with Deployment Manager, you must add the API as a type provider.

https://cloud.google.com/deployment-manager/docs/configuration/type-providers/creating-type-provider

Folders are nodes in the Cloud Platform Resource Hierarchy. A folder can contain projects, other folders, or a combination of both. Organizations can use folders to group projects under the organization node in a hierarchy. For example, your organization might contain multiple departments, each with its own set of Google Cloud resources. Folders allow you to group these resources on a per-department basis. Folders are used to group resources that share common IAM policies. While a folder can contain multiple folders or resources, a given folder or resource can have exactly one parent.https://cloud.google.com/resource-manager/docs/creating-managing-folders

Keywords, Financial data (large data) used globally, data stored and queried using relational structure (SQL), clients should get exact identical copies(Strong Consistency), Multiple region, low latency to end user, select storage option to minimize latency.

Configure the ACLs on each image in Cloud Storage to give read-only access to the default Compute Engine service account. is not right.

As mentioned above, Container Registry ignores permissions set on individual objects within the storage bucket so this isnt going to work.

Ref: https://cloud.google.com/container-registry/docs/access-control

The goal is to enable teams to deploy infrastructure independently while ensuring compliance with company practices, without requiring teams to understand the underlying details of those practices.

Option A provides deployment capability but doesn't enforce practices. The Editor role is overly broad, and using the gcloud CLI directly requires knowledge of how to configure resources compliantly.

Option B requires teams to learn all the practices, contradicting the requirement that they don't need to know the implementation details.

Option C (Organization Policies) is useful for setting constraints (e.g., disallowing public IPs, restricting regions), but it doesn't provide pre-configured, deployable components that embody best practices. Teams still need to figure out how to build compliant resources within the policy constraints.

Option D (Terraform Modules): This approach encapsulates the company's required practices within reusable infrastructure-as-code modules. Engineering teams can then use these modules as building blocks, providing only the necessary input parameters (like application name orsize). The module handles the compliant implementation details internally. This allows teams to deploy independently and ensures compliance without needing deep knowledge of every practice.

Using standardized, compliant modules is a common pattern for enabling self-service infrastructure deployment while maintaining standards and governance.

In Google compute engine, if predefined machine types don’t meet your needs, you can create an instance with custom virtualized hardware settings. Specifically, you can create an instance with a custom number of vCPUs and custom memory, effectively using a custom machine type. Custom machine types are ideal for the following scenarios:

   1. Workloads that aren’t a good fit for the predefined machine types that are available to you.

   2. Workloads that require more processing power or more memory but don’t need all of the upgrades that are provided by the next machine type level.

In our scenario, we only need a memory upgrade. Moving to a bigger instance would also bump up the CPU which we don’t need so we have to use a custom machine type. It is not possible to change memory while the instance is running so you need to first stop the instance, change the memory and then start it again. See below a screenshot that shows how CPU/Memory can be customized for an instance that has been stopped.

Ref: https://cloud.google.com/compute/docs/instances/creating-instance-with-custom-machine-type

Billing Administrators can not create a new billing account, and the project is presumably already created. Project Billing Manager allows you to link the created billing account to the project. It is vague on how the billing account gets created but by process of elimination

"...we recommend that you enable Cloud Billing data export to BigQuery at the same time that you create a Cloud Billing account. "https://cloud.google.com/billing/docs/how-to/export-data-bigquery

https://medium.com/google-cloud/analyzing-google-cloud-billing-data-with-big-query-30bae1c2aae4

As mentioned in Schema and data model, you should be careful when choosing a primary key to not accidentally create hotspots in your database. One cause of hotspots is having a column whose value monotonically increases as the first key part, because this results in all inserts occurring at the end of your key space. This pattern is undesirable because Cloud Spanner divides data among servers by key ranges, which means all your inserts will be directed at a single server that will end up doing all the work.https://cloud.google.com/spanner/docs/schema-design#primary-key-prevent-hotspots

https://cloud.google.com/compute/docs/troubleshooting/troubleshooting-migs

https://cloud.google.com/compute/docs/instance-templates#how_to_update_instance_templates

https://support.google.com/cloudidentity/answer/6262987?hl=en &ref_topic=7558767

Set budgets and budget alerts Overview Avoid surprises on your bill by creating Cloud Billing budgets to monitor all of your Google Cloud charges in one place. A budget enables you to track your actual Google Cloud spend against your planned spend. After you've set a budget amount, you set budget alert threshold rules that are used to trigger email notifications. Budget alert emails help you stay informed about how your spend is tracking against your budget. 2. Set budget scope Set the budget Scope and then click Next. In the Projects field, select one or more projects that you want to apply the budget alert to. To apply the budget alert to all the projects in the Cloud Billing account, choose Select all.https://cloud.google.com/billing/docs/how-to/budgets#budget-scop

The requirement is for a stable set of egress IP addresses from a GKE cluster for allowlisting by a third party, following best practices.

Option A is not recommended: Using a single node lacks scalability and high availability. Relying on a single node's static IP creates a single point of failure and doesn't align with GKE's design principles. Disabling autoscaling hinders elasticity.

Option C is complex and unreliable: Public nodes typically have ephemeral external IPs (unless manually configured per node, which is difficult to manage with autoscaling). Dynamically tracking and emailing IPs daily is operationally burdensome and prone to race conditions where the allowlist might lag behind IP changes.

Option D uses Cloud NAT but with dynamic IPs. Dynamic IPs change over time, making them unsuitable for stable firewall allowlists.

Option B is the Google-recommended practice: Configuring the GKE cluster with private nodes enhances security as nodes don't have direct external IPs. Cloud NAT provides managed network address translation for these private nodes to access the internet. By configuring Cloud NAT with a static allocation of external IP addresses, all egress traffic from the private GKE nodes will appear to originate from this stable, predictable set of IPs. This set can be given to the vendor for allowlisting without worrying about node IP changes due to scaling or maintenance.

This approach decouples the application's egress IP from the individual nodes, providing stability and adhering to the principle of least privilege (private nodes).

The requirement is for an infrastructure-as-code (IaC) solution that can manage Kubernetes resources and other cloud resources across multiple cloud providers (Google Cloud and two others).

Option A (YAML manifests): YAML manifests are primarily used for defining Kubernetes objects, not for provisioning general cloud resources (like VPCs, IAM policies, databases) across different cloud providers.

Option C (Python + SDKs): While possible, writing custom scripts using each provider's SDK requires significant development effort to handle state management, dependencies, and provider differences. It essentially reinvents much of what dedicated IaC tools provide and is not a standard IaC approach.

Option D (Config Connector): Config Connector allows managing Google Cloud resources using Kubernetes-style manifests and APIs. It is specific to Google Cloud and cannot manage resources in other cloud providers.

Option B (Terraform): Terraform is an open-source IaC tool explicitly designed for building, changing, and versioning infrastructure safely and efficiently across multiple cloud providers and on-premises data centers. It uses providers for different platforms (GCP, AWS, Azure, Kubernetes, etc.), allowing a unified workflow to manage diverse resources across the required environments (Google Cloud, other clouds, Kubernetes).

Terraform is the standard tool for multi-cloud IaC automation as described in the scenario.

https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset

https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset#usage_patterns

DaemonSets attempt to adhere to a one-Pod-per-node model, either across the entire cluster or a subset of nodes. As you add nodes to a node pool, DaemonSets automatically add Pods to the new nodes as needed.

In GKE, DaemonSets manage groups of replicated Pods and adhere to a one-Pod-per-node model, either across the entire cluster or a subset of nodes. As you add nodes to a node pool, DaemonSets automatically add Pods to the new nodes as needed. So, this is a perfect fit for our monitoring pod.

Ref: https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset

DaemonSets are useful for deploying ongoing background tasks that you need to run on all or certain nodes, and which do not require user intervention. Examples of such tasks include storage daemons like ceph, log collection daemons like fluentd, and node monitoring daemons like collectd. For example, you could have DaemonSets for each type of daemon run on all of your nodes. Alternatively, you could run multiple DaemonSets for a single type of daemon, but have them use different configurations for different hardware types and resource needs.

This command correctly lists pods that have the label app=prod. When creating the deployment, we used the label app=prod so listing pods that have this label retrieve the pods belonging to nginx deployments. You can list pods by using Kubernetes CLI  kubectl get pods.

Ref: https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/

Ref: https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/#list-containers-filtering-by-pod-label

Comprehensive and Detailed In Depth Explanation:

Let's evaluate each database option against the requirements: automatic scaling, multi-region support, and low latency for a growing social media platform:

A. Bigtable: Bigtable is a highly scalable NoSQL database designed for large analytical and operational workloads with low latency. It offers excellent horizontal scalability and can be deployed across multiple regions for high availability and lower latency for a global user base. However, it's a NoSQL database and might require significant changes to your existing PostgreSQL data model and application code.

B. BigQuery: BigQuery is a fully managed, serverless data warehouse optimized for analytical queries on large datasets. It's not designed for low-latency transactional workloads that a social media platformwould require for real-time user interactions. While it's globally available, its primary use case is not operational database needs.

C. Spanner: Spanner is a globally distributed, horizontally scalable relational database service with strong consistency. It offers automatic scaling, built-in multi-region and multi-continental configurations for high availability and low latency across a global user base, and supports standard SQL (with some extensions). This makes it a strong candidate for a rapidly growing platform needing scalability, global presence, and low latency. While it's not directly PostgreSQL, it offers a relational model and tools to aid migration.

D. Cloud SQL for PostgreSQL: Cloud SQL offers managed PostgreSQL instances with automatic scaling capabilities. It supports high availability within a region and cross-region read replicas for disaster recovery and read scaling. However, its multi-region capabilities for write operations and automatic scaling across regions are more limited compared to Spanner. For a rapidly growing platform with a primarily North American user base but future global expansion in mind and a need for low latency, Spanner's architecture is better suited for true multi-region write capabilities and consistent low latency globally.

Considering the requirements for automatic scaling, multi-region support for both reads and writes with low latency for a growing user base, Spanner is the most appropriate choice.

Google Cloud Documentation References:

Cloud Spanner Overview: https://cloud.google.com/spanner/docs/overview - This document highlights Spanner's global scalability, strong consistency, and multi-region capabilities.

Cloud Bigtable Overview: https://cloud.google.com/bigtable/docs/overview - While scalable and low-latency, it's a NoSQL database, which might require significant application changes.

BigQuery Overview: https://cloud.google.com/bigquery/docs/introduction - Focuses on analytics, not low-latency transactional workloads.

Cloud SQL for PostgreSQL Overview: https://cloud.google.com/sql/docs/postgres/overview - While it offers scaling and regional HA, its multi-region write capabilities are not as robust as Spanner's.

===========

Google Cloud provides Cloud Audit Logs, which is an integral part of Cloud Logging. It consists of two log streams for each project: Admin Activity and Data Access, which are generated by Google Cloud services to help you answer the question of who did what, where, and when? within your Google Cloud projects.

Ref: https://cloud.google.com/iam/docs/job-functions/auditing#scenario_external_auditors

This command correctly deletes the deployment. Pods are managed by kubernetes workloads (deployments). When a pod is deleted, the deployment detects the pod is unavailable and brings up another pod to maintain the replica count. The only way to delete the workload is by deleting the deployment itself using the kubectl delete deployment command.

$ kubectldeletedeployment nginx

deployment.apps nginx deleted

Ref: https://kubernetes.io/docs/reference/kubectl/cheatsheet/#deleting-resources

You can create two configurations – one for the development project and another for the production project. And you do that by running “gcloud config configurations create” command.

https://cloud.google.com/sdk/gcloud/reference/config/configurations/create

In your custom script, you can load these configurations one at a time and execute gcloud compute instances list to list Google Compute Engine instances in the project that is active in the gcloud configuration.

Ref: https://cloud.google.com/sdk/gcloud/reference/compute/instances/list

Once you have this information, you can export it in a suitable format to a suitable target e.g. export as CSV or export to Cloud Storage/BigQuery/SQL, etc

For a web application (typically using HTTP/HTTPS), an Application Load Balancer is the recommended choice as it operates at Layer 7, providing features like content-based routing,SSL termination, and improved security. To expose the application publicly, you would need to use a public DNS zone. An A record in a public DNS zone maps a domain name to the public IP address of the Application Load Balancer. Using a CNAME record would also work but is generally recommended for aliasing one domain name to another, not directly to an IP address.

Option A & B: Network Load Balancers operate at Layer 4 (TCP/UDP) and lack the application-level features of an Application Load Balancer. Private DNS zones are for internal name resolution within your VPC, not for public access.

Option C: While an Application Load Balancer is the correct type, using a private DNS zone wouldn't make the web application publicly accessible.

Reference to Google Cloud Certified - Associate Cloud Engineer Documents:

The best practices for load balancing web applications on Google Cloud, including the use of Application Load Balancers for Layer 7 traffic and the configuration of public DNS records (A or CNAME) for public access, are detailed in the Google Cloud Load Balancing and Cloud DNS documentation, both important for the Associate Cloud Engineer certification.

C:\GCP\appeng>gcloud config list

[core]

account = xxx@gmail.com

disable_usage_reporting = False

project = my-first-demo-xxxx

https://cloud.google.com/endpoints/docs/openapi/troubleshoot-gce-deployment

https://gtseres.medium.com/using-service-accounts-across-projects-in-gcp-cf9473fef8f0

You create the service account in proj-sa and take note of the service account email, then you go to proj-vm in IAM > ADD and add the service account's email as new member and give it the Compute Storage Admin role.

https://cloud.google.com/compute/docs/access/iam#compute.storageAdmin

Creating scheduled snapshots for persistent disk This document describes how to create a snapshot schedule to regularly and automatically back up your zonal and regional persistent disks. Use snapshot schedules as a best practice to back up your Compute Engine workloads. After creating a snapshot schedule, you can apply it to one or more persistent disks.https://cloud.google.com/compute/docs/disks/scheduled-snapshots

https://medium.com/google-cloud/how-to-deploy-cassandra-and-connect-on-google-cloud-platform-with-a-few-clicks-11ee3d7001d1

https://cloud.google.com/blog/products/databases/open-source-cassandra-now-managed-on-google-cloud

https://cloud.google.com/marketplace

You can deploy Cassandra as a Service, called Astra, on the Google Cloud Marketplace. Not only do you get a unified bill for all GCP services, your Developers can now create Cassandra clusters on Google Cloud in minutes and build applications with Cassandra as a database as a service without the operational overhead of managing Cassandra

The requirement is for private communication between a Cloud Run service and a GKE API, following best practices.

Option A exposes the GKE API to the public internet, which violates the "privately reach" requirement. Relying on dynamic IP allowlisting with Cloud Armor is complex and less secure than private networking.

Options B and C configure overly permissive firewall rules (allowing all egress or ingress) and do not establish the necessary private network path between Cloud Run (which normally runs outside your VPC) and the GKE cluster within your VPC.

Option D describes the standard Google-recommended pattern for this scenario:

Internal Application Load Balancer (ILB): Expose the GKE service (API) using an ILB. This gives the service a private IP address accessible only within the VPC network (or connected networks).

Cloud DNS: Create a private DNS zone and record pointing a fully qualified domain name (FQDN) to the ILB's private IP address. This allows services to reach the API via a stable name instead of an IP.

Serverless VPC Access Connector: This connector creates a bridge allowing serverless services like Cloud Run to send traffic into your VPC network.

Cloud Run Configuration: Configure the Cloud Run service to use the VPC Access connector. The application code can then call the GKE API using its private FQDN registered in Cloud DNS.

This setup ensures traffic flows entirely over private networks (within the VPC via the ILB and through the VPC Access connector), meeting the private communication requirement securely and reliably.

https://cloud.google.com/config-connector/docs/how-to/secrets#gcloud

(A) Object Lifecycle Management

Delete

The Delete action deletes an object when the object meets all conditions specified in the lifecycle rule.

Exception: In buckets with Object Versioning enabled, deleting the live version of an object causes it to become a noncurrent version, while deleting a noncurrent version deletes that version permanently.

https://cloud.google.com/storage/docs/lifecycle#delete

(B) Signed URLs

This page provides an overview of signed URLs, which you use to give time-limited resource access to anyone in possession of the URL, regardless of whether they have a Google account

https://cloud.google.com/storage/docs/access-control/signed-urls

HTTP(S) load balancing is a Google-recommended practice for distributing web traffic across multiple regions and zones, and providing high availability, scalability, and security for web applications. It supports both IPv4 and IPv6 addresses, and can handle SSL/TLS termination and encryption. It also integrates with Cloud CDN, Cloud Armor, and Cloud Identity-Aware Proxy for enhanced performance and protection. A MIG can be used as a backend service for HTTP(S) load balancing, and can automatically scale and heal the VM instances that host the web application.

To configure DNS for HTTP(S) load balancing, you need to create an A record in your DNS public zone with the load balancer’s IP address. This will map your domain name to the load balancer’s IP address, and allow users to access your web application using the domain name. A CNAME record is not recommended, as it can cause latency and DNS resolution issues. A private zone is not suitable, as it is only visible within your VPC network, and not to the public internet.

HTTP(S) Load Balancing documentation

Setting up DNS records for HTTP(S) load balancing

Choosing a load balancer

From this source:https://cloud.google.com/billing/docs/how-to/custom-roles#permission_association_and_inheritance

"For example, associating a project with a billing account requires the billing.resourceAssociations.create permission on the billing account and also the resourcemanager.projects.createBillingAssignment permission on the project. This is because project permissions are required for actions where project owners control access, while billing account permissions are required for actions where billing account administrators control access. When both should be involved, both permissions are necessary."

To set a default cluster for gcloud commands, run the following command: gcloud config set container/cluster CLUSTER_NAMEhttps://cloud.google.com/kubernetes-engine/docs/how-to/managing-clusters?hl=en

/25:

CIDR to IP Range

Result

CIDR Range 172.16.20.128/25

Netmask 255.255.255.128

Wildcard Bits 0.0.0.127

First IP 172.16.20.128

First IP (Decimal) 2886734976

Last IP 172.16.20.255

Last IP (Decimal) 2886735103

Total Host 128

CIDR

172.16.20.128/25

/24:

CIDR to IP Range

Result

CIDR Range 172.16.20.128/24

Netmask 255.255.255.0

Wildcard Bits 0.0.0.255

First IP 172.16.20.0

First IP (Decimal) 2886734848

Last IP 172.16.20.255

Last IP (Decimal) 2886735103

Total Host 256

CIDR

172.16.20.128/24

The best option is to attach a single service account to the compute instances and add minimal rights to the service account. Then, allow the service account to impersonate a Cloud Identity user with elevated permissions to create, update, or delete resources. This way, the service account can use short-lived access tokens to authenticate to Google Cloud APIs without needing to manage service account keys. This option follows the principle of least privilege and reduces the risk of credential leakage and misuse.

Option A is not recommended because it requires human intervention, which can slow down the CI/CD pipeline and introduce human errors. Option C is not secure because it grants all required IAM permissions to a single service account, which can increase the impact of a compromised key. Option D is not cost-effective because it requires creating and managing multiple service accounts and keys, as well as using a secret manager service.

1: https://cloud.google.com/iam/docs/impersonating-service-accounts

2: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys

3: https://cloud.google.com/iam/docs/understanding-service-accounts

https://cloud.google.com/spanner/docs/cpu-utilization#recommended-max

What is Google Cloud Memorystore?

Overview. Cloud Memorystore for Redis is a fully managed Redis service for Google Cloud Platform. Applications running on Google Cloud Platform can achieve extreme performance by leveraging the highly scalable, highly available, and secure Redis service without the burden of managing complex Redis deployments.

Showing Deployment Manager templates to your team will allow you to define the changes you want to implement in your cloud infrastructure. You can use Cloud Source Repositories to store Deployment Manager templates and collaborate with your team. Cloud Source Repositories are fully-featured, scalable, and private Git repositories you can use to store, manage and track changes to your code.

https://cloud.google.com/source-repositories/docs/features

as the instances (normal or preemptible) would be terminated and relaunched if the health check fails either due to application not configured properly or the instances firewall do not allow health check to happen.

GCP provides health check systems that connect to virtual machine (VM) instances on a configurable, periodic basis. Each connection attempt is called a probe. GCP records the success or failure of each probe.

Health checks and load balancers work together. Based on a configurable number of sequential successful or failed probes, GCP computes an overall health state for each VM in the load balancer. VMs that respond successfully for the configured number of times are considered healthy. VMs that fail to respond successfully for a separate number of times are unhealthy.

GCP uses the overall health state of each VM to determine its eligibility for receiving new requests. In addition to being able to configure probe frequency and health state thresholds, you can configure the criteria that define a successful probe.

Best practices for persistent disk snapshots

You can create persistent disk snapshots at any time, but you can create snapshots more quickly and with greater reliability if you use the following best practices.

Creating frequent snapshots efficiently

Use snapshots to manage your data efficiently.

Create a snapshot of your data on a regular schedule to minimize data loss due to unexpected failure.

Improve performance by eliminating excessive snapshot downloads and by creating an image and reusing it.

Set your snapshot schedule to off-peak hours to reduce snapshot time.

Snapshot frequency limits

Creating snapshots from persistent disks

You can snapshot your disks at most once every 10 minutes. If you want to issue a burst of requests to snapshot your disks, you can issue at most 6 requests in 60 minutes.

If the limit is exceeded, the operation fails and returns the following error:

https://cloud.google.com/compute/docs/disks/snapshot-best-practices

https://cloud.google.com/iam/docs/job-functions/auditing#scenario_external_auditors

Because if you directly add users to the IAM roles, then if any users left the organization then you have to remove the users from multiple places and need to revoke his/her access from multiple places. But, if you put a user into a group then its very easy to manage these type of situations. Now, if any user left then you just need to remove the user from the group and all the access got revoked

The organization creates a Google group for these external auditors and adds the current auditor to the group. This group is monitored and is typically granted access to the dashboard application. During normal access, the auditors' Google group is only granted access to view the historic logs stored in BigQuery. If any anomalies are discovered, the group is granted permission to view the actual Cloud Logging Admin Activity logs via the dashboard's elevated access mode. At the end of each audit period, the group's access is then revoked. Data is redacted using Cloud DLP before being made accessible for viewing via the dashboard application. The table below explains IAM logging roles that an Organization Administrator can grant to the service account used by the dashboard, as well as the resource level at which the role is granted.

https://cloud.google.com/kuberun/docs/architecture-overview#components_in_the_default_installation

gcloud compute ssh ensures that the user’s public SSH key is present in the project’s metadata. If the user does not have a public SSH key, one is generated using ssh-keygen and added to the project’s metadata. This is similar to the other option where we copy the key explicitly to the project’s metadata but here it is done automatically for us. There are also security benefits with this approach. When we use gcloud compute ssh to connect to Linux instances, we are adding a layer of security by storing your host keys as guest attributes. Storing SSH host keys as guest attributes improve the security of your connections by helping to protect against vulnerabilities such as man-in-the-middle (MITM) attacks. On the initial boot of a VM instance, if guest attributes are enabled, Compute Engine stores your generated host keys as guest attributes.

Compute Engine then uses these host keys that were stored during the initial boot to verify all subsequent connections to the VM instance.

Ref: https://cloud.google.com/compute/docs/instances/connecting-to-instance

Ref: https://cloud.google.com/sdk/gcloud/reference/compute/ssh

We need to apply the GCP Best practices. roles/browser Browser Read access to browse the hierarchy for a project, including the folder, organization, and IAM policy. This role doesn't include permission to view resources in the project.https://cloud.google.com/iam/docs/understanding-roles

The goal is to create a serverless, scalable, and asynchronous transaction processing system with minimal management overhead.

Serverless Requirement:Options involving installing Kafka on VMs (A, B) or using VM instances for processing (B, C) introduce management overhead associated with VMs (patching, scaling configuration, OS management) and Kafka cluster management, violating the serverless and minimal management criteria.

Asynchronous Requirement:Both Kafka and Pub/Sub can handle asynchronous messaging. However, Pub/Sub is Google Cloud's fully managed, serverless messaging service, inherently minimizing management overhead compared to self-managed Kafka on VMs.

Scalability and Processing:Cloud Run is a fully managed, serverless platform that automatically scales based on traffic, suitable for processing transactions without managing underlying infrastructure. VM instances require manual scaling configuration or managed instance groups, adding overhead.

Combining Pub/Sub for asynchronous message ingestion (fully managed, serverless) and Cloud Run for processing (fully managed, serverless, scalable) directly meets all requirements: serverless, scalable, asynchronous, and minimal management overhead. Option D is the only one that uses fully serverless components for both ingestion and processing.

Understanding IAM custom roles

Key Point: Custom roles enable you to enforce the principle of least privilege, ensuring that the user and service accounts in your organization have only the permissions essential to performing their intended functions.

Basic concepts

Custom roles are user-defined, and allow you to bundle one or more supported permissions to meet your specific needs. Custom roles are not maintained by Google; when new permissions, features, or services are added to Google Cloud, your custom roles will not be updated automatically.

When you create a custom role, you must choose an organization or project to create it in. You can then grant the custom role on the organization or project, as well as any resources within that organization or project.

https://cloud.google.com/iam/docs/understanding-custom-roles#basic_concepts

Let's analyze each option:

A. Using gsutil: While gsutil can transfer data to Cloud Storage, for 150 TB of infrequently accessed data, direct transfer over the network might be slow and consume significant bandwidth, potentially impacting other network operations. It also lacks built-in mechanisms for filtering files based on age.

B. Using Cloud Interconnect and Filestore: Cloud Interconnect provides a dedicated connection, but Filestore is a fully managed NFS service primarily designed for high-performance file sharing for applications running in Google Cloud. Migrating 150 TB of infrequently accessed data to Filestore would be cost-inefficient compared to Cloud Storage and doesn't directly address the requirement of moving older than one year files.

C. Using Transfer Appliance: Transfer Appliance is suitable for very large datasets (petabytes) or when network connectivity is poor or unreliable. While it addresses bandwidth concerns, it involves a physical appliance and might be an overkill for 150 TB of data, especially if network connectivity is reasonable.

D. Using Storage Transfer Service: Storage Transfer Service is specifically designed for moving large amounts of data between online storage systems, including on-premises file systems and Cloud Storage. It offers features like filtering by file age, scheduling transfers, and bandwidth control, directly addressing all the requirements of the question: migrating infrequently accessed files older than one year to Cloud Storage, minimizing costs (by using appropriate Cloud Storage classes for infrequent access), and controlling bandwidth usage.

Google Cloud Documentation References:

Storage Transfer Service Overview: https://cloud.google.com/storage-transfer-service/docs/overview - This page details the capabilities and use cases of Storage Transfer Service, including transferring from on-premises.

Storage Transfer Service for on-premises data: https://cloud.google.com/storage-transfer-service/docs/on-prem-overview - This specifically covers transferring data from on-premises file systems.

Cloud Storage Classes: https://cloud.google.com/storage/docs/storage-classes - Understanding the different storage classes (Standard, Nearline, Coldline, Archive) is crucial for cost optimization of infrequently accessed data. Storage Transfer Service can be configured to move data to a cost-effective class like Nearline or Coldline.

===========

Cloud Spanner is a relational database and is highly scalable. Cloud Spanner is a highly scalable, enterprise-grade, globally-distributed, and strongly consistent database service built for the cloud specifically to combine the benefits of relational database structure with a non-relational horizontal scale. This combination delivers high-performance transactions and strong consistency across rows, regions, and continents with an industry-leading 99.999% availability SLA, no planned downtime, and enterprise-grade security

Ref: https://cloud.google.com/spanner

Graphical user interface, application, Teams Description automatically generated

An external HTTP(S) load balancer is a Google-recommended solution for distributing web traffic across multiple regions and zones, and providing high availability, scalability, and security for web applications. It supports both IPv4 and IPv6 addresses, and can handle SSL/TLS termination and encryption. It also integrates with Cloud CDN, Cloud Armor, and Cloud Identity-Aware Proxy for enhanced performance and protection. A managed instance group (MIG) can be used as a backend service for the HTTP(S) load balancer, and can automatically scale the number of VM instances based on the CPU load. A Cloud Storage bucket can also be used as a backend service for the HTTP(S) load balancer, and can serve static content such as images, videos, or HTML files. A URL map can be used to route requests to different backend services based on the path or host of the request. For example, a URL map can send requests for /static/* to the Cloud Storage bucket, and requests for /dynamic/* to the MIG. A managed SSL certificate can be used to secure the connection between the clients and the load balancer, and can be automatically provisioned and renewed by Google.

A is incorrect because an internal HTTP(S) load balancer is only visible within a VPC network, and not to the public internet. It is used for internal applications that need to communicate with other internal services. Identity-Aware Proxy is a service that provides secure access to web applications without using a VPN. It is not a load balancer, and it does not distribute user traffic.

B is incorrect because installing HTTPS certificates on the instance is not necessary, as the HTTP(S) load balancer can handle SSL/TLS termination and encryption. It is also more complex and less secure to manage the certificates on the instance level, as they need to be updated and synchronized across multiple instances.

D is incorrect because an external network load balancer is a TCP/UDP load balancer that operates at the network layer. It is not suitable for web applications that use HTTP(S) protocols, as it does not support SSL/TLS termination and encryption, URL maps, or Cloud Storage backends. It is also less efficient and scalable to forward the requests to the Cloud Storage from the web servers, as it adds an extra hop and latency.

HTTP(S) Load Balancing documentation

Setting up HTTP(S) Load Balancing with Cloud Storage

Creating and using SSL certificates

Choosing a load balancer

(https://cloud.google.com/resource-manager/docs/project-migration#change_billing_account)

https://cloud.google.com/billing/docs/concepts

https://cloud.google.com/resource-manager/docs/project-migration

VMs cannot communicate over the internet without a public IP address. Private Google Access permits access to Google APIs and services in Google's production infrastructure.https://cloud.google.com/vpc/docs/private-google-access

https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints This list constraint defines the set of domains that email addresses added to Essential Contacts can have. By default, email addresses with any domain can be added to Essential Contacts. The allowed/denied list must specify one or more domains of the form @example.com. If this constraint is active and configured with allowed values, only email addresses with a suffix matching one of the entries from the list of allowed domains can be added in Essential Contacts. This constraint has no effect on updating or removing existing contacts. constraints/essentialcontacts.allowedContactDomains

Comprehensive and Detailed In Depth Explanation:

The problem states that a subnet is nearing its IP address capacity, and the requirement is to expand it without downtime and with minimal operational cost, following Google-recommended practices.

A. Creating a second VPC with the same subnet IP range and peering: While VPC Network Peering allows communication between VPCs, having overlapping IP ranges is generally not recommended and can lead to routing complexities if not managed carefully. It also adds operational overhead of managing two VPCs. This is not the most straightforward or cost-effective solution for simply expanding IP capacity within the same logical network.

B. Deleting and recreating the subnet: Deleting a subnet that contains active VM instances will cause downtime for those instances, violating a key requirement.

C. Using the Google Cloud CLI tool to expand the primary IP range of your subnet: Google Cloud allows you to expand the primary IP range of an existing subnet after it's created, as long as there are no conflicting subnets in the VPC. This operation does not require deleting the subnet or restarting the existing VMs within it, thus avoiding downtime. It's a direct and cost-effective way to increase the available IP address space within the existing subnet. This is a Google-recommended practice for handling subnet capacity issues.

D. Permitting additional traffic with firewall rules: Firewall rules control network traffic based on IP ranges, protocols, and ports. They do not increase the number of available private IP addresses within the subnet. This option does not address the core issue of IP address exhaustion.

Therefore, expanding the primary IP range of the existing subnet using the Google Cloud CLI is the recommended solution that meets all the requirements: addressing IP capacity, minimizing operational costs, and avoiding downtime.

Google Cloud Documentation References:

Expanding Subnet IP Ranges: https://cloud.google.com/vpc/docs/expand-subnet - This documentation explicitly describes how to expand the IP range of an existing subnet without downtime. It outlines the prerequisites and steps involved using the gcloud CLI or the Google Cloud Console.

VPC Network Overview: https://cloud.google.com/vpc/docs/vpc - Provides context on subnet IP ranges and their management.

===========

"large quantity" : Cloud Storage or BigQuery "files" a file is nothing but an Object

When you intially click on Monitoring(Stackdriver Monitoring) it creates a workspac(a stackdriver account) linked to the ACTIVE(CURRENT) Project from which it was clicked.

Now if you change the project and again click onto Monitoring it would create an another workspace(a stackdriver account) linked to the changed ACTIVE(CURRENT) Project, we don't want this as this would not consolidate our result into a single dashboard(workspace/stackdriver account).

If you have accidently created two diff workspaces merge them under Monitoring > Settings > Merge Workspaces > MERGE.

If we have only one workspace and two projects we can simply add other GCP Project under

Monitoring > Settings > GCP Projects > Add GCP Projects.

https://cloud.google.com/monitoring/settings/multiple-projects

Nothing about groupshttps://cloud.google.com/monitoring/settings?hl=en

https://cloud.google.com/compute/docs/instance-groups/autohealing-instances-in-migs#setting_up_an_autohealing_policy