Which type of account would you use in code when you want to interact with Google Cloud services?

Using an object storage service is often the primary choice in cloud computing. Its benefits, including ease of use and small operational overhead, are extremely appealing to cloud customers. In Google Cloud, the object storage service is provided by the suitably named Google Cloud Storage service.

In this article we are going to explain how to use Google Cloud Storage via the command-line tool gsutil, and other ways to interact with Google Cloud Storage. This can be also helpful for users who are getting started with the new NetApp® Cloud Volumes ONTAP for Google Cloud.

To get started now, click the links below:

What Is Object Storage?

In object storage, files are simply referenced as objects and organized into buckets—logical namespaces that are able to hold one or more objects. While a bucket can have multiple objects, an object can only belong to just one bucket. The object storage service offered by Google Cloud is called Google Cloud Storage. Google Cloud Storage comes with built-in capabilities to manage and automate bucket lifecycle rulesIn Google Cloud Storage, the availability of the data can be configured by choosing a storage class for the bucket and its objects: Standard, Nearline, and Coldline. In addition, the data location can be specified, which also impacts the availability, by choosing between region, dual-region and multi-region. Google Cloud Storage comes with built-in capabilities to manage and automate bucket lifecycle rules.

Different Ways to Interact with Google Cloud Platform

Using cloud computing resources has never been easier. The number of tools and overall methods one has at their disposal to interact with these resources is useful to a range of different users.

In Google Cloud Platform, there are three different methods that can be used:

  • Google Cloud Console: Using the Console, or Web User Interface, is the most typical option while interacting with Google Cloud. It lowers the barrier to entry-level users to start using GCP, and at the same time it provides a rich set of features and intuitive interface. However, it is worth noting that not all the features are available in the Console—users may often need to use either the SDK or command-line tool to perform certain actions.
  • SDK: An SDK, or Software Development Kit, is often the preferred way of applications to interact with Google Cloud. Google Cloud has SDK’s available for multiple programming languages including Go, Python, and node.js.
  • Command-Line Tool: While, in theory, Google Cloud’s command-line tools are part of the SDK, they are often referred to separately. Google Cloud Platform has one command-line tool gcloud that can be used to interact with any GCP service. In addition to that, Google Cloud Storage itself has also a command-line tool of its own: gsutil. This tool enables the customer to interact with GCP using all the available features at their disposal.

What would you use if you had multiple files to store in object storage in the cloud? The gsutil command-line tool of course! Now let’s see how to do it.

How to Use the gsutil Command-Line Tool

In this section we are going to explore how the gsutil command-line tool works in practice. This tool can help with the transfer of several objects, including folders, and is also well suited for automating tasks that are not so easy to achieve via the Google Cloud Console.

How to Set Up a Workspace with Google Cloud Tools

1. Start by logging in to the Google Cloud Console (the web user interface) and create a new Google Cloud Platform project with billing configured and enabled.

2. Download and install the latest version of Google Cloud SDK from the official documentation website. Depending on your machine operating system, the installation process may vary slightly, but in general should be quite straightforward.

3. You can verify that the install went successfully by opening up your machine terminal and executing the command gsutil -v. You will be greeted with the version of the gsutil installed on your machine (Fig. 1). If not, please make sure to go back to steps 1 and 2 before proceeding further.

Which type of account would you use in code when you want to interact with Google Cloud services?

4. To start using gsutil, you first need to authenticate it with your Google Cloud Platform account. Issue the command gcloud auth login and you will be directed to your browser to authenticate. Once the process is successful, both your email and active GCP project will be displayed in the terminal (Fig. 2).

Which type of account would you use in code when you want to interact with Google Cloud services?
Fig.2 Authenticate to GCP using gcloud.

Creating a Google Cloud Storage Bucket with gsutil

Creating a Google Cloud Storage bucket is the first step before any files (i.e. objects) can be uploaded to Google Cloud Storage. The bucket is a virtual namespace for our objects and, as you might recall, an object can only belong to a single bucket.

1. Using your machine terminal, issue the command gsutil help mb (Fig. 2.). This help command will provide you all the syntax details of the mb (make bucket) option. You can use the help command with any of the gsutil options, which is an invaluable resource for providing information and detailed explanations on all the mandatory and optional flags.

Which type of account would you use in code when you want to interact with Google Cloud services?
Fig.3 gsutil help pages

2. Creating the Google Cloud Storage bucket is, as you might have noticed from the previous output (Fig. 3 above), quite simple. Use the command gsutil mb gs://<YOUR_BUCKET_NAME> to create it (Fig. 4). You will need to make sure that <YOUR_BUCKET_NAME> is a unique name, not just in your GCP account but in the entire global Google Cloud Platform.

Which type of account would you use in code when you want to interact with Google Cloud services?
Fig.4 Create a GCP Bucket using gsutil

Adding/Removing Data from a Google Storage Bucket with gsutil

The process of adding and removing data from a Google storage bucket is quite simple and fast, with the only potential bottleneck being the speed of your internet connection. If you are used to the terminal, you may be familiar with the concept of the commands cp (copy) and rm (remove). The Google Cloud Storage command-line tool gsutil uses the exact same concept.


1. Start by changing the directory to the folder in your local machine where the files you want to upload are located. In our example case, this would be a folder in our Desktop called “photos” (Fig. 5).

2. In order to copy all the files in the current directory to the storage bucket we previously created, we will use the command gsutil cp * gs://<YOUR_BUCKET_NAME> (Fig. 5).


One important aspect to note is that when performing an upload of multiple large files, using the flag -m, to perform a parallel (multi-threaded/multi-processing) copy, will significantly improve the performance.

Also, if you wish to include local subfolders in your upload, you will be required to use the flag -r, i.e., to perform a recursive copy.

3. You can verify the files were properly created by using the command gsutil ls gs://<YOUR_BUCKET_NAME> and list the contents of the bucket (Fig. 5).

Which type of account would you use in code when you want to interact with Google Cloud services?
Fig.5 Copying multiple files using gsutil

4. Likewise, if you wish to remove the objects from the storage bucket, you can issue the command gsutil rm gs://<YOUR_BUCKET_NAME>/* to remove all the files inside the bucket (Fig. 6). 

5. Clean up the environment by deleting the Cloud Storage bucket you created, by using the subcommand rb (remove bucket), i.e., with the command gsutil rb gs://<YOUR_BUCKET_NAME> (Fig. 6).

Which type of account would you use in code when you want to interact with Google Cloud services?
Fig.6 Removing files and bucket using gsutil

Read More on How to Use Google Cloud

You've just seen how to interact with the Google Cloud Platform's console, command-line tools, and SDK’s.

Now that you know how to install and authenticate with Google Cloud using your machine terminal, there's a lot more on Google Cloud and Google Cloud Storage with Cloud Volumes ONTAP. Follow the links below to learn how to:

Which type of account would you use in code when you want to interact with Google Cloud services?

Key Concepts in week two:

  • Identify the purpose of projects, folders, and organization nodes on Google Cloud Platform
  • Describe the purpose of and use cases for Identity and Access Management
  • List the methods of interacting with Google Cloud Platform

Introduction

When you run workloads in GCP, you use projects to organize them. You use Google Cloud Identity, and Access Management also called IM, or IAM to control who can do what. You also use your choice of several interfaces to connect. We will learn these basics this week.

  • though a web-based console
  • through the SDK and its command-line tools
  • though the API’s
  • though mobile app

The GCP Resource Hierarchy

We are going to start from the bottom up. All the resources you use, whether they’re virtual machines, cloud storage buckets, tables, and big query or anything else in GCP are organized into projects. Projects can be organized into folders. Folders can contain other folders. All folders and projects used by our org can be brought together under an organization node. Projects, folders, and organization nodes are all places where the polices can be defined. Some GCP recesses let you put policies on individual resources too, like Cloud Storage buckets. Policies are inherited downwards in the hierarchy.

Review — The Google Cloud Platform resource hierarchy

Choose the correct completion: Services and APIs are enabled on a per-__________ basis.
Project

Identity and Access Management (IAM)

IAM lets admins authorize who can take action on specific resources. An IAM policy has a “who”, and “can do what”, and an “on which resource”.

  • “who?” — can be defined either by a Google account, a Google group, a Service account, an entire G Suite, or a Cloud Identity domain.
  • “can do what” is defined by the IAM role (a collection of permissions)
  • owner — do everything an editor can do plus manage permissions and roles on the resource (also lets you set up billing)
  • editor — do everything a viewer can plus change state
  • viewer — examine but not change state

IAM roles

Compute engines InstanceAdmin Role lets whoever has that role perform a certain set of actions on virtual machines

  • listing them
  • reading and changing configurations
  • starting and stopping

Quiz — Resources and IAM

When would you choose to have an organization node?
When you want to create folders. Folders require an organization node. Organization nodes are optional, but if you want to create folders, having one is mandatory.

Compared to AWS IAM

These bad boys kinda do the same thing. It’s all just a mechanism to secure user authentication and permission for the cloud. Here is a chart briefly showing the differences

Interacting with GCP

There are four ways to interact with GCP: the Console, the SDK & Cloud Shell, the Mobile App, and the APIs.

API Explorer

The GCP Console includes a tool called the APIs Explorer that helps you learn about the APIs interactively. It lets you see what APIs are available and in what versions. These APIs expect parameters and documentation on them is built-in. You can try the APIs interactively even with user authentication. Google provides client libraries that take a lot of the work out of the task of calling GCP from your code.

Cloud marketplace

Cloud marketplace is for staring up with GCP with little to no effort. Quickly deploying software packages on GCP. pre-packaged, ready-to-deploy solutions.

Quiz — Getting Started with Google Cloud Platform

True or False: In Google Cloud IAM: if a policy applied at the project level gives you Owner permissions, your access to an individual resource in that project might be restricted to View permission if someone applies a more restrictive policy directly to that
False. Policies are a union of those applied on the resource itself and those inherited from higher levels in the hierarchy. If a parent policy is**less**restrictive, it overrides a more restrictive policy applied on the resource. If a parent policy is**more**restrictive, it does not override a less restrictive policy applied on the resource. Therefore, access granted at a higher level in the hierarchy cannot be taken away by policies applied at a lower level in the hierarchy.