Google Cloud Storage: Managing Files and Objects

In the first part of this two-part tutorial series, we had an overview of how buckets are used on Google Cloud Storage to organize files. We saw how to manage buckets on Google Cloud Storage from Google Cloud Console. This was followed by a Python script in which these operations were performed programmatically.

In this part, I will demonstrate how to manage objects, i.e. files and folders inside GCS buckets. The structure of this tutorial will be similar to that of the previous one. First I will demonstrate how to perform basic operations related to file management using Google Cloud Console. This will be followed by a Python script to do the same operations programmatically.

Just as bucket naming in GCS had some guidelines and constraints, object naming follows a set of guidelines as well. Object names should contain valid Unicode characters and should not contain Carriage Return or Line Feed characters. Some recommendations include not to have characters like "#", "[", "]", "*", "?" or illegal XML control characters because they can be interpreted wrongly and can lead to ambiguity.

Also, object names in GCS follow a flat namespace. This means physically there are no directories and subdirectories on GCS. For example, if you create a file with name /tutsplus/tutorials/gcs.pdf, it will appear as though gcs.pdf resides in a directory named tutorials which in turn is a subdirectory of tutsplus. But according to GCS, the object simply resides in a bucket with the name /tutsplus/tutorials/gcs.pdf.

Let's look at how to manage objects using Google Cloud Console and then jump onto the Python script to do the same thing programmatically.

Using Google Cloud Console

I will continue from where we left in the last tutorial. Let's start by creating a folder.

Create a folder or upload files directly to GCS

To create a new folder, click on the Create Folder button highlighted above. Create a folder by filling in the desired name as shown below. The name should follow the object naming conventions.

Creating a folder in GCS

Now let's upload a file in the newly created folder.

Uploading a file in GCS

After the creation, the GCS browser will list the newly created objects. Objects can be deleted by selecting them from the list and clicking on the delete button.

Delete an object from GCS

Clicking on the refresh button will populate the UI with any changes to the list of objects without refreshing the whole page.

Managing Objects Programmatically

In the first part, we saw how to create a Compute Engine instance. I will use the same here and build upon the Python script from the last part.

Writing the Python Script

There are no additional installation steps that need to be followed for this tutorial. Refer to the first part for any more details about installation or development environment.

gcs_objects.py

The above Python script demonstrates the major operations that can be performed on objects. These include:

  • creation of a new object in a bucket
  • listing of all objects in a bucket
  • deletion of a specific object

Let's see how each of the above operations looks when the script is run.

Conclusion

In this tutorial series, we saw how Google Cloud Storage works from a bird's eye view, which was followed by in-depth analysis of buckets and objects. We then saw how to perform major bucket and object related operations via Google Cloud Console. 

Then we performed the same using Python scripts. There is more that can be done with Google Cloud Storage, but that is left for you to explore.

Tags:

Comments

Related Articles