S3

Switch Cloud S3 (SCS3) is an object storage service that ensures performance, data safety, availability and security. SCS3 is designed to tackle a variety of workloads, from backups and media storage to HPC and AI.

What is an object store?

An object store is a simple storage system where data is stored in dictionaries/hashtables called buckets. An S3 bucket is a logical container where objects (such as files, images, videos, backups, and other data) are stored. To operate on objects all you have to do is to provide a bucket name and a filename.

For example, to upload a file to S3, you can just run aws s3 cp myfile s3://my-bucket/myfile, more examples will follow.

What does SCS3 offer?

Scalability: S3 buckets are designed to handle large volumes of data, making it an ideal solution for large-scale storage needs. The system itself can extend without downtime.
Accessibility: Objects stored in S3 buckets can be accessed via a variety of S3 compatible clients like AWS CLI, Rclone, etc.
Security: S3 supports IAM and bucket policies and server side encryption (SSE-C). You can define who has access to your buckets and what actions they can perform, ensuring that your data is protected.
Safety: S3 offers robust data durability and redundancy, ensuring that your data is safe even in the case of hardware failures.
Versioning and immutability: Enabling object-locking on a bucket allows you to keep files immutable. These files can still be edited and deleted, but when that happens, the older version of the file is kept. The file's older versions stay immutable and can only be cleaned up via lifecycle policies.

Choose Your S3 Bucket Storage Type

SCS3 offers two distinct types of storage solutions depending on your current workload:

SSD (Solid-State Drive) Storage:
- High Performance: Engineered for applications that require rapid data access and low latency.
- Suitable for frequently accessed data: Ideal for environments where data is read and written frequently.
HDD (Hard Disk Drive) Storage:
- Cost Effective: HDD storage is ideal for applications with large volumes of data where cost-efficiency is critical. It provides ample storage capacity at a lower price point compared to SSDs.
- Suitable for Archival: HDD storage is well-suited for data archiving, backup solutions, and other use cases where access speed is less critical but large storage is required.

How to Use S3?

Create a Bucket: To start using S3 storage, you first need to create a bucket. Each bucket must have a unique name within the scope of your S3 solution. This name is used to reference the bucket in API calls and to access the stored objects. For more information on how to create an S3 Bucket visit Create an S3 Bucket.
Upload Objects: Once a bucket is created, you may upload objects to it. Objects are stored with a unique identifier known as a key, which helps in retrieving the object later. For more information about uploading objects to S3 visit Data Management.
Organise Data: While S3 buckets do not support a traditional file system hierarchy, you can simulate a folder structure using object keys that include slashes /. For example, an object with a name like photos/2023/vacation.jpg can represent a nested structure within a bucket.
Access Objects: Objects stored in buckets may be accessed using their keys through the S3 API or SDKs. You may also generate pre-signed URLs for temporary access to objects without exposing your credentials.
Manage Users and Keys: Create users for your project, assign them to groups, give them permissions. If you don't want to manually create users, you can leverage the Secure Token Service (STS) to allow your own authetication authority to create users for you.
Manage Policies: Policies are applied to either a bucket or user level to control who can access which objects during an S3 query. Policies specify who may read, write, or delete objects, and may be fine-tuned to meet specific security requirements. For more information about policies, please visit the Policies tab.