Object storage (S3 API)

Put, get, list and delete objects and presign URLs on Ceph RGW or any S3 compatible endpoint.

Blocks

Add these blocks on the canvas by searching for their name. Each block page lists its pins and how it behaves.

• delete object s3connection_delete_object Removes an object. Deleting a key that is not there is not an error.
• get object s3connection_get_object Downloads an object to a file. The object is streamed to disk, so a large file costs no extra memory.
• object exists s3connection_head_object Checks whether an object exists and reads its size, type and metadata : without downloading it.
• list objects s3connection_list_objects Lists one page of a bucket's objects under a prefix. Feed next page token back into page token to walk a large bucket.
• open connection s3connection_open Opens a connection to S3-compatible object storage (Ceph RGW, MinIO, Amazon S3). Point it at an object storage service provider so the secret key stays in the provider store and never appears in this blueprint : the key pins are the older way and leave the secret readable to anyone who can open the flow.
• presign get url s3connection_presign_get Creates a temporary URL that lets anyone read one private object, without proxying it through this server. Not available in a simulation : the URL is a real credential.
• presign put url s3connection_presign_put Creates a temporary URL that lets a browser upload one object directly to storage, without the file passing through this server. Not available in a simulation : the URL is a real credential.
• put object s3connection_put_object Uploads a file to object storage. The file is streamed from disk, so an object of any size costs the same memory.
• put value s3connection_put_value Uploads a value as an object : JSON, rendered text, or any payload the flow computed rather than read from disk.

Next steps