How to mount your S3 server on Linux

In this section we provide a step-by-step guide for mounting your S3 server on Linux via s3fs.

Installation (Debian based systems)

s3fs (free)

sudo apt-get install s3fs

Credentials

Place a file (.passwd-s3fs) containing the access credentials in the home directory. Create the file using a text editor or via echo:

$ echo ACCESS_KEY_ID:SECRET_ACCESS_KEY > ${HOME}/.passwd-s3fs

Example:

ACCESS_KEY_ID = ``IXUAXUQ52BD2JQ1L1QTH``
SECRET_ACCESS_KEY= ``gYte/E87JyOf0aCK22bw9Bh6KBHQdBequAay3Vh4``

$ echo IXUAXUQ52BD2JQ1L1QTH:gYte/E87JyOf0aCK22bw9Bh6KBHQdBequAay3Vh4 > ${HOME}/.passwd-s3fs
$ cat ${HOME}/.passwd-s3fs
IXUAXUQ52BD2JQ1L1QTH:gYte/E87JyOf0aCK22bw9Bh6KBHQdBequAay3Vh4

Limit access to credentials file to current user only:

$ chmod 600 ${HOME}/.passwd-s3fs

Mount S3 bucket

Below assumes that the bucket is mounted by the active user.

Get the active user id and ground id:

$ id
uid=1000(pi) gid=1000(pi)

Mount the bucket using:

s3fs mybucket /path/to/mountpoint -o passwd_file=${HOME}/.passwd-s3fs -o url=https://url:port/ -o use_path_request_style -o uid=value,gid=value,umask=value

With:

  • uid: User level access right
  • gid: Group level access right
  • umask: Level of access (read/write/execute)

Example:

  • Bucket name: office
  • Mount directory: ${HOME}/s3_mount/ (already created with mkdir)
  • Path to credentials file: ${HOME}/.passwd-s3fs
  • S3 server url: http://pi4server:9001/
  • User id: uid=1000
  • Group id: gid=1000
  • R/W/E access: umask=0077 (Octal format = 0, Owner = 0, Group = 7, Other users = 7)
s3fs office ${HOME}/s3_mount/ -o passwd_file=${HOME}/.passwd-s3fs -o url=http://pi4server:9001/ -o use_path_request_style -o uid=1000,gid=1000,umask=0077

The bucket is now mounted in {HOME}/s3_mount/:

$ ls ${HOME}/s3_mount/
 071E61AD   26AEB06C   51697F4D   D2920F21
 0DB63A93   2FB30D1C   A9110452   E04A72BB
 0EEBBC43   3B912722   ADC02130   F7261DAD
 12C1A48D   4B23D6A0   C8B4DC35   server

On a system with a desktop interface, the mounted directory is listed as any other directory.

Unmount

Unmount with: fusermount -u ${HOME}/s3_mount/

Options (commandline / fstab)

The configuration options passed to s3fs take two different formats depending on the usage. When using the commandline, options are passed using -o option=value - and when using the fstab configuration file, the options need to be grouped together without spaces:

option1=value,option_without_value,option3=some_other_value

Connecting to a non-AWS server

There are two methods to describe the S3 storage path:

  • bucket before the hostname (bucket.host)
  • bucket after the hostname (host/bucket)

AWS uses the first configuration, while MinIO uses the latter. To switch s3fs to use the latter configuration, the option use_path_request_style needs to be passed.

Persist S3 mounting options

To persist the mounting options on a system-wide basis, an entry with the relevant information can be added to the fstab file under /etc/fstab.

Start by creating a file with the access information in /etc/passwd-s3fs.

echo IXUAXUQ52BD2JQ1L1QTH:gYte/E87JyOf0aCK22bw9Bh6KBHQdBequAay3Vh4 | sudo tee /etc/passwd-s3fs
sudo chmod 600 /etc/passwd-s3fs

s3fs#bucketname /path/to/mount/point fuse _netdev,allow_other,use_path_request_style,passwd_file=/path/to/passwd/file,uid=value,gid=value,umask=value,url=host 0 0

Example:

  • Bucket name: office
  • Mount directory: /mnt/s3root/ (already created with mkdir)
  • Path to credentials file: /etc/passwd-s3fs
  • S3 server url: https://pi4server:9001/
  • User id: uid=1000
  • Group id: gid=1000
  • R/W/E access: umask=0077 (Octal format = 0, Owner = 0, Group = 7, Other users = 7)
s3fs#office /mnt/s3root fuse _netdev,allow_other,use_path_request_style,passwd_file=/etc/passwd-s3fs,uid=1000,gid=1000,umask=0077,url=https://pi4server:9001/ 0 0

The device can now be mounted with:

sudo mount /mnt/s3root

To unmount the drive:

sudo umount /mnt/s3root

Debugging

There are two debugging flags which can be enabled, resulting in information messages in the system log. The first flag is only for s3fs, the second is for the underlying network tool curl.

  • debuglvl (s3fs)
  • curldbg (curl)

There are multiple log levels for s3fs, but debuglvl=info is a good place to start.

HTTPS

When connecting to a self-hosted S3 server, it is important to setup the certificates to get the connection to work.

Apart from the public/private keys on the S3 server (e.g. MinIO), the public key/certificate also needs to be present on the client, and since it is self-signed it needs to be installed as a certificate authority. For debian, this requires copying the certificate to the correct folder and repackaging the root certificates:

sudo mkdir /usr/share/ca-certificates/minio-server
sudo cp public.crt /usr/share/ca-certificates/minio-server
sudo chmod 644 /usr/share/ca-certificates/minio-server/public.crt

Next, repackage the certificates (select ask and mark the new certificate manually):

sudo dpkg-reconfigure ca-certificates

A quick test if it is installed correctly can be executed by calling curl directly:

curl https://host:port/

If the certificate is not correctly installed, a message akin to the following can be observed:

curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

If the certificate is correctly installed, a response akin to the following can instead be observed:

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Resource>/</Resource><RequestId>15C3F8CB6FAB2CC8</RequestId><HostId>72bb169c-6b92-402f-bf26-4c5d35ad250f</HostId></Error>%

If the certificate is valid, but does not contain the correct hostname, the below is reported by curl:

curl: (60) SSL: certificate subject name 'certificate host' does not match target host name 'host'
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

This can also occur if the IP is used instead of the hostname or the hostname instead of the IP when calling curl, depending on what was used when generating the certificate.