Up and running with LibcephFS

Recently, I was trying to play with LibcephFS and I wasn’t able to find a comprehensive tutorial to get me started experimenting with CephFS real quick. So, I had to learn the API the hard way by looking into different articles dealing with the FUSE-based client and the LibcephFS Python SDK’s source code. But, since that’s time-consuming, I will show you with an example application on how to get started with LibcephFS smoothly in almost no time.

1. Create a very basic Ceph environment. I like to use the Ceph Docker image for this purpose where we can start all the required daemons inside the container to replicate a single node Ceph cluster setup.

$ docker pull ceph/ceph:v15.2.4
$ docker run --rm -it ceph/ceph:v15.2.4 bash

2. Now that we are inside a Ceph container, we would create a single node cluster with one daemon each of an OSD, MGR, MDS, and MON. This can be achieved by running the script given below inside the container. This script was is a slightly simpler version of the original script present here.

3. Next, we would create 2 Rados pools, a data pool, and a metadata pool for use by the CephFS. These pools can be either replicated or erasure-coded pools.

$ ceph osd pool create cephfs_data 64
$ ceph osd pool create cephfs_metadata 64

4. Create a new filesystem on top of the data and the metadata pool. Initializing the filesystem would change the state of the MDS from standby to active. You can verify this by doing a quick ceph -s .

$ ceph fs new cephfs cephfs_metadata cephfs_data

5. Now it’s time to create an example CephFS client and start experimenting. Since the easiest way is to get started with a Python client application, we will start by installing the python-cephfs package.

$ yum install python-cephfs

6. Lastly, we would write a very minimal Python client application. In this example, we create a directory, write a file inside the directory, read from the file, explore the file layout, and also convert a file inode to a physical object id. The detailed API reference can be found here.

And, that’s it. I hope, now you can explore more of CephFS using LibcephFS. Feel free to comment with any suggestions you have. I will be more than happy to update the article. Thanks and Cheers!