Fix CephFS Filesystem Read-Only

After a reboot of the MDS Server it can happen that the CephFS Filesystem becomes read-only:

HEALTH_WARN 1 MDSs are read only
[WRN] MDS_READ_ONLY: 1 MDSs are read only
    mds.XXX(mds.0): MDS in read-only mode

In the MDS log you will find following entry

log_channel(cluster) log [ERR] : failed to commit dir 0x1 object, errno -22
mds.0.11963 unhandled write error (22) Invalid argument, force readonly...
mds.0.cache force file system read-only
log_channel(cluster) log [WRN] : force file system read-only
mds.0.server force_clients_readonly

This is a known upstream issue thought the fix is still not merged

As a workaround you can use following steps:

ceph config set mds mds_dir_max_commit_size 80
ceph fs fail <fs_name>
ceph fs set <fs_name> joinable true

If not successful you may need to increase the mds_dir_max_commit_size, e.g. to 160