8:28 a.m.
The issue is that CephFS is failing to mount. You'll have to look in
the Ceph client logs to find out why. My guess is somethings wrong with
the ceph.conf that you're using, or that you can't reach/authenticate
with the Mons.
For what it's worth, you probably don't want to export this with a
Pseudo of /. You probably want to export with a Pseudo of
/something-else. Using Pseudo of / can work, but it has issues when
exporting other filesystems, and it's now how NFSv4 was designed to
work. That's not the cause of your problem here, but it may save you
issues down the line.
Daniel
On 8/25/21 10:30 PM, ethan.deson(a)oracle.com wrote:
I have a ceph filesystem I'm trying to export using nfs ganesha.
Ceph indicates that the cluster is fine:
ceph status
cluster:
id: 108a355b-1e9b-493d-91a0-ab898a67aa51
health: HEALTH_OK
...
mds: cephfs:1 {0=unique=up:active}
But when I try to mount the filesystem over nfs with nfs ganesha I get:
$ mount -t nfs -o nfsvers=4.1,noauto,soft,sync localhost:/ test
$ touch test/i
touch: cannot touch ‘test/i’: Read-only file system
And there are errors in ganesha.log
Anything obvious wrong with my configuration?
{
Export_Id = 25;
Path = "/";
Pseudo = "/";
Access_Type = RW;
Protocols = 4;
Transports = TCP;
SecType = sys;
Squash = No_Root_Squash;
FSAL {
Name = "CEPH";
Filesystem = "cephfs";
User_Id = "testuser";
Secret_Access_Key = "<removed secret for post>";
}
}
CEPH {
Ceph_Conf = "/etc/ceph/ceph.conf";
}
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version 3.4
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_set_param_from_conf :NFS STARTUP :EVENT :Configuration file
successfully parsed
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] init_server_pkgs :NFS STARTUP :EVENT :Initializing ID Mapper.
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] init_server_pkgs :NFS STARTUP :EVENT :ID Mapper successfully
initialized.
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_start_grace :STATE :EVENT :NFS Server Now IN GRACE, duration
90
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] create_export :FSAL :CRIT :Unable to init Ceph handle for /.
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] mdcache_fsal_create_export :FSAL :MAJ :Failed to call
create_export on underlying FSAL Ceph
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/) to
(/)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] main :NFS STARTUP :WARN :No export entries found in configuration
file !!!
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] config_errs_to_log :CONFIG :CRIT :Config File
(/etc/ganesha/ganesha.conf:14): 1 validation errors in block FSAL
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] config_errs_to_log :CONFIG :CRIT :Config File
(/etc/ganesha/ganesha.conf:14): Errors processing block (FSAL)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] config_errs_to_log :CONFIG :CRIT :Config File
(/etc/ganesha/ganesha.conf:1): 1 validation errors in block EXPORT
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] config_errs_to_log :CONFIG :CRIT :Config File
(/etc/ganesha/ganesha.conf:1): Errors processing block (EXPORT)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] lower_my_caps :NFS STARTUP :EVENT :CAP_SYS_RESOURCE was
successfully removed for proper quota management in FSAL
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] lower_my_caps :NFS STARTUP :EVENT :currenty set capabilities are:
=
cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,35,36,37+eip
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] gsh_dbus_pkginit :DBUS :CRIT :dbus_bus_get failed (Failed to
connect to socket /run/dbus/system_bus_socket: No such file or directory)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] gsh_dbus_register_path :DBUS :CRIT
:dbus_connection_register_object_path called with no DBUS connection
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] gsh_dbus_register_path :DBUS :CRIT
:dbus_connection_register_object_path called with no DBUS connection
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] gsh_dbus_register_path :DBUS :CRIT
:dbus_connection_register_object_path called with no DBUS connection
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Init_svc :DISP :CRIT :Cannot acquire credentials for principal
nfs
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] gsh_dbus_register_path :DBUS :CRIT
:dbus_connection_register_object_path called with no DBUS connection
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Init_admin_thread :NFS CB :EVENT :Admin thread initialized
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_rpc_cb_init_ccache :NFS STARTUP :EVENT :Callback creds
directory (/var/run/ganesha) already exists
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_rpc_cb_init_ccache :NFS STARTUP :WARN
:gssd_refresh_krb5_machine_credential failed (-1765328160:99)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Start_threads :THREAD :EVENT :Starting delayed executor.
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Start_threads :THREAD :EVENT :gsh_dbusthread was started
successfully
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Start_threads :THREAD :EVENT :admin thread was started
successfully
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Start_threads :THREAD :EVENT :reaper thread was started
successfully
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_Start_threads :THREAD :EVENT :General fridge was started
successfully
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[dbus] gsh_dbus_thread :DBUS :CRIT :DBUS not initialized, service thread
exiting
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[dbus] gsh_dbus_thread :DBUS :EVENT :shutdown
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] rpc :TIRPC :EVENT :svc_rqst_hook_events: 0x5615da3f7070 fd 1024
xp_refcnt 1 sr_rec 0x5615da183cc0 evchan 2 ev_refcnt 3 epoll_fd 20 control fd pair (18:19)
direction in hook failed (9)
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nsm_connect :NLM :EVENT :connect to statd failed: RPC: Unknown
protocol
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nsm_unmonitor_all :NLM :EVENT :Unmonitor all nsm_connect failed
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_start :NFS STARTUP :EVENT
:-------------------------------------------------
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_start :NFS STARTUP :EVENT : NFS SERVER
INITIALIZED
26/08/2021 02:24:34 : epoch 6126fb62 : otec-blockstorage-imaging-red-1 :
ganesha.nfsd-1221[main] nfs_start :NFS STARTUP :EVENT
:-------------------------------------------------
_______________________________________________
Support mailing list -- support(a)lists.nfs-ganesha.org
To unsubscribe send an email to support-leave(a)lists.nfs-ganesha.org