Kernel does not seem to mount NFS rootfs #591

armhzjz · 2021-02-19T15:20:00Z

armhzjz
Feb 19, 2021

Hello,
As previously posted in this post, I want to boot a jeston Xavier from tftp and nfs servers. I have managed to make the bootloader retrieve the dtb and kernel from a tftp server. Then the kernel boots.
My next step would be for the kernel to access the rootfs via nfs, but despite having the correct kernel arguments being passed to the kernel (they are included in the dtb), the kernel does not mount rootfs from nfs but from eMMC (or so I think).
This is what I see in the console:

...
[   14.434310] ., OK
[   14.466313] IP-Config: Got DHCP answer from 10.42.0.240, my address is 10.42.1.81
[   14.466476] IP-Config: Complete:
[   14.466541]      device=eth0, hwaddr=00:04:4b:cb:b0:37, ipaddr=10.42.1.81, mask=255.255.0.0, gw=10.42.0.2
[   14.466692]      host=jetsonXavier_1, domain=HDK.local, nis-domain=(none)
[   14.466801]      bootserver=10.42.3.180, rootserver=10.42.3.180, rootpath=/opt/data/jetsXavier_fs     nameserver0=10.42.0.2
[   14.471172] vdd-1v8-cvb: disabling
[   14.471286] vdd-1v8-sd: disabling
[   14.471352] vdd-epb-1v0: disabling
[   14.471428] avdd-cam-2v8: disabling
[   14.471512] vdd-sata-1v5: disabling
[   14.471574] vdd-3v3-slt: disabling
[   14.471637] vdd-fan: disabling
[   14.471697] vdd_sys_en: disabling
[   14.471797] ALSA device list:
[   14.471852]   #0: tegra-hda-galen-t194 at 0x3518000 irq 70
[   14.471981]   #1: tegra-snd-t19x-mobile-rt565x
[   14.475722] Freeing unused kernel memory: 10816K
[   14.880828] EXT4-fs (mmcblk0p1): couldn't mount as ext3 due to feature incompatibilities
[   14.881531] EXT4-fs (mmcblk0p1): couldn't mount as ext2 due to feature incompatibilities
[   14.891340] EXT4-fs (mmcblk0p1): recovery complete
[   14.892232] EXT4-fs (mmcblk0p1): mounted filesystem with ordered data mode. Opts: (null)
[   14.955998] cgroup: cgroup2: unknown option "nsdelegate"
[   14.963790] systemd[1]: systemd 244.5+ running in system mode. (-PAM -AUDIT -SELINUX +IMA -APPARMOR -SMACK +SYSVINIT +UTMP -LIBCRYPTSETUP -GCRYPT -GNUTLS $
ACL +XZ -LZ4 -SECCOMP +BLKID -ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=hybrid)
[   14.965010] systemd[1]: Detected architecture arm64.
[   14.972288] systemd[1]: Set hostname to <jetson-xavier>.
[   15.156819] random: systemd: uninitialized urandom read (16 bytes read)
[   15.160239] systemd[1]: Created slice system-getty.slice.
[   15.160696] random: systemd: uninitialized urandom read (16 bytes read)
[   15.161469] systemd[1]: Created slice system-serial\x2dgetty.slice.
[   15.161756] random: systemd: uninitialized urandom read (16 bytes read)
[   15.162453] systemd[1]: Created slice User and Session Slice.
[   15.162868] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[   15.163247] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[   15.163605] systemd[1]: Reached target Paths.
[   15.163889] systemd[1]: Reached target Remote File Systems.
[   15.164127] systemd[1]: Reached target Slices.
[   15.164355] systemd[1]: Reached target Swap.
[   15.165373] systemd[1]: Listening on Syslog Socket.
....

The kernek arguments as follow:

[    0.000000] Kernel command line: console=ttyTCU0,115200 console=tty0 fbcon=map:0 root=/dev/nfs nfsroot=10.42.3.180:/opt/data/jetsXavier_fs,vers=3,proto=tcp rw rootfstype=ext4 ip=dhcp nfsrootdebug rootwait elevator=deadline video=tegrafb no_console_suspend=1 earlycon=tegra_comb_uart,mmio32,0x0c168000 gpt usbcore.old_scheme_first=1 tegraid=19.1.2.0.0 maxcpus=8 boot.slot_suffix= boot.ratchetvalues=0.4.2 vpr_resize sdhci_tegra.en_boot_part_access=1

I can see that dhcp did its thing in configuring the ip address of my device and that the root is set to the pseudo device /dev/nfs. I also can see where the nfsroot is set to (both ip address and location on the nfs server) among other nfs options passed to the kernel.
But the rootfs is simply not mounted.

Does anybody have any idea as of why this might be?
By the way, the kernel has NFS client support enabled only for version 1, 2 and 3 - no NFSv4 or NFSv4.1.

Answered by madisongh

Feb 20, 2021

Are you booting with tegra-minimal-initrafms as the initramfs? The initrd scripts included there might need some modifications to handle NFS-mounting the rootfs.

View full answer

madisongh · 2021-02-20T12:59:04Z

madisongh
Feb 20, 2021
Maintainer

Are you booting with tegra-minimal-initrafms as the initramfs? The initrd scripts included there might need some modifications to handle NFS-mounting the rootfs.

5 replies

armhzjz Feb 22, 2021
Author

Hello.
No, I am not using tegra-minimal-initramfs (is this mandatory for nfs booting?).
I simply modify the dtb to include the relevant kernel arguments that are passed when booting:

root=/dev/nfs nfsroot=10.42.3.180:/opt/data/jetsXavier_fs,vers=3,proto=tcp rw rootfstype=nfs ip=dhcp rootwait elevator=deadline

Do you know if there is something additional to this that I should take into consideration (i.e. any special NVIDIA mechanism that prevents me from getting the nfs rootfs mounted)?

madisongh Feb 22, 2021
Maintainer

No, I am not using tegra-minimal-initramfs (is this mandatory for nfs booting?).

So you're explicitly setting INITRAMFS_IMAGE = "" (or something else) in your machine config? Because we include that initramfs image in the build by default.

Do you know if there is something additional to this that I should take into consideration (i.e. any special NVIDIA mechanism that prevents me from getting the nfs rootfs mounted)?

Nothing that I know of that's specific to NVIDIA (and with the latest L4T/JetPack release, they support NFS mounting the rootfs). I mentioned the initramfs image because that is something that we include that could interfere.

armhzjz Feb 22, 2021
Author

That means I need to explicitly set INITRAMFS_IMAGE = "" ? I would have never found that out by myself. Thanks a lot for the clarification. This brings hope back to me :D
Another thing that I would like to clarify further; I am building JetPack 4.4 and AFAIK that is not the latest JetPack release.
So, you mean to say that JetPack version 4.4 doesn't support NFS mouning?
I feel both hopefull and embarrased for not having known this before.

madisongh Feb 22, 2021
Maintainer

So, you mean to say that JetPack version 4.4 doesn't support NFS mouning?

Not out-of-the-box with the stock L4T/JetPack setup that NVIDIA provides (i.e., their Ubuntu distro), although you could probably get it to work (even the instructions they provide for L4T32.5/JP4.5 look complicated to me). With OE/Yocto builds, you should be able to make it work.

armhzjz Feb 22, 2021
Author

Hi @madisongh,
Again thanks for the help.
I tried to create a build in which I set INITRAMFS_IMAGE = "" but my build fails on the do_image_tegraflash task of the core-image-minimal recipe when attempting to copy the unexistent file "Image-initramfs-jetson-xavier.cboot". See part of the log below or completeness:

Initialising tasks: 100% |#####################################################################################################################################################################| Time: 0:00:02
Sstate summary: Wanted 192 Found 191 Missed 1 Current 1472 (99% match, 99% complete)
NOTE: Executing Tasks
ERROR: core-image-minimal-1.0-r0 do_image_tegraflash: Execution of '/workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_tegraflash.287' failed with exit code 1:
cp: cannot stat '/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot': No such file or directory
WARNING: /workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_tegraflash.287:1 exit 1 from 'cp "/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot" ./boot.img'

ERROR: Logfile of failure stored in: /workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/log.do_image_tegraflash.287
Log data follows:
| DEBUG: Executing python function set_image_size
| DEBUG: 1980326.400000 = 1523328 * 1.300000
| DEBUG: 29360128.000000 = max(1980326.400000, 29356032)[29356032.000000] + 4096
| DEBUG: 29360128.000000 = int(29360128.000000)
| DEBUG: 29360128 = aligned(29360128)
| DEBUG: returning 29360128
| DEBUG: Python function set_image_size finished
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['virtual:native:/workdir/poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.4.2.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/patch/patch_2.7.6.bb:do_populate_sysroot', 'vi
rtual:native:/workdir/poky/meta/recipes-devtools/python/python3_3.8.2.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-extended/pigz/pigz_2.4.bb:do_populate_sysroot', '/workdir/meta-tegra/contrib/recipes-devtools/gcc-8/libgcc_8.3.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/e2fsprogs/e2fsprogs_1.45.4.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-devtools/binutils/binutils-cross_2.34.bb:do_populate_sysroot', '/w
orkdir/poky/meta/recipes-core/glibc/glibc_2.31.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-core/glibc/cross-localedef-native_2.31.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/makedevs/makedevs_1.0.1.bb:do_populate_sysroot', '/workdir/meta-tegra/../meta-tegra/recipes-bsp/tegra-binaries/tegra186-flashtools-native_32.4.3.bb:do_populate_sysroot', '/workdir/meta-tegra/contrib/recipes-devtools/gcc-8/gcc-cross_8.3.bb:do_populate_sysro
ot', 'virtual:native:/workdir/poky/meta/recipes-devtools/opkg/opkg_0.4.2.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-kernel/kmod/depmodwrapper-cross_1.0.bb:do_populate_sysroot', '/workdir/meta-tegra/../meta-tegra/recipes-bsp/te
gra-binaries/tegra-redundant-boot-rollback_32.4.3.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/bison/bison_3.5.4.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-devtools/mklibs/mklibs-native_0.1.44.b
b:do_populate_sysroot', '/workdir/poky/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-core/glibc/ldconfig-native_2.12.1.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/me
ta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/fdisk/gptfdisk_1.0.4.bb:do_populate_sysroot', '/workdir/meta-tegra/../meta-tegra/recipes-kernel/linux/linux-tegra_4.9.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/perl/perl_5.30.1.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-devtools/qemu/qemuwrapper-cross_1.0.bb:do_populate_sysroot', 'virtual:native:/workdir/
poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot', '/workdir/poky/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-kernel/dtc/dtc_1.6.0.bb:do_populate_s
ysroot', 'virtual:native:/workdir/poky/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/workdir/meta-tegra/contrib/recipes-devtools/gcc-8/gcc-runtime_8.3.bb:do_populate_sysroot', '/workdir/meta-tegra/../meta-tegra/recipes-b
sp/tegra-binaries/tegra-bootfiles_32.4.3.bb:do_populate_sysroot', 'virtual:native:/workdir/poky/meta/recipes-devtools/prelink/prelink_git.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['opkg-utils-native', 'patch-native', 'python3-native', 'pigz-native', 'libgcc', 'e2fsprogs-native', 'binutils-cross-aarch64', 'glibc', 'cross-localedef-native', 'makedevs-native', 'tegra18
6-flashtools-native', 'gcc-cross-aarch64', 'opkg-native', 'depmodwrapper-cross', 'tegra-redundant-boot-rollback', 'bison-native', 'mklibs-native', 'kern-tools-native', 'ldconfig-native', 'update-rc.d-native', 'gptfdisk-native', 'linux-t
egra', 'perl-native', 'qemuwrapper-cross', 'pseudo-native', 'quilt-native', 'dtc-native', 'bc-native', 'gcc-runtime', 'tegra-bootfiles', 'prelink-native', 'xz-native', 'attr-native', 'libtool-native', 'gnu-config-native', 'automake-nati
ve', 'autoconf-native', 'systemd-systemctl-native', 'libtirpc-native', 'util-linux-native', 'libnsl2-native', 'openssl-native', 'libffi-native', 'zlib-native', 'readline-native', 'gdbm-native', 'sqlite3-native', 'pkgconfig-native', 'bzi
p2-native', 'shadow-native', 'linux-libc-headers', 'gettext-minimal-native', 'texinfo-dummy-native', 'flex-native', 'tegra-helper-scripts-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'libsolv-native', 'libarchive-native', 'kmo
d-native', 'shared-mime-info-native', 'popt-native', 'ncurses-native', 'db-native', 'qemu-native', 'debianutils-native', 'tegra-flashvars', 'elfutils-native', 'binutils-native', 'm4-native', 'libpcre2-native', 'libcap-ng-native', 'expat
-native', 'ninja-native', 'cmake-native', 'lzo-native', 'gtk-doc-native', 'libxml2-native', 'glib-2.0-native', 'itstool-native', 're2c-native', 'curl-native', 'meson-native', 'gettext-native', 'libpcre-native', 'python3-setuptools-nativ
e', 'unzip-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function do_image_tegraflash
| cp: cannot stat '/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot': No such file or directory
| WARNING: /workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_tegraflash.287:1 exit 1 from 'cp "/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot" ./boot.img'
| ERROR: Execution of '/workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_tegraflash.287' failed with exit code 1:
| cp: cannot stat '/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot': No such file or directory
| WARNING: /workdir/build/tmp/work/jetson_xavier-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_tegraflash.287:1 exit 1 from 'cp "/workdir/build/tmp/deploy/images/jetson-xavier/Image-initramfs-jetson-xavier.cboot" ./boot.img'
|
ERROR: Task (/workdir/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_tegraflash) failed with exit code '1'
NOTE: Tasks Summary: Attempted 4195 tasks of which 4194 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /workdir/poky/meta/recipes-core/images/core-image-minimal.bb:do_image_tegraflash
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
2021-02-22 22:35:49 - ERROR    - Command returned non-zero exit status 1
pokyuser@79306ab36f05:/workdir$

I will ask this question but I am affraid I might know the answer already; is it a must to define any kind of initramfs when building for Jetson Xavier? I ask this because after checking the initrd_script you pointed me at, it seems to me that the easiest way here would be to actually use a modified version of the initramfs scripts (or of one of them) when in fact I would have expected (or would have preferred) to simply define the INITRAMFS_IMAGE as blank in the configuration.

madisongh · 2021-02-23T14:15:42Z

madisongh
Feb 23, 2021
Maintainer

Yeah, I guess it's been a while since I tested a setup with a kernel and no initramfs. Sorry about that, a fix should be in shortly. The main reason for including an initramfs has been to ensure that the USB controller firmware gets loaded early in the boot process.

In the meantime, yes, updating the initrd script to better handle NFS mounts would simpler.

4 replies

madisongh Feb 23, 2021
Maintainer

I've updated all the recent-vintage branches with fixes for handling the no-initramfs/initrd case, so just nullfiying INITRAMFS_IMAGE should work now.

armhzjz Feb 23, 2021
Author

Hi,
Thanks a lot for the update. I tried your changes but unfortunatelly I was not able to build for branch dunfell-l4t-r32.4.3. For the mentioned branch I keep getting the same error I mentioned on an earlier comment.
I also tried with dunfell-l4t-r32.5.0 but also could not get it working (linux-tegra-4.9 cannot be fetched, but I think this must be a different issue present in my local host/network).

In regards of branch dunfell-l4t-r32.4.3, my understanding is I should simply define INITRAMFS_IMAGE = "" in my local.conf configuration file and that should suffice --> Am I right?

madisongh Feb 24, 2021
Maintainer

That should suffice, yes. Turns out I missed porting one patch from master yesterday. It's there now, so it should work if you update again to the latest on dunfell-l4t-r32.4.3.

armhzjz Feb 25, 2021
Author

Thanks a lot @madisongh. It works like charm!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kernel does not seem to mount NFS rootfs #591

{{title}}

Replies: 2 comments 9 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Kernel does not seem to mount NFS rootfs #591

armhzjz Feb 19, 2021

Replies: 2 comments · 9 replies

madisongh Feb 20, 2021 Maintainer

armhzjz Feb 22, 2021 Author

madisongh Feb 22, 2021 Maintainer

armhzjz Feb 22, 2021 Author

madisongh Feb 22, 2021 Maintainer

armhzjz Feb 22, 2021 Author

madisongh Feb 23, 2021 Maintainer

madisongh Feb 23, 2021 Maintainer

armhzjz Feb 23, 2021 Author

madisongh Feb 24, 2021 Maintainer

armhzjz Feb 25, 2021 Author

armhzjz
Feb 19, 2021

Replies: 2 comments 9 replies

madisongh
Feb 20, 2021
Maintainer

armhzjz Feb 22, 2021
Author

madisongh Feb 22, 2021
Maintainer

armhzjz Feb 22, 2021
Author

madisongh Feb 22, 2021
Maintainer

armhzjz Feb 22, 2021
Author

madisongh
Feb 23, 2021
Maintainer

madisongh Feb 23, 2021
Maintainer

armhzjz Feb 23, 2021
Author

madisongh Feb 24, 2021
Maintainer

armhzjz Feb 25, 2021
Author