Investigate do we need to enable ARMV8_A_NS for R-Car ARM64 boards or not

[xakep-amatop]

During implementing functionality for invalidating data cache for Xen domains, I had found that all memory pages are marked as Secure, but Zephyr running in NS-EL1 and, thus, this attribute of memory pages are ignored at all, look at MMU debug for qemu_cortex_a53:

mmap [zephyr_data]: virt 48000000 phys 48000000 size 71000 attr 60000000000710
. [0]0x4800a008: [Table] [1]0x4800b000
. . [1]0x4800b200: [Table] [2]0x4800c000
. . . [2]0x4800c000: [Page] MEM-RW-S-ELh-PXN-UXN
. . . [2]0x4800c008: [Page] MEM-RW-S-ELh-PXN-UXN

mmap [zephyr_code]: virt 48000000 phys 48000000 size 7000 attr 7d0
. . . [2]0x4800c000: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c008: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c010: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c018: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c020: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c028: [Page] MEM-RO-S-ELx-PX-UX
. . . [2]0x4800c030: [Page] MEM-RO-S-ELx-PX-UX
mmap [zephyr_rodata]: virt 48007000 phys 48007000 size 2000 attr 600000000007d0
. . . [2]0x4800c038: [Page] MEM-RO-S-ELx-PXN-UXN
. . . [2]0x4800c040: [Page] MEM-RO-S-ELx-PXN-UXN

so, it is not a big problem, because this flag ignored in NS.
But I've dived deeper in usage of ARMV8_A_NS config and seen one interesting usage for GICv3:

#ifdef CONFIG_ARMV8_A_NS
	/* Enable distributor with ARE */
	sys_write32(BIT(GICD_CTRL_ARE_NS) | BIT(GICD_CTLR_ENABLE_G1NS),
		    GICD_CTLR);
#elif defined(CONFIG_GIC_SINGLE_SECURITY_STATE)
	/*
	 * For GIC single security state, the config GIC_SINGLE_SECURITY_STATE
	 * means the GIC is under single security state which has only two
	 * groups: group 0 and group 1.
	 * Then set GICD_CTLR_ARE and GICD_CTLR_ENABLE_G1 to enable Group 1
	 * interrupt.
	 * Since the GICD_CTLR_ARE and GICD_CTRL_ARE_S share BIT(4), and
	 * similarly the GICD_CTLR_ENABLE_G1 and GICD_CTLR_ENABLE_G1NS share
	 * BIT(1), we can reuse them.
	 */
	sys_write32(BIT(GICD_CTRL_ARE_S) | BIT(GICD_CTLR_ENABLE_G1NS),
		    GICD_CTLR);
#else
	/* enable Group 1 secure interrupts */
	sys_set_bit(GICD_CTLR, GICD_CTLR_ENABLE_G1S);
#endif

so, here we can see that it enabls NS Affinity Routing for GIC, it also should be a big problem, because it may just speed up the routing of IRQs to NS world.

I believe that we need to investigate this question more, because almost all ARM64 boards have this parameter enabled by default.

[xakep-amatop]

The questions shown here already have answers in the next topic: zephyrproject-rtos#32492.

I believe we need to enable this configuration for all ARM64 boards. First and foremost, we definitely run Zephyr on the boards in the NS-EL1 state. The EL3 is used for TF-A, EL2 is used for U-Boot, and Xen in the case when we run Zephyr as Dom-0. The S-EL1 is used for OPTEE, and S-EL0 is used for OPTEE Apps.

Moreover, the bare-metal version of Zephyr + Spider board without enabling ARMV8_A_NS doesn't work at all; there are problems with GICv3.

So, within the scope of this task, it is necessary to enable ARMV8_A_NS on Spider, Salvator, and H3ULCB boards.

[xakep-amatop]

related PRs:

• boards: arm64: enable ARMV8_A_NS by default for R-Car boards zephyrproject-rtos/zephyr#68208
• https:/zephyrproject-rtos/zephyr/pull/68141/files#diff-e16f32d469118e896537b5a435b79349093b02be73aa0cea79386dae179f45a5R23