Vulkan backend support.

[cheako]

I just learned about rust and now am lamenting my decision to write vkwayland in CPP. This project looks to have most everything required to support a Vulkan Backend and I'd be interested in contributing.

Might I suggest to make use of vulkan-malloc and by extension dacite. I looked at vulkano and fear that it goes against the spirit of Vulakn by implementing runtime checks when a big selling point of Vulkan is that it mostly eliminates them.

[elinorbgr]

This is at least partially related to #129

[elinorbgr]

To expand a little more, vulkan support is definitely somewhere on our roadmap, though where exactly is pretty unclear. If you're interested in helping us get there, well I can only thank you for your interest, this is very welcome!

I myself am not that familiar with the whole graphics stack though, this is more @Drakulix's domain. Some discussion has already been started in #129 about support for gfx-hal, which I believe is related to Vulkan support.

The exact specifics on how we'll support Vulkan are still pretty blurry though, and your expertise on that matter will likely help. :)

[cheako]

I took a look at gfx-rs/hal and it's a Vulkan oriented abstraction across a range of graphical APIs. Of note there is no support for Direct to Display drawing, a must for any serious Compositor.

It's not clear how legacy GL and Vulkan are made to be compatible. My assumption is that it's a case of gfx-hal being a Jack of All Trades. I don't think it's a suitable API for high performance(I.E. low latency) applications.

[cheako]

I started a branch to develop a Vulkan compositor, I figured that code changes would happen in parallel. Thus far I've only made changes to cargo, though I don't know what I'm doing.

cheako@7805efb#diff-80398c5faae3c069e4e6aa2ed11b28c0
cheako@7805efb#diff-7688ece93ac60e73de73f8ceaaf70408

[elinorbgr]

I don't think adding vulkan support would require creating a new crate. Rather, you might want to look into the backend structure of smithay:

• src/backend/egl contains most of the code for initializing and manipulating an EGL context, it uses the traits defined in egl/native.rs to abstract away the concept of "native display" and "native surface" so that the same code can handle EGL contetextes from x11/wayland environments (via winit) or DRM/GBM directly
• src/backend/drm contains the code for handling DRM and GBM, and provides type implement the required trait for being used with the EGL machinnery
• src/backend/winit.rs somilarly provides types built on top of winit implementing the same set of trait

My guess would be that the road forward for Vulkan support would be to introduce a new module on the same structure as the EGL one for handling Vulkan, similarly based on traits to abstract away the notion of "native stuff" (I don't know if Vulkan has a similar notion of display / surface ?), traits that would be implemented by the relevant types in the Drm and Winit modules. And maybe these traits could even be unified with the EGL ones?

[cheako]

I don't know much about EGL or how it works, so I'm not able to understand the existing code.

I've looked at src/backend/drm/mod.rs and I don't see how that would map to the Vulkan API. In particular you don't/can't pass a DRM device to Vulkan.

For the CPP vkwayland there are two impl objects that initialize Vulkan.

Firstly we have the usual GLFW path:

1. We create a *somewhat short-lived VkInstance. GLFW chooses the Instance Extensions it needs. Specifying the version of Vulkan your code is written against, it also gives Vulkan a chance to filter by Engine and Application names and versions. Note that GLFW does not support run-time detection of Wayland/X11, I'm interested in discovering how winit works.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-93

• It's used sporadically for generating function pointers for the DMABUF extension.

2. Create the Window and Surface using GLFW to abstract the X11/Wayland displays.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-111
3. Assume the first Physical Device, but probes the command queue to use. Normally this filtering would be a lot more advanced, but without a ton of systems to test against it's largely a guessing game.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-134
4. Create the Logical Device, this might be much like an OpenGL Context. Notice the Device Extensions.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/glfw/vulkanio.cpp#lines-192

Then there is the fbcon(Direct to Display) path:

1. VkInstance, Notice line 256(VK_KHR_DISPLAY_EXTENSION_NAME).
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-315
2. There simply is no analog to creating Windows and Surfaces when using Direct to Display.
3. This time we use every Physical Device that has a graphics command queue. I might have a bug here, but it's not important... Like I said this part of Vulkan needs testing across many devices/systems.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-342
4. Logical Device, wow look at all the Device Extensions.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-412
5. There is some code run on each frame looking at the Displays and comparing that to the configured Displays. This is used to setup each Monitor for each Physical Device.
   https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/fbcon/vulkanio.cpp#lines-201

After the Logical Device is created the two code paths merge. That API is defined here:
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/include/render.hpp#lines-89

Lines 102 to 104 are in reference to Devices and Displays and the following "Surface" are Wayland surfaces. At the time I was writing the Monitor hotplug I didn't know Wayland used Surfaces also, but CPP doesn't care they are named the same.

All these functions iterate over m_devices(line 153) and call similar functions where the functions are run against each "OutSurface" where I got a tad better at naming things:
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/render/device.hpp#lines-27

[elinorbgr]

I'm sorry, this whole thing is getting way out of my expertise, I'll let @Drakulix take things from now... 😅

[elinorbgr]

But that won't stop me to try and understand anyway.

So first, to give you some more context about winit: as opposed to glfw, winit only handles the windowing itself, and doesn't know anything about opengl or vulkan. So it'll just provide pointers to the x11/wayland display and the x11/wayland surface, and we'd be left to setup the Vulkan WSI using them. The retreival of these pointers from a winit window is done via the WindowExt extension trait from winit: https://docs.rs/winit/0.19.0/winit/os/unix/trait.WindowExt.html

Now, regarding vulkan on tty, currently the GBM/DRM code of smithay allows to control how rendering devices are connected to monitors. With vulkan, all of this is handled directly via the vulkan API, rather than relying on libdrm and libgbm? And does Vulkan have some native mechanisms for importing dmabufs from clients?

[Drakulix]

Alright, I will try to clear up some confusion here.

I looked at vulkano and fear that it goes against the spirit of Vulkan by implementing runtime checks when a big selling point of Vulkan is that it mostly eliminates them.

Vulkano is still miles ahead in terms of performance compared to OpenGL and some user of smithay might prefer a safer way to use vulkan. No matter what solution we settle upon, I want to support vulkano as well. But as long as we do not close off vulkano support, I am open to support additional crates.

I myself am not that familiar with the whole graphics stack though, this is more @Drakulix's domain. Some discussion has already been started in #129 about support for gfx-hal, which I believe is related to Vulkan support.

gfx-hal is actually more a user-side issue, then a backend side. What gfx-hal does is mapping a vulkan-like API to different backends (vulkan, opengl, dx12, metal, etc).

So gfx-hal will be a useful abstraction for users to unify access to all the different backends, but we still need to implement a vulkan-backend.

I took a look at gfx-rs/hal and it's a Vulkan oriented abstraction across a range of graphical APIs. Of note there is no support for Direct to Display drawing, a must for any serious Compositor.

gfx-hal in general is missing support for direct display rendering in any backend. #129 tries to change gfx-hal's OpenGL backend to work on our provided implementation.

I am sure similar adjustments can be made once we have a working vulkan backend to gfx-hal to support that as well.

But yes, in its current state it is not usable for smithay.

It's not clear how legacy GL and Vulkan are made to be compatible. My assumption is that it's a case of gfx-hal being a Jack of All Trades. I don't think it's a suitable API for high performance(I.E. low latency) applications.

If the underlying backend is vulkan the overhead is almost zero, since calls are just mapped. The overhead is also quite small if dx12 or metal is used and absolutely usable as gfx-portability shows.

The GL layer requires at least OpenGL 3.2, I think, which I would not really describe as legacy. (There are still a bunch of OpenGL 2.1 cards on the market for embedded/mobile/low-cost systems.)
There is definitely some overhead involved in mapping those api's, but OpenGL is not the primary target of gfx-hal anyway.

I hope this helps to understand, how gfx-hal will work with smithay, and that it is irrelevant for this issue, so back to implementing a vulkan backend.

I've looked at src/backend/drm/mod.rs and I don't see how that would map to the Vulkan API. In particular you don't/can't pass a DRM device to Vulkan.

There are two ways right now to implement a compositor with Vulkan support.

One is the one you are talking about, namely the VK_KHR_display extension.

With vulkan, all of this is handled directly via the vulkan API, rather than relying on libdrm and libgbm?

And that solution does in fact not rely libdrm or libgbm. An implementation would require a completely new backend in smithay as an alternative to the existing drm-gbm-egl-stack.

And I am not opposed to add and support that as well, but I rather prefer another approach, that is rendering via gbm.

Rendering via gbm is not very straightforward, there is no vulkan extension that allows to create a VkSurface from a gbm-Surface. However it is desirable, a discussion similar to this one was held in the wlroots project: swaywm/wlroots#642.

To cite them:

Why would VK_KHR_display not work for this?

We're definitely are keeping GLES2 support, because it has much more hardware support than Vulkan, so the GBM/EGL stuff is sticking around. Trying to graft a VKDisplayKHR on top of all of DRM would be a complete mess; it would only be viable as a completely new backend, which I wouldn't actually be opposed to someone doing.

However, it just doesn't have all of the features we need. We can't do things like DPMS, gamma control, hotplugging(?), or many of the other nuanced things that DRM allows us to do. That's why I said I think it's for VR headsets; those kinds of features aren't important for those, but are expected from a proper display server.

Or:

On the server side, KhronosGroup/Vulkan-Docs#294 has a reasonably complete sketch of what Vulkan compositor support looks like. You can use VK_KHR_display to drive KMS for you (also handling buffer allocation through swapchains); it is easier to implement, but you lose some control over display timing, and it also means you need to composite all clients, and can't directly display clients on planes.

So basically we cannot do a lot of desirable features, if we choose to rely VK_KHR_display alone, especially not having a good way to do hot-plugging, other then frequently enumerating via vulkan is an absolute deal-breaker. Not mentioning planes, DPMS and gamma control.

That said, if you wanted to implement a backend based on VK_KHR_display, I have no objections, we can have multiple vulkan backends.

The approach I am favoring is also described in the linked wlroots issue:

The other approach is to use GBM as a buffer allocator, wrap those buffers in VkImages, render to them, extract the dma-fence FD from the rendering's signal VkSemaphore, and pass that into KMS with IN_FENCE_FD, whilst continuing to drive KMS like you do now. IN_FENCE_FD does require atomic; not sure if you have that.

So an implementation using this approach would still rely on our DrmDevice/Surface and GbmDevice/Surface code, but would wrap the GbmSurface into a VulkanSurface - I suppose - that would expose a VkImage to render into, which is mapped to the device directly using gbm, which in turn means this is also an approach using direct display rendering, although it does not appear to be at first glance.

However this does require atomic-modesetting support for the timing stuff via VkSemaphore, a feature we are currently lacking and is blocked on the drm-rs rewrite: Smithay/drm-rs#25.

Good news is, that I am working on that issue as well, because it also blocks plane-support and therefor nvidia-eglstream-support.

And does Vulkan have some native mechanisms for importing dmabufs from clients?

Yeah vulkan can import dmabufs.

I have not much experience with vulkan itself, this is mostly information I gathered by reading about it a lot, but I hope this is still helpful.

[cheako]

cheako@798ae3f

I've got to the point with winit.rs where the code will be shared with ?udev.rs? Where should I put the ten or twenty pages of common code? I already have some of it create_swapchain().

From https://gitlab.com/dennis-hamester/dacite/blob/master/examples/triangle/src/main.rs#L207 to 582.

[cheako]

I created a file called vulkan_init.rs.

I can't find where GliumDrawer??init() is called from?

[skyne98]

@cheako, nice to hear that someone is interested in writing the Vulkan support right now! Nice! Also, I wanted to mention, that I have opened an issue about GFX-hal because supporting it, even in the most inefficient way, in the beginning, will allow us to start using GFX-hal libraries (such as "rendy") to build up the compositors. Then, by the time a Vulkan backend for GFX-hal will be supported by Smithay, we will have code that can make full use of it.

[Drakulix]

I created a file called vulkan_init.rs.

I can't find where GliumDrawer??init() is called from?

https:/Smithay/smithay/blob/master/anvil/src/udev.rs#L376

GliumDrawer::init is called for each initialized monitor. So that will probably need to be changed in your example.

[cheako]

This comment is related
b950714#diff-de2d590aa0ea06ee5dc1ac59876c690cR30

[cheako]

I propose that the winit Window Resize and HiDpiChange events be disabled. Instead of handled by the winit Backend, because that might not be possible with Vulkan. As a replacement I propose a new API for these events to be sent from a Smithay Client to the Drawing Backends.
b950714#commitcomment-32901728

[elinorbgr]

We need to be careful with how we handle Resize and HiDPIChanged events from winit, because winit does assume in its behavior that they are respected. Simply ignoring them will likely result in weird and surprising behavior.

The issue if that resizing the canvas requires destroying recreating the vulkan swapchain, which is an expensive operation, do I understand correctly?

[cheako]

I just read https://docs.rs/winit/0.18.1/winit/struct.WindowBuilder.html#method.with_resizable and I understand the problem.

It's about synchronization of ImageView objects with Smithay Clients, not performance or even ownership. Every-time the Window Resizes we need to create an array of new ImageView objects, typically 3 sometimes 2. These objects are used as targets for drawing operations. Currently I'd have the Smithay Clients perform this operation in Vulkan. This requires the event(or the created ImageView array) to be passed to the Client and that adds an API interface that's only used when running under winit.

** when running under Vulkan at least **
As winit is mainly used for testing I think it would be ok to disable resize and *exit/crash when the events are triggered. * Given the alternative ignoring the events sounds better.

[elinorbgr]

Currently I'd have the Smithay Clients perform this operation in Vulkan.

Wait, I feel like I'm missing some piece of the picture here. My understanding is that hte forwarding of surface contents from client to the wayland server is done via wl_shm, wl_drm or zwp_linux_dmabuf. Both of which involve the client forwarding an FD to the server, which then reads the contents from it.

From what you described, it seems as if this process if different with Vulkan clients, where the server has to send ressources to the client to draw on instead... ?

[cheako]

Smithay Clients refer to the code the calls into the Smithay API.

[elinorbgr]

Ah right. Sorry for the misunderstanding.

Well, the winit backend already has a specific API to handle events that are specific to it, via the WinitEventsHandler trait. It'd guess the resized callback already contains all the necessary information, to process the resizing, if smithay must leave it to the compositor using it, no?

[cheako]

I don't think it would be wise to have winit specific parts to the Smithay API. This is because winit is a niche part of what Smithay does, it's only really use by developers. Winit shouldn't be a focus.

[elinorbgr]

Smithay already has some support of winit to be runnable as wayland or x11 client, as is quite helpful for debugging.

Now, that does not mean the winit backend is the focus, and I'd be perfectly okay with the vulkan backend explicitly not supporting the winit window to be resized or the dpi to change. I mean, that's what weston does for example.

It's just that I don't think there is any need to decide right now that "we should always ignore these events", for example. There's no saying we can't find a way to handle this more gracefully sometime in the future.

[skyne98]

Hey everyone, any news?

[cheako]

I've been working on it and making progress. Today I wrote some traits for output surfaces, but right now I'm on a bus at Kennedy space center after the falcon heavy launch.

I don't think this will turn out to be much of a backend as I don't see where Smithay would be making vulkan calls, I more see Vulkan support as an API extention that would enable an arbitrary drawer... but who knows someone may see where Vulkan and Smithay intersect.

[skyne98]

@cheako, I am not an expert at all in this field, but, what about using Vulkan as a headless renderer copying the results of it rendering into EGL context for final display? Can that have any benefits?

I already see that copying can introduce overhead that will destroy the purpose, as well as multi-GPU support and ability to choose GPUs will be more or less fake, because in final stage EGL will use whatever it wants to use.

Any thoughts?

[cheako]

@skyne98 I don't know anything about EGL and I'm not interested.

I try to make regular pushes to https:/cheako/smithay/tree/vkwayland

[skyne98]

@cheako, anyways, vulkan-based rendering is still infinitely exciting. As far as I understand you are porting your code from another project to smithay at the moment?

[cheako]

@skyne98 The main reason is it's what vulkan-malloc(based on AMD's Vulkan Memory Allocator (VMA)) uses. I'm not aware of what the allocation looks like under vulkano.

Secondary reason is that vulkano flies in the face of one of Vulkan's features by implementing run-time checks for issues that should be impossible if applications are written correctly. Vulkan has a built in solution for validating proper usage of the API.

[skyne98]

@cheako, what about Ash? It is used by GFX-rs and using Ash might help a lot in making a backend for GFX in the future. This will allow building on top of existing ecosystem surrounding GFX's abstract graphics implementation (e.g. Rendy).

Also there is an issue I have previously opened where we came to a conclusion that implementing a basic OpenGL backend would be the best option. However, GFX-rs itself is very close to vulkan and its vulkan bindings are nearly 1-to-1. So, I think using ash might help with it.

[cheako]

https://travis-ci.org/cheako/smithay/jobs/521528533

I worked out all the errors, though there are still functions missing, it would be a good time to look at my learning rust project and point out anything I've done wrong or areas that can be improved.

Thanks.

[cheako]

@skyne98 I don't think there will be any Vulkan calls from Smithay, I just don't see where it's appropriate. Given that any drawing API should be able to make use of the solution made for dacite.

[Drakulix]

@cheako I will try to take a look at it this weekend.

[skyne98]

@cheako can you maybe explain it in more details?

[elinorbgr]

@skyne98 I don't think there will be any Vulkan calls from Smithay, I just don't see where it's appropriate. Given that any drawing API should be able to make use of the solution made for dacite.

I'd like to nuance this a little though.

• As explained earlier by @Drakulix, in the long-run we'll need a GBM-based vulkan stack in order to provide certain features wrt to DRM control that are not exposed by VK_KHR_display (DPMS, gamma-control, planes, monitor hotplugging, ...)
• There is the question of importing client DMA-bufs into the OpenGL/Vulkan context, so Smithay will need at least some awareness of the VUlkan context to handle that.

In this vein, the long-term run will likely look like what we currently do for OpenGL: Smithay initializes a Vulkan context, and handles the link between it and GBM/DRM and clients DMA-bufs, and then lets you use whatever API you want to draw on it (gfx, vulkano, you name it). It only requires the API to support some backend-plugging mechanism.

For example for OpenGL glium can handle any OpenGL-context provider as long as it provides an implementation of the glium::backend::Backend trait, which Smithay does.

The plan discussed in #129 is to ensure gfx-rs provides a similar backend-plugin mechanism, so that it could be used as a drawing API on top of the context provided by Smithay.

[skyne98]

@vberger thanks for a detailed explanation! I will try to find some time myself to work on #129 in the next couple of days.

Also, yes, I think that the plan in #129 was to provide a new OpenGL backend for GFX-rs based on Smithay's own OpenGL stack. However, doesn't that mean that the same thing would have to be done if we wanted to provide a Smithay-based Vulkan backend for it?

My consideration is that providing a new Vulkan backend for gfx-rs can be much easier as it nearly directly follows the Vulkan APIs.

[elinorbgr]

Also, yes, I think that the plan in #129 was to provide a new OpenGL backend for GFX-rs based on Smithay's own OpenGL stack. However, doesn't that mean that the same thing would have to be done if we wanted to provide a Smithay-based Vulkan backend for it?

From smithay point of view, as long as the API supports plugging a backend in a reasonnable way, this is not a lot of work. See for example the compatibility code for glium, its pretty small : https:/Smithay/smithay/blob/master/src/backend/graphics/glium.rs

Now, if the API does not support plugging a backend, that is an other question. I'll move the details to #129 though.

[cheako]

TLDR; I think there is enough reason to justify performing the buffer handling(the reading of the DMABUF and shm) in the Smithay Client(the code that calls into the Smithay API). To support this there should be a callback for surface commit requests and an API call to send multiple buffer release events.

I'm still doing a lot of learning! When a DMABUF surface receives a commit event it's tempting to want to clone the image in Smithay and release the buffer right away, but that might work against performance as it causes Smithay to block on a Vulkan Fence. My CPP vkwayland does this, to match the shm interface, and it's an obvious case for improvement.

Even for shm I don't think it's a good idea to have Smithay do the Vulkan calls to create an image as there are a number of ways to accomplish this and a good compositor will have fast paths on some hardware. Given that, it's likely simpler to have the Smithay Client perform the memcpy(or whatever) as well.

What I did in CPP was to have all the values collected passed to commit(), overloaded for DMABUF, and handled the copy operations in the renderer. For shm this caused an immediate release of the buffer, but for DMABUF that release should be delayed until there are all the DMABUFs for the next frame.

Edit: Perhaps the implementation links would have been a better choice.
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/render/surface.cpp#lines-73
https://bitbucket.org/cheako/vkwayland/src/ab2ef75c73b142c716c529030836ca9188701ac3/src/render/surface.cpp#lines-283

Given a chance to do-over I'd also pass a wl_buffer pointer to the commit() callback and collect those with a singleton_wayland.release() that takes a vector. Knowing that for shm it'll most likely result in a call to release() with a single wl_buffer during the commit() callback, but as long as it's simple to support a deferred memcpy() why not.

I understand the reason to want to release the buffer, but I don't think it's necessary as long as the release is done for every frame.

[elinorbgr]

To support this there should be a callback for surface commit requests and an API call to send multiple buffer release events.

The current API allows you to rather access the wl_buffer via the CompositorToken::with_surface_data method, from this you can attempt to access its SHM contents via the shm::with_buffer_contents method, which gives you access to a slice into the memmaped region, so no copy is done. The helpers for the wp_dmabuf protocol will have a similar API, giving you access to the dmabuf FD and its metadata. Similarly, you can release a buffer by just calling buffer.release().

I never intended to say that Smithay should always handle to import & release of the buffers itself, but it is in its scope to provide optional helpers to handle this for compositor writers that want to use them. Also, if one day Smithay gains some code to handle rendering itself like wlroots does, this would then be built on these helpers.

[cheako]

@vberger There is still a lot I don't know.

I'll think about how Smithay could export helpers for reading shm/DMABUF aimed at Vulkan renders, currently I think the API would be burdened with the handful of implementations that Vulkan affords. Off the top of my head these would be broken-down into:

shm:
a. Manage CPU buffer allocation and memcpy into said buffer. This affords a quick Buffer Release and AFAIK is the only option for Vulkan to handle shm.
Done every time the image size or format changes, including the first time obviously(122-195). Then the copy(205-216). Note that implementation (b) is setup at (255-285).

It's possible to choose the best options here for most cases, but there are different options that may work better on some hardware combinations. I don't have access to all hardware so I can't even test my implementation, it's just my best guess.

b. (a plus) Manage Image allocation(presumably with GPU storage, instead of CPU) and blit the CPU buffer over. Ideally it's better to batch these up and combine them with the rest of the frame drawing operations, not sure how that would look as an API.
Here is where I begin the command buffer for each frame(145-147). Then I write-out commands to copy the shm buffers into the previously created Images(textures) (151-209).

Where we get into trouble trying to provide a simple API is the Memory Barrier(s), considered expensive.
Ideally there would be a single set for all images, including the images Smithay would ordinarily have no knowledge of.

Conclusion:
A total of 2 options were each has a number of parameters that could vary from hardware to hardware.

DMABUF:
a. Manage importing of the DMABUF as an Image(344-420). Notice this does not involve reading the data.

I could see a helper written for this, maybe Smithay just supplies a usable MemoryAllocateInfo{} so that the Smithay Client can make alterations(also allowing for *non-Vulkan APIs to use this interface).

• Perhaps it's best to just supply the information to the Smithay Client and provide a copy/paste example of how to build the Vulkan specific structures. That's Basically what I've suggested.

b. Manage Image for Xfer destination(421-478).

There are options for how to provision the Image, again just my best guess. Though admittedly we don't have to bother with CPU visibility or cache coherence so it's orders of magnitude simpler than the Buffer we needed to memcpy() into previously. Also applies to shm b.

c. (a&b plus) Copy the data(517-577).

This is where things can be optimized, from what I've written in CPP, notice the PipelineStageFlagBits match those used against m_oustandingBuffers.

Ideally the Device.m_commandBuffer would be split into two, separating the Xfers and the Drawing. This shouldn't be too bad because the drawing is held up by the barrier with *PipelineStageFlagBits::eFragmentShader anyway. This would allow for a Fence to signal the CPU when the Xfer is done and it's safe to Release DMABUF Buffers.

• This is where Smithay Clients can have an issue with how Smithay handles the copying. If for some reason there is a lot to do in VertexShader(s), having one Command Buffer for both the Xfers and the Drawing could speed things up.

d. (a plus) Don't copy, just use the provided DMABUF directly.
This holds up the Wayland Buffer Release Events, but it could be great for performance overall. The API for this would *require having the Smithay Client informing Smithay when it's done with the buffer(s), so it's one possibility provided the interface I've suggested.

• Unless Smithay does the Drawing as well.

Conclusion:
A total of 4 options.

Overall:
Keeping in mind the Wayland Protocol does not forbid migrating a Surface between DMABUF and shm variants, there are a total of 8 APIs to expose for Wayland Surface(s) and their Buffer(s). None are clear winners, as doing a lot causes compromise and doing a little offers no real benefit for Smithay Clients.

After consideration for all the options I can see, I still think it's best to push the data necessary on Surface Commit and export a call for Buffer Release. The code needed outside of this suggestion is trivial and there is not one good way for Smithay to simplify things for Smithay Clients.

[cheako]

@vberger
What I was thinking is that the best performance comes from *setting-up the reading for DMABUF pushed(as opposed to polling for updated data) by Wayland Surface Commit Requests. What could slow things down is for Smithay to block until the reading is complete. The Wayland Clients(literally) could, especially in the case of only 2 buffers, be blocked waiting for Wayland Buffer Release Events if these are not pushed as well.

• In Vulkan there is a little busy work to perform now that we finally have hard numbers on the width, height, and format of the buffer.

[Drakulix]

I worked out all the errors, though there are still functions missing, it would be a good time to look at my learning rust project and point out anything I've done wrong or areas that can be improved.

Thanks.

So from a code-standpoint this looks mostly fine. Obviously a lot needs to be cleaned up (e.g. unused udev/drm code in udev.rs). I also do strongly prefer to integrate the vulkan solution into anvil instead of forking it. I do absolutely understand, why you did it this way for experimentation, but now you have everything figured out, we need to think about merging it.

Anvil's code base is in a reasonable, but bad state. @vberger and myself have been talking about redesigning anvil to be a better compositor again and again, maybe this is a good opportunity. Essentially we want a minimal, but usable compositor, that utilizes every feature smithay has to offer, including vulkan once that is done. Right now anvil is using so many hacks, you cannot call it usable. It does not need to be the most efficient compositor, but it should perform reasonable well. Much more important however is making anvil more readable and better documented, so I can serve as a real reference compositor for newbies.

Adding vulkan support on top of the current code base will work against that goal, at least for the winit-backend. (anvil's udev backends and vkwayland's vkdisplay backend do not share enough code to be merged right now, but this will change, when we support a GBM-based pipeline.)

@skyne98 I don't think there will be any Vulkan calls from Smithay, I just don't see where it's appropriate. Given that any drawing API should be able to make use of the solution made for dacite.

The use case is clear, we want to provide basic functionality to get started writing your own compositor easily. On the long run a lot of compositors will add their own rendering code, but I think there is room for vulkan-helpers.

• As explained earlier by @Drakulix, in the long-run we'll need a GBM-based vulkan stack in order to provide certain features wrt to DRM control that are not exposed by VK_KHR_display (DPMS, gamma-control, planes, monitor hotplugging, ...)

Yes, but I think this support needs to live along the VK_KHR_DISPLAY support. We can support both.

• There is the question of importing client DMA-bufs into the OpenGL/Vulkan context, so Smithay will need at least some awareness of the VUlkan context to handle that.

In this vein, the long-term run will likely look like what we currently do for OpenGL: Smithay initializes a Vulkan context, and handles the link between it and GBM/DRM and clients DMA-bufs, and then lets you use whatever API you want to draw on it (gfx, vulkano, you name it). It only requires the API to support some backend-plugging mechanism.

This is more difficult, as it is for OpenGL. The main reason for this, is that there is not unified low-level function loader, as there is for OpenGL. Basically every Rust-GL code uses gl-rs. Either directly or through gfx, glium, etc.

For vulkan there a several libraries and every library uses another loader. But luckily every library exposes the underlying object somehow.

Library	Loader	Native Trait	Supports casting to native-obj	Supports casting from native-obj	Supports VK_KHR_DISPLAY out-of-the-box
ash	ash-sys (TBD)	ash::vk::Handle	yes	yes	no
vulkano	vk-sys	vulkano::VulkanObject	yes	sometimes	yes (but untested)
dacite	vks	dacite::VulkanObject	yes	yes	yes

So the only slightly problematic library here is vulkano being picky about foreign objects, but I think the most critical ones are supported.

What that means is, we could provide helper functions to enumerate and create surfaces for different screens, which I would like to see in smithay. Input variables could be converted into native types for any given vulkan-library, output variables could be converted back to the original library and internally we can use any library supporting VK_KHR_DISPLAY. dacite might be a good choice here.

TLDR; I think there is enough reason to justify performing the buffer handling(the reading of the DMABUF and shm) in the Smithay Client(the code that calls into the Smithay API). To support this there should be a callback for surface commit requests and an API call to send multiple buffer release events.

Some clients might handle buffers this way. But buffer handling is difficult, I would like to have reasonable, although maybe not the fastest possible, solution to this problem in smithay. This could be a similar helper function and get smithay-users on track faster. This will not be as straight forward as with egl/opengl for sure (here you want to convert to a texture in 99% of all cases).

I never intended to say that Smithay should always handle to import & release of the buffers itself, but it is in its scope to provide optional helpers to handle this for compositor writers that want to use them. Also, if one day Smithay gains some code to handle rendering itself like wlroots does, this would then be built on these helpers.

And eventually I want to see this code happening.

Of course our first vulkan implementation will not get everything right and might be cumbersome to use or inefficient (or both), but that should not stop us. This was the same for the first drm-backend implementations and egl-implementations. Right now those helpers are quite helpful and in a good state. Just because it is easier to off load this to the client, I would not shy away from this problem.

[cheako]

I think this is related to the explicit-fencing protocol, which would fix this issue?

Yes, that sounds great. This is one question I had about wayland... Wouldn't the community want to eventually remove shm and force all clients to use DMABUF? The same with this synchronization, having the Compositor watch a Fence and then fire off an event(over IPC) when it signals sounds awful when compared to exporting the Fence to the client.

[Drakulix]

Yes, that sounds great. This is one question I had about wayland... Wouldn't the community want to eventually remove shm and force all clients to use DMABUF? The same with this synchronization, having the Compositor watch a Fence and then fire off an event(over IPC) when it signals sounds awful when compared to exporting the Fence to the client.

DMABUFs would be absolutely be preferred for most clients. But shm will never be deprecated, because wayland is not linux-only and even using linux does not mean that you have access to GPU-acceleration.

E.g. there is a linux-port on the 3DS running weston.

Wayland is just a protocol and shared-buffer rendering is the minimum viable target for every platform. Which means even if you will might never see shm-clients on most desktops in the future, this will very much continue to exists somewhere.

[elinorbgr]

Also, shm buffers can be used to draw small UI elements like tooltips or window decorations. This is for example how SCTK draws its decorations.

Generally, not every program needs to use GPU rendering, especially for rarely-updated UI parts.

[cheako]

After looking at EGL for the past few days I've come to the conclusion that I will need to duplicate https://bitbucket.org/cheako/vkwayland/src/master/src/wayland/linux_dmabuf.cpp

Should I create a VulkanBackend object much like the EGLBackend object or should I place this elsewhere?

What I'm looking to do is not at all like what EGL does.

I can't see where zwp_linux_dmabuf_v1_interface would need a callback as it's just where the wlbuffer is created, nothing noteworthy happens until the surfaces commit requests comes in and there is already a callback for that.

I just intend to collect the function parameters into a struct and then somehow feed that into a wlbuffer that can then be queried for the wayland request parameters.

I don't know if I agree with the, here is a WlBuffer u work out it's type on your own, but it should work doing it that way.

[elinorbgr]

To follow the way smithay is organized, we plan to add handlers in the smithay::wayland module to handle the linux-dmabuf and linux-explicit-synchronization protocol extensions, I'm actually currently working on that.

Like the other handlers, these would only handle the "wayland protocol" part of the question, and provide an interface to obtain the buffers and their details, pretty similarly to what the SHM handler does.

[cheako]

Thanks, I'll only focus on shm for now then... I can't do much with dmabuf anyway as I'd rather wait for dacite to support them than bring in another API.

[Drakulix]

After looking at EGL for the past few days I've come to the conclusion that I will need to duplicate https://bitbucket.org/cheako/vkwayland/src/master/src/wayland/linux_dmabuf.cpp

Should I create a VulkanBackend object much like the EGLBackend object or should I place this elsewhere?

The EGLBackend stuff is mostly used by EglDrmDevice and EglDrmSurface.

What I'm looking to do is not at all like what EGL does.

And it should not be. I would currently (given my limited understanding) expect something like a VulkanDevice, which you may use to create VulkanSurfaces to use with any of the three mentioned frameworks.

No code would be shared with either the DRM or the EGL code.

Thanks, I'll only focus on shm for now then... I can't do much with dmabuf anyway as I'd rather wait for dacite to support them than bring in another API.

This is a good idea for now. Once @vberger has actually implemented the wayland side of the protocols and we tested them with the old OpenGL backends, we can get back to this issue.

[Ella-0]

I've been looking into this possibility and built a very very simple compositor based on smithay/anvil and vulkano as a proof of concept. Currently it doesn't work very well for things like weston-simple-shm but alacritty seams to work fine as shown below.

I'd be willing to share my code if it would be deemed useful.

[Ella-0]

I can also verify it works with VK_KHR_display though it's a bit difficult to take a screenshot of that. Works on both intel mesa and proprietary Nvidia drivers

[Ella-0]

I might work on a pull request when I sort out all of the issues I've been having with the NVIDIA driver and vulkano. It works perfectly on Intel GPUs.

[skyne98]

That's amazing, keep up the good work! I would also assume it would work on AMD if it worked on Intel drivers as both of them are open-source.

[Ella-0]

Yes, I assume it would too, though it may not if there are driver bugs.

[skyne98]

Would actually be interesting to be able to hook it up to some existing rendering, UI or even game libraries with Vulkan rendering support. In my mind it can greatly help by giving a possibility to focus on improving user experience rather than fiddling around in the technical details.