blog.gtk.orgGTK Development Blog – All things GTK

GTK Development Blog All things GTK Graphics offload revisited We first introduced support for dmabufs and graphics offload last fall, and it is included in GTK 4.14. Since then, some improvements have happened, so it is time for an update. Improvements down the stack The GStreamer 1.24 release has improved support for explicit modifiers, and the GStreamer media backend in GTK has been updated to request dmabufs from GStreamer. Another thing that happens on the GStreamer side is that dmabufs sometimes come with padding: in that case GStreamer will give us a buffer with a viewport and expect us to only show that part of the buffer. This is sometimes necessary to accommodate stride and size requirements of hardware decoders. GTK 4.14 supports this when offloading, and only shows the part of the dmabuf indicated by the viewport. Improvements inside GTK We’ve merged new GSK renderers for GTK 4.14. The new renderers support dmabufs in the same way as the old gl renderer. In addition, the new Vulkan renderer produces dmabufs when rendering to a texture. In GTK 4.16, the GtkGLArea widget will also provide dmabuf textures if it can, so you can put it in a GtkGraphicsOffload widget to send its output directly to the compositor. You can see this in action in the shadertoy demo in gtk4-demo in git main. Shadertoy demo with golden outline around offloaded graphics Improved compositor interaction One nice thing about graphics offload is that the compositor may be able to pass the dmabuf to the KMS apis of the kernel without any extra copies or compositing. This is known as direct scanout and it helps reduce power consumption since large parts of the GPU aren’t used. The compositor can only do this if the dmabuf is attached to a fullscreen surface and has the right dimensions to cover it fully. If it does not cover it fully, the compositor needs some assurance that it is ok to leave the outside parts black. One way for clients to provide that assurance is to attach a specially constructed black buffer to a surface below the one that has the dmabuf attached. GSK will do this now if it finds black color node in the rendernode tree, and the GtkGraphicsOffload widget will put that color there if you set the black-background” property. This should greatly increase the chances that you can enjoy the benefits of direct scanout when playing fullscreen video. Offloaded content with fullscreen black background In implementing this for GTK 4.16, we found some issues with mutter’s support for single-pixel buffers, but these have been fixed quickly. To see graphics offload and direct scanout in action in a GTK4 video player, you can try the Light Video Player . If you want to find out if graphics offload works on your system or debug why it doesn’t, this recent post by Benjamin is very helpful. Summary GTK 4 continues to improve for efficient video playback and drives improvements in this area up and down the stack. A big thank you for pushing all of this forward goes to Robert Mader. ❤️ Author mclasen Posted on April 17, 2024 April 17, 2024 Categories Uncategorized Accessibility improvements in GTK 4.14 GTK 4.14 brings various improvements on the accessibility front, especially for applications showing complex, formatted text; for WebKitGTK; and for notifications. Accessible text interface The accessibility rewrite for 4.0 provided an implementation for complex, selectable, and formatted text in widgets provided by GTK, like GtkTextView , but out of tree widgets would not be able to do the same, as the API was kept private while we discussed what ATs (assistive technologies) actually needed, and while we were looking at non-Linux implementations. For GTK 4.14 we finally have a public interface that out of tree widgets can implement to provide complex, formatted text to ATs: GtkAccessibleText . GtkAccessibleText allows widgets to provide the text contents at given offsets; the text attributes applied to the contents; and to notify assistive technologies of changes in the text, caret position, or selection boundaries. Text widgets implementing GtkAccessibleText should notify ATs in these cases: if the text widget has a caret cursor , it needs to call gtk_accessible_text_update_caret_position() every time the caret moves if the text widget has a selection, it needs to call gtk_accessible_text_update_selection_bound() every time the selection changes when the text changes, the widget needs to call gtk_accessible_text_update_contents() with the description of what changed, and the boundaries of the change Text attributes are mainly left to applications to implement—both in naming and serialization; GTK provides support for common text attributes already in use by various toolkits and assistive technologies, and they are available as constants under the GTK_ACCESSIBLE_ATTRIBUTE_* prefix in the API reference . The GtkAccessibleText interface is a requirement for implementing the accessibility of virtual terminals; the most common GTK-based library for virtual terminals, VTE , has been ported to GTK4 thanks to the efforts of Christian Hergert and in GNOME 46 will support accessibility through the new GTK interface. Bridging AT-SPI trees There are cases when a library or an application implements its own accessible tree using AT-SPI, whether in the same process or out of process. One such library is WebKitGTK, which generates the accessible object tree from the web tree inside separate processes. These processes do not use GTK, so they cannot use the GtkAccessible API to describe their contents. Thanks to the work of Georges Stavracas GTK now can bridge those accessibility object trees under the GTK widget’s own, allowing ATs to navigate into a web page using WebKit from the UI. Currently, like the rest of the accessibility API in GTK, this is specific to the AT-SPI protocol on Linux, which means it requires libraries and applications that wish to take advantage of it to ensure that the API is available at compile time, through the use of a pkg-config file and a separate C header, similarly to how the printing API is exposed. Notifications Applications using in-app notifications that are decoupled by the current widget’s focus, like AdwToast in libadwaita, can now raise the notification message to ATs via the gtk_accessible_announce() method, thanks to Lukáš Tyrychtr, in a way that is respectful of the current ATs output. Other improvements GTK 4.12 ensured that the computed accessible labels and descriptions were up to date with the ARIA specification ; GTK 4.14 iterates on those improvements, by removing special cases and duplicates. Thanks to the work of Michael Weghorn from The Document Foundation , there are new roles for text-related accessible objects, like paragraphs and comments, as well as various fixes in the AT-SPI implementation of the accessibility API. The accessibility support in GTK4 is incrementally improving with every cycle, thanks to the contributions of many people; ideally, these improvements should also lead to a better, more efficient protocol for toolkits and assistive technologies to share. We are still exploring the possibility of adding backends for other accessibility platforms, like UIAutomation; and for other libraries, like AccessKit . Author Emmanuele Bassi Posted on March 8, 2024 March 7, 2024 Categories Uncategorized On fractional scales, fonts and hinting GTK 4.14 will be released very soon, with new renderers that were introduced earlier this year. The new renderers have much improved support for fractional scaling—on my system, I now use 125% scaling instead of the ‘Large Text’ setting, and I find that works fine for my needs. Magical numbers Ever since 4.0, GTK has been advocating for linear layout. The idea is that we just place glyphs where the coordinates tell us, and if that is a fractional position somewhere between pixels, so be it, we can render the outline at that offset just fine. This approach works—if your output...