Quantcast
Jump to content

New Game Changing Vulkan Extensions for Mobile: Descriptor Indexing


STF News
 Share

Recommended Posts

2021-06-28-01-banner.jpg

The Samsung Developers team works with many companies in the mobile and gaming ecosystems. We're excited to support our partner, Arm, as they bring timely and relevant content to developers looking to build games and high-performance experiences. This Vulkan Extensions series will help developers get the most out of the new and game-changing Vulkan extensions on Samsung mobile devices.

As I mentioned previously, Android is enabling a host of useful new Vulkan extensions for mobile. These new extensions are set to improve the state of graphics APIs for modern applications, enabling new use cases and changing how developers can design graphics renderers going forward. These extensions will be available across various Android smartphones, including the new Samsung Galaxy S21, which was recently launched on 14 January. Existing Samsung Galaxy S models, such as the Samsung Galaxy S20, also allow upgrades to Android R.

I have already discussed two of these extensions in previous blogs - Maintenance Extensions and Legacy Support Extensions. However, there are three further Vulkan extensions for Android that I believe are ‘game changers’. In the first of three blogs, I will explore these individual game changer extensions – what they do, why they can be useful and how to use them. The goal here is to not provide complete samples, but there should be enough to get you started. The first Vulkan extension is ‘Descriptor Indexing.’ Descriptor indexing can be available in handsets prior to Android R release. To check what Android devices are available with 'Descriptor Indexing' check here. You can also directly view the Khronos Group/ Vulkan samples that are relevant to this blog here.

VK_EXT_descriptor_indexing

Introduction

In recent years, we have seen graphics APIs greatly evolve in their resource binding flexibility. All modern graphics APIs now have some answer to how we can access a large swathes of resources in a shader.

Bindless

A common buzzword that is thrown around in modern rendering tech is “bindless”. The core philosophy is that resources like textures and buffers are accessed through simple indices or pointers, and not singular “resource bindings”. To pass down resources to our shaders, we do not really bind them like in the graphics APIs of old. Simply write a descriptor to some memory and a shader can come in and read it later. This means the API machinery to drive this is kept to a minimum.

This is a fundamental shift away from the older style where our rendering loop looked something like:

render_scene() {
    foreach(drawable) {
        command_buffer->update_descriptors(drawable);
        command_buffer->draw();
    }
}

Now it looks more like:

render_scene() {
    command_buffer->bind_large_descriptor_heap();
    large_descriptor_heap->write_global_descriptors(scene, lighting, shadowmaps);
    foreach(drawable) {
        offset = large_descriptor_heap->allocate_and_write_descriptors(drawable);  
        command_buffer->push_descriptor_heap_offsets(offset);
        command_buffer->draw();
    }
}

Since we have free-form access to resources now, it is much simpler to take advantage of features like multi-draw or other GPU driven approaches. We no longer require the CPU to rebind descriptor sets between draw calls like we used to.

Going forward when we look at ray-tracing, this style of design is going to be mandatory since shooting a ray means we can hit anything, so all descriptors are potentially used. It is useful to start thinking about designing for this pattern going forward.

The other side of the coin with this feature is that it is easier to shoot yourself in the foot. It is easy to access the wrong resource, but as I will get to later, there are tools available to help you along the way.

VK_EXT_descriptor_indexing features

This extension is a large one and landed in Vulkan 1.2 as a core feature. To enable bindless algorithms, there are two major features exposed by this extension.

Non-uniform indexing of resources

How resources are accessed has evolved quite a lot over the years. Hardware capabilities used to be quite limited, with a tiny bank of descriptors being visible to shaders at any one time. In more modern hardware however, shaders can access descriptors freely from memory and the limits are somewhat theoretical.

Constant indexing

Arrays of resources have been with us for a long time, but mostly as syntactic sugar, where we can only index into arrays with a constant index. This is equivalent to not using arrays at all from a compiler point of view.

layout(set = 0, binding = 0) uniform sampler2D Textures[4];
const int CONSTANT_VALUE = 2;
color = texture(Textures[CONSTANT_VALUE], UV);

HLSL in D3D11 has this restriction as well, but it has been more relaxed about it, since it only requires that the index is constant after optimization passes are run.

Dynamic indexing

As an optional feature, dynamic indexing allows applications to perform dynamic indexing into arrays of resources. This allows for a very restricted form of bindless. Outside compute shaders however, using this feature correctly is quite awkward, due to the requirement of the resource index being dynamically uniform.

Dynamically uniform is a somewhat intricate subject, and the details are left to the accompanying sample in KhronosGroup/Vulkan-Samples.

Non-uniform indexing

Most hardware assumes that the resource index is dynamically uniform, as this has been the restriction in APIs for a long time. If you are not accessing resources with a dynamically uniform index, you must notify the compiler of your intent.

The rationale here is that hardware is optimized for dynamically uniform (or subgroup uniform) indices, so there is often an internal loop emitted by either compiler or hardware to handle every unique index that is used. This means performance tends to depend a bit on how divergent resource indices are.

#extension GL_EXT_nonuniform_qualifier : require
layout(set = 0, binding = 0) uniform texture2D Tex[];
layout(set = 1, binding = 0) uniform sampler Sampler;
color = texture(nonuniformEXT(sampler2D(Tex[index], Sampler)), UV);

In HLSL, there is a similar mechanism where you use NonUniformResourceIndex, for example.

Texture2D<float4> Textures[] : register(t0, space0);
SamplerState Samp : register(s0, space0);
float4 color = Textures[NonUniformResourceIndex(index)].Sample(Samp, UV);

All descriptor types can make use of this feature, not just textures, which is quite handy! The nonuniformEXT qualifier removes the requirement to use dynamically uniform indices. See the code sample for more detail.

Update-after-bind

A key component to make the bindless style work is that we do not have to … bind descriptor sets all the time. With the update-after-bind feature, we effectively block the driver from consuming descriptors at command recording time, which gives a lot of flexibility back to the application. The shader consumes descriptors as they are used and the application can freely update descriptors, even from multiple threads.

To enable, update-after-bind we modify the VkDescriptorSetLayout by adding new binding flags. The way to do this is somewhat verbose, but at least update-after-bind is something that is generally used for just one or two descriptor set layouts throughout most applications:

VkDescriptorSetLayoutCreateInfo info = { … };
info.flags = VK_DESCRIPTOR_SET_LAYOUT_CREATE_UPDATE_AFTER_BIND_POOL_BIT_EXT;
const VkDescriptorBindingFlagsEXT flags =
    VK_DESCRIPTOR_BINDING_VARIABLE_DESCRIPTOR_COUNT_BIT_EXT |    
    VK_DESCRIPTOR_BINDING_PARTIALLY_BOUND_BIT_EXT |
    VK_DESCRIPTOR_BINDING_UPDATE_AFTER_BIND_BIT_EXT |
    VK_DESCRIPTOR_BINDING_UPDATE_UNUSED_WHILE_PENDING_BIT_EXT;
VkDescriptorSetLayoutBindingFlagsCreateInfoEXT binding_flags = { … };
binding_flags.bindingCount = info.bindingCount;
binding_flags.pBindingFlags = &flags;
info.pNext = &binding_flags;

For each pBinding entry, we have a corresponding flags field where we can specify various flags. The descriptor_indexing extension has very fine-grained support, but UPDATE_AFTER_BIND_BIT and VARIABLE_DESCRIPTOR_COUNT_BIT are the most interesting ones to discuss.

VARIABLE_DESCRIPTOR_COUNT deserves special attention as it makes descriptor management far more flexible. Having to use a fixed array size can be somewhat awkward, since in a common usage pattern with a large descriptor heap, there is no natural upper limit to how many descriptors we want to use. We could settle for some arbitrarily high limit like 500k, but that means all descriptor sets we allocate have to be of that size and all pipelines have to be tied to that specific number. This is not necessarily what we want, and VARIABLE_DESCRIPTOR_COUNT allows us to allocate just the number of descriptors we need per descriptor set. This makes it far more practical to use multiple bindless descriptor sets.

When allocating a descriptor set, we pass down the actual number of descriptors to allocate:

VkDescriptorSetVariableDescriptorCountAllocateInfoEXT variable_info = { … };
variable_info.sType =
        VK_STRUCTURE_TYPE_DESCRIPTOR_SET_VARIABLE_DESCRIPTOR_COUNT_ALLOCATE_INFO_EXT;
variable_info.descriptorSetCount = 1;
allocate_info.pNext = &variable_info;
variable_info.pDescriptorCounts = &NumDescriptorsStreaming;
VK_CHECK(vkAllocateDescriptorSets(get_device().get_handle(), &allocate_info, 
        &descriptors.descriptor_set_update_after_bind));

GPU-assisted validation and debugging

When we enter the world of descriptor indexing, there is a flipside where debugging and validation is much more difficult. The major benefit of the older binding models is that it is fairly easy for validation layers and debuggers to know what is going on. This is because the number of available resources to a shader is small and focused.

With UPDATE_AFTER_BIND in particular, we do not know anything at draw time, which makes this awkward.

It is possible to enable GPU assisted validation in the Khronos validation layers. This lets you catch issues like:

"UNASSIGNED-Descriptor uninitialized: Validation Error: [ UNASSIGNED-Descriptor uninitialized ] Object 0: handle = 0x55625acf5600, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x893513c7 | Descriptor index 67 is uninitialized__.  Command buffer (0x55625b184d60). Draw Index 0x4. Pipeline (0x520000000052). Shader Module (0x510000000051). Shader Instruction Index = 59.  Stage = Fragment.  Fragment coord (x,y) = (944.5, 0.5).  Unable to find SPIR-V OpLine for source information.  Build shader with debug info to get source information."

Or:

"UNASSIGNED-Descriptor uninitialized: Validation Error: [ UNASSIGNED-Descriptor uninitialized ] Object 0: handle = 0x55625acf5600, type = VK_OBJECT_TYPE_QUEUE; | MessageID = 0x893513c7 | Descriptor index 131 is uninitialized__.  Command buffer (0x55625b1893c0). Draw Index 0x4. Pipeline (0x520000000052). Shader Module (0x510000000051). Shader Instruction Index = 59.  Stage = Fragment.  Fragment coord (x,y) = (944.5, 0.5).  Unable to find SPIR-V OpLine for source information.  Build shader with debug info to get source information."

RenderDoc supports debugging descriptor indexing through shader instrumentation, and this allows you to inspect which resources were accessed. When you have several thousand resources bound to a pipeline, this feature is critical to make any sense of the inputs.

If we are using the update-after-bind style, we can inspect the exact resources we used.

In a non-uniform indexing style, we can inspect all unique resources we used.

Conclusion

Descriptor indexing unlocks many design possibilities in your engine and is a real game changer for modern rendering techniques. Use with care, and make sure to take advantage of all debugging tools available to you. You need them.

This blog has explored the first Vulkan extension game changer, with two more parts in this game changer blog series still to come. The next part will focus on ‘Buffer Device Address’ and how developers can use this new feature to enhance their games.

Follow Up

Thanks to Hans-Kristian Arntzen and the team at Arm for bringing this great content to the Samsung Developers community. We hope you find this information about Vulkan extensions useful for developing your upcoming mobile games. The original version of this article can be viewed at Arm Community.

The Samsung Developers site has many resources for developers looking to build for and integrate with Samsung devices and services. Stay in touch with the latest news by creating a free account or by subscribing to our monthly newsletter. Visit the Marketing Resources page for information on promoting and distributing your apps and games. Finally, our developer forum is an excellent way to stay up-to-date on all things related to the Galaxy ecosystem.

View the full blog at its source

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Similar Topics

    • By STF News
      View the full blog at its source
    • By STF News
      Samsung Electronics today announced that its 2022 QLED and Lifestyle TVs have been recognized by leading global certification institutes for eye safety and color technology. The news comes as the company announced its newest QLED and Lifestyle TVs at CES 2022.
       

       
      The 2022 Samsung Lifestyle TVs won the ‘Eye Care’ Certification from Verband Deutscher Elektrotechniker (VDE) in Germany, one of Europe’s largest technical-scientific associations with more than 36,000 members. The certification applies to Samsung’s 2022 Lifestyle TV models including The Frame, The Serif and The Sero. The screens are evaluated on various categories, including ‘Safety’, ‘Gentle to the eyes’, flicker level, uniformity and color fidelity.
       
      (From left) Seokwoo Yong, EVP and Head of R&D Team, Visual Display Business at Samsung Electronics and Cherif Kedir, President & CEO of VDE
       
      The new Lifestyle TVs were assessed for safety from blue light emission and melatonin inhibition levels based on a light hazard classification method set by the International Electrotechnical Commission (IEC). Samsung’s 2022 Lifestyle TVs satisfy the IEC’s standards for screen flickering, which can cause eye fatigue or headache for viewers. They were also recognized for excellence in color fidelity and picture quality uniformity, both elements of which contribute to eye comfort level while watching TV.
       
      (From left) Seokwoo Yong, EVP and Head of R&D Team, Visual Display Business at Samsung Electronics and Wyatt Brannan, Vice President of North America Consumer, Medical & Information Technology of UL
       
      Samsung’s 2022 Lifestyle TVs1 were also verified as ‘Glare-Free’ by Underwriters Laboratories (UL), a leading independent safety science company. UL’s verification validates the ‘Glare-Free’ claim by assessing the products against Unified Glare Rating (UGR) testing standard set by the International Commission on Illumination (CIE). Samsung’s new Lifestyle TV models use a new Matte Display with anti-glare, anti-reflection and anti-fingerprint properties to deliver the optimal brightness and provide the best picture quality without glare.
       
      Reflected Glare, which determines whether the objects on a TV screen are visible even when external light is reflected on the surface Discomfort Glare, which determines whether a TV screen is too bright Disability Glare, which determines whether a TV screen is overly bright when watching TV in a dark room  
      These glare assessments were calculated based on test results of watching TV in both 300 lux, which is equivalent to a brightly lit work area, and in 150 lux, which is usually the value for a dimly lit work area.
       
      (From left) Seokwoo Yong, EVP and Head of R&D Team, Visual Display Business at Samsung Electronics and Raj Shah, Vice President at Marketing of Pantone
       
      Additionally, Samsung’s all new 2022 QLED models received the world’s first ‘Pantone Validated’ certification from Pantone, the world-famous brand in the global color industry and creator of the Pantone Matching System (PMS).
       
      These models receiving this recognition from Pantone include all 20 newly released models – 15 QLED TVs in both 4K and 8K and five monitors. Samsung’s 2022 QLED TV line-up was recognized for its accurate expression of 2,030 Pantone colors and newly added 110 skin tone shades.
       
      “As TVs become more of an entertainment hub in the home, there’s an increased demand for screens with top-tier picture quality that minimize eye strain,” said Seokwoo Yong, Executive Vice President and Head of R&D Team, Visual Display Business at Samsung Electronics. “This recognition from leading global institutes validates our technology that delivers best-in-class images along with the most comfortable watching experience.”
       
       
      1 Applicable to Samsung’s 2022 Lifestyle TVs consisting of The Frame, The Serif and The Sero.
      View the full article
    • By regenitin
      Hello everyone.
      I recently purchased Samsung The Frame 2020 (QA55LS03TAKXXL) in India 2 very important observations from my end.
      I am planning to purchase the Samsung Q600A Soundbar to pair with my TV and wanted to know if my TV supports Q-Symphony. I reached out the support team and they confirmed me that the Q-Symphony function is not a feature of my TV. I wanted to know if Samsung plans to update the operating System to support Q Symphony as this feature is present in Samsung The Frame 2021 which has exact same hardware as Samsung The Frame 2020. Since the support did not have any idea i thought of reaching out to folks on this forum. I am not able to find Amazon Prime Music & Youtube Music apps on the Samsung App Store. These are supposed to be pretty necessary and widely used apps and I am not able to figure out the reason they are not in the store. I would like to know if there are any plans to launch these apps soon. Also let me know if there is any work around to get the apps installed. Thanks in advance for your inputs.
      Best Regards
      Nitin
    • By Alex
      Starting this year, Samsung's Tizen app store is no longer accessible, both to new and existing users. Last year in June, the company closed registrations and made the store available only to existing users and they could only get previously downloaded apps.

      After December 31, 2021, however, the Tizen app store is permanently closed. So in case you are using a Samsung Z series smartphone, it might be time to switch over to Android or iOS. The last Samsung Z4 phone running Tizen OS was released back in 2017 so it was kind of an expected turn of events.
      It seems like the company is dropping its Tizen project after this year's Galaxy Watch4 series is running on Google's Wear OS and all future Galaxy watches will do the same.
      Source: https://www.gsmarena.com/samsung_shuts_down_the_tizen_app_store-news-52598.php
    • By STF News
      Samsung Electronics announced that its 32-inch Odyssey Neo G8 monitor earned a Best of Innovation Award in the Gaming category at CES 2022.
       
      Samsung’s lineup received 2022 Honoree accolades across the board, including for the 55-inch Odyssey Ark, the 49-inch Odyssey Neo G9, the 32-inch Odyssey Neo G8, the 34-inch Odyssey G8, the 32-inch Smart Monitor M8 and the 32-inch High Resolution Monitor S8. This marks the ninth accolade that Samsung monitors have received at this year’s CES, setting a new record for the lineup at the world’s largest electronics show.
       
      Sponsored by the CTA, which owns and organizes CES, the CES Innovation Awards program spotlights standout examples of design and engineering across multiple consumer product categories. This marks the sixth straight year that Samsung monitors have earned CES Innovation Awards, solidifying the company’s position as the global leader in the gaming monitor market.
       
      Check out the infographic below to view this year’s complete list of award-winning monitors.
       

      View the full article
×
×
  • Create New...