Quantcast
Jump to content


New Game Changing Vulkan Extensions for Mobile: Timeline Semaphores


Recommended Posts

2021-06-28-01-banner.jpg

The Samsung Developers team works with many companies in the mobile and gaming ecosystems. We're excited to support our partner, Arm, as they bring timely and relevant content to developers looking to build games and high-performance experiences. This Vulkan Extensions series will help developers get the most out of the new and game-changing Vulkan extensions on Samsung mobile devices.

In previous blogs, we have already explored two key Vulkan extension game changers that will be enabled by Android R. These are Descriptor Indexing and Buffer Device Address. In this blog, we explore the third and final game changer, which is 'Timeline Semaphores'.

The introduction of timeline semaphores is a large improvement to the synchronization model of Vulkan and is a required feature in Vulkan 1.2. It solves some fundamental grievances with the existing synchronization APIs in Vulkan.

The problems with VkFence and VkSemaphore

In earlier Vulkan extensions, there are two distinct synchronization objects for dealing with CPU <-> GPU synchronization and GPU queue <-> GPU queue synchronization.

The VkFence object only deals with GPU -> CPU synchronization. Due to the explicit nature of Vulkan, you must keep track of when the GPU completes the work you submit to it.

vkQueueSubmit(queue, …, fence);

The previous code is the way we would use a fence, and later this fence can be waited on. When the fence signals, we know it is safe to free resources, read back data written by GPU, and so on. Overall, the VkFence interface was never a real problem in practice, except that it feels strange to have two entirely different API objects which essentially do the same thing.

VkSemaphore on the other hand has some quirks which makes it difficult to use properly in sophisticated applications. VkSemaphore by default is a binary semaphore. The fundamental problem with binary semaphores is that we can only wait for a semaphore once. After we have waited for it, it automatically becomes unsignaled again. This binary nature is very annoying to deal with when we use multiple queues. For example, consider a scenario where we perform some work in the graphics queue, and want to synchronize that work with two different compute queues. If we know this scenario is coming up, we will then have to allocate two VkSemaphore objects, signal both objects, and wait for each of them in the different compute queues. This works, but we might not have the knowledge up front that this scenario will play out. Often where we are dealing with multiple queues, we have to be somewhat conservative and signal semaphore objects we never end up waiting for. This leads to another problem …

A signaled semaphore, which is never waited for, is basically a dead and useless semaphore and should be destroyed. We cannot reset a VkSemaphore object on the CPU, so we cannot ever signal it again if we want to recycle VkSemaphore objects. A workaround would be to wait for the semaphore on the GPU in a random queue just to unsignal it, but this feels like a gross hack. It could also potentially cause performance issues, as waiting for a semaphore is a full GPU memory barrier.

Object bloat is another considerable pitfall of the existing APIs. For every synchronization point we need, we require a new object. All these objects must be managed, and their lifetimes must be considered. This creates a lot of annoying “bloat” for engines.

The timeline – fixing object bloat – fixing multiple waits

The first observation we can make of a Vulkan queue is that submissions should generally complete in-order. To signal a synchronization object in vkQueueSubmit, the GPU waits for all previously submitted work to the queue, which includes the signaling operation of previous synchronization objects. Rather than assigning one object per submission, we synchronize in terms of number of submissions. A plain uint64_t counter can be used for each queue. When a submission completes, the number is monotonically increased, usually by one each time. This counter is contained inside a single timeline semaphore object. Rather than waiting for a specific synchronization object which matches a particular submission, we could wait for a single object and specify “wait until graphics queue submission #157 completes.”

We can wait for any value multiple times as we wish, so there is no binary semaphore problem. Essentially, for each VkQueue we can create a single timeline semaphore on startup and leave it alone (uint64_t will not overflow until the heat death of the sun, do not worry about it). This is extremely convenient and makes it so much easier to implement complicated dependency management schemes.

Unifying VkFence and VkSemaphore

Timeline semaphores can be used very effectively on CPU as well:

VkSemaphoreWaitInfoKHR info = { VK_STRUCTURE_TYPE_SEMAPHORE_WAIT_INFO_KHR };
info.semaphoreCount = 1;
info.pSemaphores = &semaphore;
info.pValues = &value;
vkWaitSemaphoresKHR(device, &info, timeout);

This completely removes the need to use VkFence. Another advantage of this method is that multiple threads can wait for a timeline semaphore. With VkFence, only one thread could access a VkFence at any one time.

A timeline semaphore can even be signaled from the CPU as well, although this feature feels somewhat niche. It allows use cases where you submit work to the GPU early, but then 'kick' the submission using vkSignalSemaphoreKHR. The accompanying sample demonstrates a particular scenario where this function might be useful:

VkSemaphoreSignalInfoKHR info = { VK_STRUCTURE_TYPE_SEMAPHORE_SIGNAL_INFO_KHR };
info.semaphore = semaphore;
info.value = value;
vkSignalSemaphoreKHR(device, &info);

Creating a timeline semaphore

When creating a semaphore, you can specify the type of semaphore and give it an initial value:

VkSemaphoreCreateInfo info = { VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO };
VkSemaphoreTypeCreateInfoKHR type_info = { VK_STRUCTURE_TYPE_SEMAPHORE_TYPE_CREATE_INFO_KHR };
type_info.semaphoreType = VK_SEMAPHORE_TYPE_TIMELINE_KHR;
type_info.initialValue = 0;
info.pNext = &type_info;
vkCreateSemaphore(device, &info, NULL, &semaphore);

Signaling and waiting on timeline semaphores

When submitting work with vkQueueSubmit, you can chain another struct which provides counter values when using timeline semaphores, for example:

VkSubmitInfo submit = { VK_STRUCTURE_TYPE_SUBMIT_INFO };
submit.waitSemaphoreCount = 1;
submit.pWaitSemaphores = &compute_queue_semaphore;
submit.pWaitDstStageMask = &wait_stage;
submit.commandBufferCount = 1;
submit.pCommandBuffers = &cmd;
submit.signalSemaphoreCount = 1;
submit.pSignalSemaphores = &graphics_queue_semaphore;
 VkTimelineSemaphoreSubmitInfoKHR timeline = {
VK_STRUCTURE_TYPE_TIMELINE_SEMAPHORE_SUBMIT_INFO_KHR };
timeline.waitSemaphoreValueCount = 1;
timeline.pWaitSemaphoreValues = &wait_value;
timeline.signalSemaphoreValueCount = 1;
timeline.pSignalSemaphoreValues = &signal_value;
submit.pNext = &timeline;
 signal_value++; // Generally, you bump the timeline value once per submission.
 vkQueueSubmit(queue, 1, &submit, VK_NULL_HANDLE);

Out of order signal and wait

A strong requirement of Vulkan binary semaphores is that signals must be submitted before a wait on a semaphore can be submitted. This makes it easy to guarantee that deadlocks do not occur on the GPU, but it is also somewhat inflexible. In an application with many Vulkan queues and a task-based architecture, it is reasonable to submit work that is somewhat out of order. However, this still uses synchronization objects to ensure the right ordering when executing on the GPU. With timeline semaphores, the application can agree on the timeline values to use ahead of time, then go ahead and build commands and submit out of order. The driver is responsible for figuring out the submission order required to make it work. However, the application gets more ways to shoot itself in the foot with this approach. This is because it is possible to create a deadlock with multiple queues where queue A waits for queue B, and queue B waits for queue A at the same time.

Ease of porting

It is no secret that timeline semaphores are inherited largely from D3D12’s fence objects. From a portability angle, timeline semaphores make it much easier to have compatibility across the APIs.

Caveats

As the specification stands right now, you cannot use timeline semaphores with swap chains. This is generally not a big problem as synchronization with the swap chain tends to be explicit operations renderers need to take care of.

Another potential caveat to consider is that the timeline semaphore might not have a direct kernel equivalent on current platforms, which means some extra emulation to handle it, especially the out-of-order submission feature. As the timeline synchronization model becomes the de-facto standard, I expect platforms to get more native support for it.

Conclusion

All three key Vulkan extension game changers improve the overall development and gaming experience through improving graphics and enabling new gaming use cases. We hope that we gave you enough samples to get you started as you try out these new Vulkan extensions to help bring your games to life

Follow Up

Thanks to Hans-Kristian Arntzen and the team at Arm for bringing this great content to the Samsung Developers community. We hope you find this information about Vulkan extensions useful for developing your upcoming mobile games.

The Samsung Developers site has many resources for developers looking to build for and integrate with Samsung devices and services. Stay in touch with the latest news by creating a free account or by subscribing to our monthly newsletter. Visit the Marketing Resources page for information on promoting and distributing your apps and games. Finally, our developer forum is an excellent way to stay up-to-date on all things related to the Galaxy ecosystem.

View the full blog at its source

Link to comment
Share on other sites



  • Replies 0
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
  • Similar Topics

    • By Samsung Newsroom
      Samsung Electronics aims to enhance users’ smart home experience by creating an ecosystem of connected devices ranging from TVs to smartphones to home appliances. However, as more and more devices are connected, smart home security becomes increasingly important. The priority is to protect the smart home ecosystem by safeguarding connected devices from digital threats. With the TV potentially being one of the center pieces of that ecosystem, its security is especially vital.
       
      Samsung’s innovative security program Knox secures smart TVs from various threats, ultimately offering consumers a safe and convenient smart home experience. GNUSMAS, the cute blue alien, introduces Samsung Knox as the final piece to complete your ecosystem of connected devices below.
       

       

       

       

       

       

      View the full article
    • By Samsung Newsroom
      Samsung Electronics today announced that it will unveil a brand film starring BTS “SUGA” on October 5. As Samsung’s ambassador, SUGA will show how he uses The Freestyle 2nd Gen, Samsung’s latest versatile portable projector, in his own unique way. The film is to be unveiled on Samsung’s social media accounts including its Youtube channel.
       
      ▲ BTS SUGA shown using The Freestyle 2nd Gen instead of a monitor
       
      In the film, SUGA emphasizes that “You’ve got the freedom to use The Freestyle anytime, anywhere. No others, only SAMSUNG!” as he demonstrates how he enjoys the portable projector.
       
      ▲ A scene from the brand film starring SUGA
       
      Previously at IFA 2023 held in Berlin, Germany, SUGA surprised visitors as he greeted them from a display screen, inviting them to experience The Freestyle 2nd Gen.
       
      ▲ SUGA greeted visitors at Samsung Electronics’ booth at IFA 2023
       
      The Freestyle is Samsung’s light, portable projector which recently debuted its 2nd generation model. The Freestyle 2nd Gen aims to redefine portable screens with its newly added “Smart Edge Blending” feature and “Gaming Hub” compatibility.
      View the full article
    • By Samsung Newsroom
      Samsung Electronics today announced that it is collaborating with New York City’s Metropolitan Museum of Art to bring some of the museum’s most treasured works of art to The Frame. The selection of The Met’s iconic artworks will be unveiled today on Samsung Art Store1 — which enables users of The Frame to transform any space by displaying more than 2,300 pieces of art, including works from the most distinguished artists, museums and industry tastemakers.
       
      Samsung Art Store users can choose from 38 pieces across a wide range of The Met’s storied curatorial departments, including the American Wing, Asian Art, Egyptian Art, European Paintings, Islamic Art and more. The offering features high-resolution digital reproductions of esteemed artworks across a variety of cultures and time periods housed at The Met.
       
      Samsung Art Store users can display beloved works of art in their homes, including Edgar Degas’ “The Rehearsal of the Ballet Onstage” (circa 1874); Vincent van Gogh’s “Sunflowers” (1887); Paul Cézanne’s “Still Life with Apples and Pot of Primroses” (circa 1890); and Georges Seurat’s “Circus Sideshow” (“Parade du Cirque”) (1887-1888). Owners of The Frame can also display ancient artifacts such as an Egyptian wedjat eye amulet2 (circa 1070-664 B.C.), and medieval treasures including “The Unicorn Rests in a Garden” (1495-1505), the famed French and South Netherlandish textile from the Unicorn Tapestries. Celebrated Japanese artworks such as Katsushika Hokusai’s “Under the Mannen Bridge at Fukagawa” (circa 1830-1832) as well as Utagawa Kuniyoshi’s “Concise Illustrated Biography of Monk Nichiren: Calming the Stormy Sea at Tsunoda in Exile to Sado Island” (1835-1836) are also available. Furthermore, the collection features historically significant American artworks like Emanuel Leutze’s “Washington Crossing the Delaware” (1851).
       
      “Since its founding in 1870, The Met has been dedicated to bringing art and culture to the daily lives of visitors and art enthusiasts around the world,” said Josh Romm, Head of Global Licensing and Partnerships at The Met. “Our collaboration with Samsung activates this mission in a new and modern way, allowing consumers to enjoy iconic works from The Met’s collection at home. As users explore the selection and choose works to display, this program will create a new dialogue about art, creativity and technology.”
       
      The Met’s objective to reveal new ideas and unexpected connections across time and cultures through its collections makes for a fitting collaboration with Samsung Art Store, one of the largest digital platforms of its kind. Showcasing art from museums and galleries around the world to users of The Frame across 42 countries, Samsung Art Store explores centuries of art, from old masters like Botticelli, Leonardo, Goya and Van Gogh to contemporary artists like Shinique Smith and many more through diverse monthly programming. 
       
      “The Samsung Art Store is honored to partner with The Metropolitan Museum of Art to bring world renowned artworks into millions of homes worldwide,” said Sang Kim, EVP and General Manager of the North America Service Business, Samsung Electronics. “At Samsung, we’re constantly working to redefine the entertainment experience through technological innovation, and we’re delighted to partner with such an iconic institution to enable users for the first time to enjoy The Met’s culturally significant works of art via a modern digital canvas from the comfort of their homes.”
       
      Samsung Art Store is available on The Frame by Samsung — a 4K Smart TV with a billion shades of color and a picture frame bezel that delivers vivid, lifelike TV when it’s on and beautiful art when it’s off. Its Matte Display drastically reduces light reflections for a canvas-like finish, which is perfect for showcasing digital collections from Samsung Art Store along with personal photography and artwork. Users can choose from a collection of picture frame-like TV bezels to customize their space and can upgrade the included slim-fit wall mount to the new auto-rotating wall mount3 to display content vertically or horizontally, so all stunning artworks can be viewed exactly as the artist intended.
       
       
      1 A single user subscription for Samsung Art Store costs $4.99/month or $49.90/year.
      2 A wedjat eye amulet is an ancient Egyptian amulet that represents the healed eye of Horus, an ancient Egyptian god. The eye is often depicted as a cross between a human and falcon eye.
      3 The auto-rotating wall mount is sold separately and is compatible with the 2022-2023 models of The Frame in 43”, 50”, 55”, and 65” class screen sizes.
      View the full article
    • By Samsung Newsroom
      Samsung Electronics today announced its ongoing commitment to the future of virtual production with its newest digital signage series. The Wall for Virtual Production (model name: IVC) is now available in Europe and will be on display at IBC 2023, taking place in Amsterdam, the Netherlands from September 15 to 18. This extension builds on its North American release announced at InfoComm 2023.
       
      “The Wall for Virtual Production opens the door to limitless potential for filmmakers, producers and studios by amplifying digital effects, while making virtual content creation easier, faster and more cost effective,” said Hoon Chung, Executive Vice President of the Visual Display Business at Samsung Electronics. “Each feature has been handpicked to empower businesses across industries as they embark on their virtual content creation journey with an unprecedented level of quality.”
       
      Samsung’s leading visual display technology delivers powerful production capabilities that extend reality. Production studios can use ultra-large LED walls to create virtual content, integrating them with real-time visual effects technology to reduce the time and cost of content production. The Wall for Virtual Production is a leader in this type of application and can amplify projects by adding lifelike levels of detail, texture, volume, and shades to virtual content creation environments. The new IVC model options, P1.68 and P2.1, include a curvature range that can stretch up to 5,800R, which creates a more realistic field of view. The display also packs an updated genlock feature that keeps The Wall in line with the system’s signals, so there aren’t any dropped or doubled frames. This capability — along with the enhanced phase offset feature that adjusts the time delay between camera and screen — ensures a perfect image.
       
      To display colors as accurately and consistently as possible, The Wall for Virtual Production features 3D lookup tables (LUTs) for color correction, wide-gamut HDR color processing and color adjustment between individual cabinets or modules. Meanwhile, the integrated Virtual Production Management (VPM) software and its intuitive interface make it easy and more efficient to manage the screen and facilitate the highest possible picture quality in a virtual production environment. The VPM is also capable of detecting and resolving any potential LED-related issues.
       
      In addition, The Wall for Virtual Production is built for any studio environment, no matter how large the building demand is. Whether for a volume studio or XR studio, The Wall for Virtual Production makes the production of immersive content possible.
       
      “We’re thrilled to be supporting Samsung with the exciting launch of The Wall for Virtual Production IVC Series at this year’s IBC. With our heritage in creating cutting-edge virtual production and XR broadcast studios, we’re excited about the powerful new tools the IVC Series brings,” said Andy Hook, Technical Solutions Director at White Light. “These tools give us a huge scope for creating even more impactful and flexible virtual production environments and continuing to drive progress and transformation in this space.”
       
      The Wall for Virtual Production features top-of-the-line capabilities, including:
       
      An improved 4:9 ratio that provides a variety of installation options, including the ability to be hung or stacked with other screens to maximize results in different production environments. A 12,288 Hz refresh rate that minimizes flicker lines and ensures a clear outcome, regardless of the camera used during filming. Black Seal Technology+ that delivers the purest black levels for new depth. Users will experience unparalleled contrast that is specifically designed to be resilient to the dust and particles common in production environments. 20-bit processing to present exact color mapping with faultless accuracy and a precise linear grayscale that shows the intricacies of every scene. Users will be able to see realistic textures, volume and continuous shadow details.  
      The Wall for Virtual Production’s P1.68 and P2.1 models are now available globally including Europe.
      View the full article
    • By Samsung Newsroom
      Samsung Electronics renews its technological contribution for the Vatican with four outdoor LED signages (XHB series, two measuring 7.935m x 4.83m and two measuring 5.865m x 3.105m) and Harman Kardon embedded audio for St. Peter’s Square. This installation follows the previous collaboration in 2020, where two indoor displays were installed in the Nervi Auditorium to help more visitors take part in the ceremonies.
       
      ▲ Four Samsung outdoor LED signage (XHB series) have been installed in the St. Peter’s Square at the Vatican City
       
      St. Peter’s Square, founded on an architectural layout boasting illustrious signatures and timeless pieces of art, hosts various ceremonies where the Pope meets the public. Acknowledging the requirements for the best acoustic and visual rendering in an outdoor space, Samsung took to its outdoors audio-and-video expertise. The four outdoor LED signages have been installed in strategic points of the square to convey a clear picture of commencing events as well as audio solutions to broadcast the message.
       
      ▲ Visitors and worshippers can clearly view the proceedings through Samsung’s outdoor LED signage (XHB series) at St. Peter’s Square, Vatican City
       
      Thanks to Samsung’s LED Signage technology, spectators will be able to take part in the ceremonies at St. Peter’s Square and admire all the details. Being IP66-certified, the XHB series reproduce high-impact content even in harsh weather conditions. In addition, the extremely compact 99.4 mm-thin housing makes them perfectly integrable in an iconic space such as St. Peter’s Square.
       
      Harman Kardon’s signature audio provides balanced panoramic sound that is distributed throughout the space, conveying crisp dialogues for an immersive and engaging experience for worshippers and visitors alike. Adaptive Sound technology clearly diffuses sound even when the volume of voices is low to divulge every element of speech, even during the most whispered dialogues.
       
      “We at Samsung Electronics are excited to make our technological contribution and renew our engagement with Vatican City,” said Davide Corte, Head of IT Division, Samsung Electronics Italy. “The outdoor LED signages and audio solutions create an immersive audio and video experience to serve visitors as well as the communities of the Faithful. It is an honor for us to be present with the best of our technologies in such a unique and prestigious setting such as St. Peter’s Square.”
      View the full article





×
×
  • Create New...