Quantcast
Jump to content

Folding the Web: Enabling Real Responsive Design on Foldable Devices


STF News
 Share

Recommended Posts

2020-09-21-01-banner.jpg

Foldable devices

Although new to the industry, the concept of folding devices already has plenty of iterations. Samsung and other OEMs have presented devices with folding capabilities, that allow them to expand into small tablets, compress into smaller footprints, or enable more productive multi-tasking.

Consensus is now, that a “foldable” is a device that folds. While this might seem trivial, the details of how many screens, where/how they fold, and what size and shape the device should be, etc, were all up for discussion. Let’s agree for now that indeed, a foldable is a device that folds. Independent of whether it’s a vertical or horizontal fold, or if it is one big flexible screen or two adjacent ones, how many logical regions the folding translates to, or if it is the size of a phone or a laptop.

1*lK_WtwZiD-WnhdU2Wshuag.png
different types of “foldables”

a foldable is a device that folds

Some time ago I wrote about how to make sure websites looked good on the original Samsung Galaxy Fold. All that information, which summarizes into “make your websites responsive”, is very valid and I’d assume already a common practice. This was a good initial approach at making sure that the Web would continue to flow into different screen sizes, but these new devices were more than just expanded screen real estate. Treating them as just more space for Web content was against the intrinsic nature of the hardware itself, and against the full capabilities that could be enabled for the developer and end user. I knew that in the future I would be sharing information regarding more organic layouts for these types of devices.

Current Web on Galaxy Fold
How to make sure your website looks great on the Galaxy Fold on medium.com

We realized it made sense to embrace the physicality of the state of the device to provide a seamless (pun intended) experience for the user, by giving the developer access to more information regarding this aforementioned physicality.

Foldable Related APIs

Several lines of work started in parallel to address these challenges at Samsung, Microsoft and Intel. The CSS spanning media query and Window Segments Enum API allow developers to detect where the fold occurs, so that they can avoid positioning content on it. This is very handy especially in scenarios where you have dual screen devices with a physical separation between the screens. Microsoft have written an excellent post about these APIs. With the CSS spanning media query it is possible to know the layout of the foldable device and make sure there’s no content cut in half.

I want to introduce the work that we’ve been doing to address other use cases related to foldable devices here at Samsung with collaboration of our lovely friends at Intel.

Screen Fold API

1*Z__KbD6KvE2ZOC9smlMM8g.png
Screen Fold API logo

We envision the need to adapt content to the physical way a device is being used. I mentioned earlier that foldables are more than just extra on-screen space for content. To embrace them we need to take into account the angle in which the screen(s) is folded. We can abstract this into a concept called the “posture” of the device. Considering posture, we can immediately see the benefits of making our content react to these changes. This is the basic concept behind the Screen Fold API.

Disclaimer: All the things I’m writing about at the moment are drafts that are subject to change, evolution or may never even see the light of day, nonetheless I believe it can be exciting to get an early glimpse into the future of Web design and to hear feedback from anyone interested.

The Screen Fold API is an extension to the Screen interface that exposes an angle value, a posture attribute and a change event. It allows developers to identify which posture a device is in and rearrange content accordingly. Working in conjunction with the CSS spanning media queries, one could achieve very interesting layouts. The API identifies a set of predefined postures for common use cases as seen in the image below. (Do note that names or states are not final and we are already considering adding a ‘peek’ state and changing the ‘laptop’ moniker that doesn’t bode well for phones for example.)

1*QDYaRrLuEfPdUx-8_1vt5Q.png
diagram of the postures available in the Screen Fold API (laptop, flat, book, tablet, tent, no fold)

Use Cases

These postures respond to different use cases, as detailed in the explainer linked below, you might recognize some of this functionality already shipped in some native apps under the names of ‘flex mode’.

SamsungInternet/Explainers
See the latest draft of the specification here New types of mobile devices are appearing that have some sort of folding… github.com

These postures are not always what a developer might need so the API also allows us to directly query the angle values. The image below shows the proposed mapping between postures and angles.

1*eoAE6wVtS_jkRB4xIAKO4w.png
Mapping between postures and angles

Examples

There is a way to query these attributes both from JS and CSS. The former is through the screen object, the latter through handy CSS media queries. You can refer to an example below.

1*jBDFTZsrReoQdCQfqdxfEg.png


@media (screen-fold-posture: laptop) and (spanning: single-fold-horizontal){
   body {
     display: flex;
     flex-flow: column nowrap;
   }

   .videocall-area, .videocall-controls {
     flex: 1 1 env(fold-bottom);
   }
}

State of the API

If you’ve read up to here, it is very likely you find this interesting and are wondering what is the current state of the API? As I mentioned before, these are all early concepts. Fortunately, there is a specification draft and work under way at W3C. The Screen Fold API is expected to officially be a part of the Devices and Sensors Working Group and is already listed in the DAS WG roadmap.

1*Nti1pINXqEqlCgNpdi5CQQ.png
Devices and Sensors Working Group

The Screen Fold API
This document specifies an API that allows web applications to request the angular value to which a device with a… w3c.github.io

1*i8vZ-f2YvBoLxBmnx5i-4A.png
Samsung Internet Beta

Furthermore, at Samsung we have been experimenting internally with implementations for the Screen Fold API. We are committed to the Screen Fold API and hope to share an update and more news in the future. This is how a basic example for the Screen Fold API looks running on a Galaxy Z Fold2 device on a custom Samsung Internet build.

1*N8f45wKC_-t09vf5afYRMg.png
Screen Fold API behind a flag in Samsung Internet custom build

 

 

html {
    background-color: blue;
}

@media (max-screen-fold-angle: 120deg) {
    html {
        background-color: red;
    }
}

Finally, to enable you to jump into the fun and experimentation with the Screen Fold API, we have a very early version of a Polyfill and a settings widget for browsers that don’t support the specification draft or that do not fold.

1*iyfsRYjrUQiewKt0JJsmtA.png
Screen Fold Settings widget

Screen Fold Polyfill demo
The Screen Fold API allows you to know the angle and posture that the screen(s) of a device is in. The polyfill allows… w3c.github.io

Yes, I am aware there are compatibility issues in the settings widget at the moment in some browsers and I’m more than happy to welcome help 😉.

The Road Ahead

The journey to properly supporting the Web on foldable devices is just getting started. There are various efforts from different fronts and discussions in standards groups and among the community are under way. At Samsung, we are committed to continue the conversation and advance the implementation. We are looking into ways to bring experimental support for the API in the future and we are in talks with other vendors to make sure we have more implementations.

As technical co-editor of the specification, I am very interested in knowing what you think of the Screen Fold API and if you have any considerations of things we might have left out. Feel free to contact myself (@diekus) or Daniel Appelquist for any questions.

In the meantime, play with the polyfill and let us know your thoughts and designs, as they will help us better shape the folding future of the Web.

Acknowledgements

I’d like to finish this post with a huge thank you to Jo Franchetti. She has lent her support and her CSS expertize from the beginning with every weird idea that comes into my head. Additionally, even though in it’s infancy, this work is possible due to the great advice from my co-editor, Kenneth, and to the internal support from Daniel Appelquist, Heejin Chung and the amazing engineering team in Suwon, Baul and Kangil. ✨

1*kNL65VJsJ-2WA-Rgbyw6Xw.png

View the full blog at its source

Link to comment
Share on other sites



  • Replies 0
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Similar Topics

    • By STF News
      This month in San Francisco, the 2022 Samsung Developer Conference (SDC) spotlighted some of Samsung’s best and brightest minds across tech, marketing and product fields, sharing a compelling vision of the future, showcasing the transformational technologies that will improve consumers’ daily lives and giving users more time to focus on what matters most.
       
      With opening remarks from Jong-Hee Han, Vice Chairman, CEO and Head of Device Experience (DX) Division at Samsung Electronics, the presentations revealed how Samsung is crafting systems that help make lives smarter, safer, more convenient and more connected than ever before.
       
      ▲ Attendees take their seats at SDC 2022 in anticipation of the day’s wide-ranging Keynote
       
      Read on to see Samsung Newsroom’s recap of the innovation and inspiration revolutionizing how we live and work both now and in the future.
       
       
      Opening: A Seamless, Frictionless Experience
      Jong-Hee Han opened up the day’s talks by underscoring the importance of convenience in an age of brilliant yet complex technologies. “At Samsung, our goal has always been to make our lives easier through technology innovation,” said Han. “But innovation can also be burdensome for users if convenience is not at the forefront. Our teams are determined to deliver devices that are as smooth and accessible as they are advanced.”
       
      ▲ Jong-Hee Han, Vice Chairman, CEO and Head of DX Division at Samsung Electronics, welcomes attendees to SDC 2022.
       
      Covering everything from Calm Technology to Knox Matrix to holistic household platforms like Bixby Home Studio and SmartThings, Han outlined a device and systems ecosystem that puts the user experience at the center. “The goal of all this is to give you an easier, seamless and more frictionless experience,” Han emphasized. “So you can focus on what matters most.”
       
       
      A Home That Learns and Adapts Through SmartThings
      Following Jong-Hee Han, Jaeyeon Jung, Vice President and Head of SmartThings, Mobile eXperience Business, discussed how Covid-19 galvanized teams at Samsung to prioritize the important things in life, particularly in the home. “If you’re like me, the past few years have been a time to prioritize what matters, to find moments to pause and refresh,” said Jung.
       
      ▲ Jaeyeon Jung, Vice President and Head of SmartThings, Mobile eXperience Business, outlines how the platform is helping realize a truly seamless, intuitive household device ecosystem.
       
      Samsung’s SmartThings platform, Jung explained, is the perfect way to do this in your home. Thanks to a wide range of improvements to the household ecosystem platform, finding, setting up, onboarding and synchronizing new devices into your household will be easier and more efficient than ever. This includes Galaxy Quick Pair, Matter Hub connectivity, SmartThings Energy and SmartThings Music Sync, as well as boosted integration with partner systems and an expanded SmartThings website.
       
       
      Matter: A Truly Harmonious Household Experience
      At the end of her talk, Jung introduced Matthew McCullough from Google, who provided an overview of Matter, a system that lays the groundwork for true interoperability between devices in the home. Samsung is committed to Matter integration, and confident it will make the dream of a fully integrated household a reality.
       
      ▲ Google’s Matthew McCullough explains how the Matter interoperability system works, reinforcing Samsung and Google’s collaborative efforts to expand its integration.
       
      By working closely together, Samsung and Google aim to make life for both users and device makers simpler and smoother, with multi-admin features that cut down on manual management and make Google Home and Nest Hub more accessible. “This ease of use across Samsung and Google is just the start, and we’re looking forward to delivering a more seamless, delightful home experience,” said McCullough.
       
       
      Industry Collaboration Making Device Ecosystems More Open
      ▲ Mark Benson, Head of SmartThings U.S. at Samsung, expands on the company’s enthusiasm for Matter adoption and introduces the SmartThings Matter Early Access Program.
       
      Continuing with the theme of interoperability, Mark Benson, U.S., emphasized Samsung’s commitment to supporting Matter. “To accelerate Matter adoption, we created the SmartThings Matter Early Access Program, where we work with major device manufacturers to ensure their Matter-enabled devices work seamlessly with SmartThings,” said Benson.
       
      The SmartThings app, SmartThings Hub Core and SmartThings Cloud, Benson pointed out, are already prepared to support Matter. “It’s all about delivering interoperability right out of the box,” he said.
       
       
      The Expanding Role of Bixby
      Anil Yadav, Head of Bixby Lab at Samsung Research1 America, took to the stage to discuss SmartThings’ bespoke speech platform and de facto voice in the home, including its exciting new feature: Bixby Home Studio. “With Bixby Home Studio, you can create experiences that span multiple devices and respond more intuitively to user commands,” said Yadav. “For example, if you ask Bixby to turn on your air conditioner, Bixby can check if you have any windows open and let you know to close them, saving energy.”
       
      ▲ Voice system expert and Head of Bixby Lab Anil Yadav lays out the platform’s evolving role in the home and celebrates the launch of Bixby Home Studio.
       
      Ready to do everything from queuing up your next action movie on your Samsung Smart TV to boosting your mobile security with On-Device AI, Bixby is smarter and more powerful than ever.
       
       
      Premium Security and Privacy for an Integrated Household
      ▲ Shin Baik, Principal Engineer of Security Team, Mobile eXperience Business, underscores the significance of strong security and personalized privacy, as exemplified by systems like Samsung Knox and Knox Matrix.
       
      Samsung envisions more than just a convenient and integrated home experience. Its teams are also creating an experience that is as safe as it is smooth, as private as it is proactive. After Yadav’s Bixby talk, Shin Baik, Principal Engineer of Security Team, Mobile eXperience Business, discussed with the audience how top-notch innovations need top-notch security to go with them. Unwavering in its three security and privacy principles — transparency, choice and protection — Samsung has worked hard to develop platforms like Samsung Knox. This will keep everything from your phone to your refrigerator safe and secure, said Baik.
       
      A new platform called Knox Matrix, in particular, is now on the frontline of multi-device protection. “Knox Matrix is your own private blockchain,” Baik explained. “Multiple devices share credentials, turning the ecosystem of devices into a shield that protects everyone.”
       
      In step with these developments, of course, is Samsung’s insistence on a customizable privacy experience, which is provided by Samsung’s comprehensive Security and privacy dashboard. “We believe that you — and only you — should be making decisions about how your information is managed and stored,” said Baik.
       
       
      A More Vibrant Content Ecosystem Than Ever
      Sang Kim, Senior Vice President and Head of Product & Marketing at Samsung Electronics America, shared important insights on Samsung’s extensive media network, covering a range of recent successes.
       
      ▲ Sang Kim, Senior Vice President and Head of Product & Marketing at Samsung Electronics America, provides a roundup of Samsung’s diverse — and rapidly expanding — media network.
       
      Samsung Gaming Hub, launched in June, is an all-in-one gaming solution that, thanks to partnerships with industry leaders like Xbox, lets players connect their controllers to Samsung TVs and access all the games they love without even needing a console or PC.
       
      Samsung TV Plus, the FAST streaming TV and video platform that relaunched this summer, now reaches 24 countries and 465 million devices globally. It is projected to clock in an impressive three billion hours streamed by users by the end of 2022.
       
      Thanks to broad reach, smooth monetization integration and an advertising environment optimized for both clients and users, Samsung streaming also offers an ideal platform for premium advertising.
       
      “Today, Fortune 500 companies from around the world have leveraged Samsung TV Plus and our other advanced TV advertising solutions to reach valuable consumers,” Kim said.
       
       
      Tizen’s Global Conquest
      ▲ Yongjae Kim, Executive Vice President and Deputy Head of R&D Team, Visual Display Business discusses all things Tizen, from the operating system’s global reach to its growing spectrum of applications.
       
      Following Sang Kim, Yongjae Kim, Executive Vice President and Deputy Head of R&D Team, Visual Display Business, took to the stage to talk about Tizen, Samsung’s open source, Linux-backed operating system. Kim took the audience through a huge variety of developments in the Tizen journey, which is growing its capabilities to an ever-wider spectrum of uses and applications.
       
      Leveraging partnerships with gaming leaders like Xbox, NVIDIA and Utomik, Tizen is helping bring a truly consolidated, uninterrupted gaming experience to users with Samsung Gaming Hub, with fantastic performance and image quality, Kim explained. It delivers stunning visual displays and blockchain wallet integration to Samsung TVs for better at-home art experiences, while also helping Smart Hub deliver the best in content searches and recommendations. Tizen Enterprise, meanwhile, is boosting B2B partners’ display and signage options, while Tizen Everywhere aims to expand licensing and bring the system to more countries.
       
      “Since its launch ten years ago, Tizen now powers more than 330 million TVs, refrigerators and other smart devices,” said Kim. “At Samsung, we will continue to create Tizen experiences that transform and go beyond what many thought possible.”
       
       
      One UI 5 and the “Your Galaxy, Your Way” Philosophy
      Next up was Sally Hyesoon Jeong, Vice President of Framework R&D Group, Mobile eXperience Business, whose theme for the day was the company’s latest leap forward in the UI experience: One UI 5.
       
      ▲ Sally Hyesoon Jeong, Vice President of Framework R&D Group, Mobile eXperience Business, introduces One UI 5, the next stage of Samsung’s UI evolution and a distillation of the “Your Galaxy, Your Way” philosophy.
       
      “On this stage three years ago, I shared our vision for One UI, the essence of the Galaxy experience,” Jeong began. “One UI truly captures the ‘Your Galaxy, Your Way’ philosophy by bringing users an intuitive, connected experience that is uniquely yours.”
       
      Splitting the new UI update’s advancements into three areas — Personalization, Productivity and Possibility — Jeong shared a host of new and improved features. Personalization included deep customization enhancements, like Dynamic Lockscreen for smartphones, Watch Face Studio for Galaxy Watches and custom-built Modes and Routines, while health and security functions are also more adaptable than ever. The Productivity section included Bixby Text Call, improved phone-to-PC connectivity and multitasking upgrades like Taskbar improvements. Possibilities covered One UI 5’s integration with innovative Samsung Foldable devices and related functions like Flex Mode.
       
       
      A Look Over the Horizon: Robotics and Health in Tomorrow’s Homes
      The final talk before the wrap-up was from Sebastian Seung, President and Head of Samsung Research. While many of the day’s talks focused on recent improvements, Seung has his eye on the innovations of tomorrow, aiming for no less than the merging of the physical and digital worlds.
       
      “I’d like to invite you to look over the horizon, to a future dreamed of but not yet realized,” said Seung, who divided his presentation between the future of robotics in the home and Samsung’s ambitions to contribute to health & wellness research.
       
      In the home robotics field, Seung discussed the company’s continued success with a household supported by advanced, AI-enhanced devices, including the object-detecting JetBot AI+ vacuum, the Bot Handy chore assistant prototype and even future plans for Samsung’s latest robot arm manipulation code, which will soon be available on GitHub.
       
      ▲ Sebastian Seung, President and Head of Samsung Research, discusses the company’s plans for a future where advanced robotics technologies support households and outlines Samsung’s ambitions in health & wellness research.
       
      Moving on to the subject of health, Seung introduced Samsung Health Stack, an open source project from Samsung that aims to actively advance health & wellness research. “This is a full stack SDK, a mobile app for patients, a portal for researchers and a backend database,” said Seung. “A complete package of open-source tools designed for the easy creation of digital systems for the research of health and wellness.”
       
       
      Closing: Building a More Sustainable Future With Samsung
      To round off the talks, Jong-Hee Han returned to the stage to offer a final word, summarizing the technologies showcased at SDC and the course of the company’s future of innovation.
       
      ▲ Jong-Hee Han returns to the stage to conclude the day’s talks, ending on a proactive vision of a future driven by sustainable, eco-conscious technologies.
       
      “I would like to talk about the future one more time,” he added. “The future of something very dear to my heart: our planet. I believe we have a responsibility to help create a sustainable future. We do that through Everyday Sustainability, a belief that every stage of the product lifecycle can be more sustainable.”
       
      Han presented a range of Samsung sustainability commitments and environmental research directions, including responsible material sourcing, recycling, disposal, renewable energy and billions in investments toward meeting carbon emissions reductions.
       
      “By 2027, we will run all DX operations on renewable energy,” Han said, ending the day on an optimistic note and a glimpse of a future for customers that is easier, more seamless, more frictionless and — crucially — more eco-conscious.
       
      Watch the video below to see the complete recording of the SDC22 Keynote.
       
        
      ▲ Attendees explore the latest Samsung products at SDC22 booths.
       
       
      1 Samsung Research, acting as Samsung Electronics’ advanced R&D hub, leads the development of future technologies for the company’s Device eXperience (DX) Division.
      View the full article
    • By STF News
      Foldable technology for mobile is a groundbreaking experience not only for users but also for developers. The presence of many form factors like immersive display, app continuity, Flex mode and UX optimization challenge developers to think outside of the box to adapt to this technology. In this blog, we discuss a checklist to provide a better understanding about the adaptation, modification and design changes that are required. So, let's go through each point, one-by-one.
      Large screen optimization
      Samsung foldable devices have two different physical screens: the cover display and the main display. As the cover display is much smaller than the main display, large screen optimization is one of the key areas of designing UX for foldable devices. In a nutshell, your app can utilize the extra space in the main display by showing more information. Just having a different UI design with the same information can do the trick of optimization as well.
      Large screen optimization of a note app
      To implement this scenario, define different layout files for each display using the alternate resources option. For example, if you want to define a different UI for the main display, create a new directory named layout-sw600dp under the res directory and then create an xml file named activity_main. Then add the layout code as required.
      UI is designed separately for the main display and the cover display
      Flex mode optimization
      In Galaxy Z series devices, Samsung introduced a new mode called Flex mode. This mode allows users to use apps while the book-like phone is partially folded. Creative design can really make your app stand out from others in Flex mode.
      Google Duo app in Flex mode in Galaxy Z series devices
      Using Google’s new Jetpack library, WindowManager, you can detect the current posture of a Galaxy Z series device and update the UI accordingly by following these steps:
      Step 1: Add the dependencies in the build.gradle.
      implementation "androidx.window:window:1.0.0-alpha01" Step 2: Define a WindowManager instance.
      val windowManager = WindowManager(this, null) Step 3: Register the DeviceState change listener. The listener notices changes in the device state (for example CLOSED, OPENED, HALF_OPENED).
      windowManager.registerDeviceStateChangeCallback( mainThreadExecutor /* Executor */, callback /* Consumer<DeviceState> */ ) Step 4: Write a callback function to check deviceState.posture to get the current posture of the device. If the posture is POSTURE_HALF_OPENED, the app UI gets updated for Flex mode.
      val callback = Consumer<DeviceState> { deviceState -> if (deviceState.posture == DeviceState.POSTURE_HALF_OPENED) { // Display is folded, show Split UX } else { // Display is not folded, show Full Screen UX } } Check out the CodeLab challenge on Flex mode for a more hands-on experience.
      App continuity
      While folding and unfolding the device, the app must prevent data loss thus ensuring its continuity. This is achievable by using the onSaveInstanceState() method. First, save the data to retain the current state with onSaveInstanceState().
      @Override public void onSaveInstanceState(Bundle savedInstanceState) { //Save the current state } Then, restore the data in the onCreate() function.
      @Override protected void onCreate(Bundle savedInstanceState) { if (savedInstanceState != null) { //restore the previous state } } Stopwatch app continuity while unfolding device
      To have a better understanding of implementing app continuity, see the CodeLab challenge on app continuity.
      Responsive UI layout
      To adapt to new form factors such as a diverse aspect ratio, Flex mode, multi-window, and pop-up window, utilize the following guidelines :
      Design a responsive UI layout for your app using ConstraintLayout. Define the activity of your app as resizable, to ensure the maximum compatibility of your app with both the cover display and the main display of the device. Set the resizableActivity attribute to true in Manifest.xml. <activity android:name=".MainActivity" android:resizeableActivity="true"> … </activity> Responsive layout of the Spotify app
      Cutout and punch hole of the main display
      The main display of a Galaxy Z Fold is covered by an area on the top-right side called the “L-cut” whereas the Galaxy Z Fold2 and Fold3 have a punch hole in the upper right side and the Galaxy Z Flip devices have a punch hole in the middle. Some portion of your app’s UI could be covered by the L-cut or the punch hole.
      Content is covered by the L-cut in a Galaxy Fold device in landscape mode
      To avoid such a scenario, depending on your UI content style, define a display cutout mode. For example, the content is letterboxed in landscape mode whereas it is rendered into the cutout area in portrait mode by default. Define the display cutout mode in the style.xml as shortEdges so that your content is rendered into the cutout area in both portrait and landscape modes.
      <item name="android:windowLayoutInDisplayCutoutMode">shortEdges</item> Display cutout in the default mode and the shortEdges mode, respectively
      Last but not the least, you can test your app on our device cloud, Samsung Remote Test Lab, to make sure you have implemented all the checkpoints discussed in this blog. You can also participate in our Codelab challenges to have a clear understanding of the implementation details.
      In this blog we have discussed about how to adapt your app for foldable devices. We hope it is a good guide for you to start with. You can also check out Samsung’s official documentation and reach out to our developer community if you have any queries.
      View the full blog at its source



×
×
  • Create New...