Quantcast
Jump to content

Folding the Web: Enabling Real Responsive Design on Foldable Devices


STF News
 Share

Recommended Posts

2020-09-21-01-banner.jpg

Foldable devices

Although new to the industry, the concept of folding devices already has plenty of iterations. Samsung and other OEMs have presented devices with folding capabilities, that allow them to expand into small tablets, compress into smaller footprints, or enable more productive multi-tasking.

Consensus is now, that a “foldable” is a device that folds. While this might seem trivial, the details of how many screens, where/how they fold, and what size and shape the device should be, etc, were all up for discussion. Let’s agree for now that indeed, a foldable is a device that folds. Independent of whether it’s a vertical or horizontal fold, or if it is one big flexible screen or two adjacent ones, how many logical regions the folding translates to, or if it is the size of a phone or a laptop.

1*lK_WtwZiD-WnhdU2Wshuag.png
different types of “foldables”

a foldable is a device that folds

Some time ago I wrote about how to make sure websites looked good on the original Samsung Galaxy Fold. All that information, which summarizes into “make your websites responsive”, is very valid and I’d assume already a common practice. This was a good initial approach at making sure that the Web would continue to flow into different screen sizes, but these new devices were more than just expanded screen real estate. Treating them as just more space for Web content was against the intrinsic nature of the hardware itself, and against the full capabilities that could be enabled for the developer and end user. I knew that in the future I would be sharing information regarding more organic layouts for these types of devices.

Current Web on Galaxy Fold
How to make sure your website looks great on the Galaxy Fold on medium.com

We realized it made sense to embrace the physicality of the state of the device to provide a seamless (pun intended) experience for the user, by giving the developer access to more information regarding this aforementioned physicality.

Foldable Related APIs

Several lines of work started in parallel to address these challenges at Samsung, Microsoft and Intel. The CSS spanning media query and Window Segments Enum API allow developers to detect where the fold occurs, so that they can avoid positioning content on it. This is very handy especially in scenarios where you have dual screen devices with a physical separation between the screens. Microsoft have written an excellent post about these APIs. With the CSS spanning media query it is possible to know the layout of the foldable device and make sure there’s no content cut in half.

I want to introduce the work that we’ve been doing to address other use cases related to foldable devices here at Samsung with collaboration of our lovely friends at Intel.

Screen Fold API

1*Z__KbD6KvE2ZOC9smlMM8g.png
Screen Fold API logo

We envision the need to adapt content to the physical way a device is being used. I mentioned earlier that foldables are more than just extra on-screen space for content. To embrace them we need to take into account the angle in which the screen(s) is folded. We can abstract this into a concept called the “posture” of the device. Considering posture, we can immediately see the benefits of making our content react to these changes. This is the basic concept behind the Screen Fold API.

Disclaimer: All the things I’m writing about at the moment are drafts that are subject to change, evolution or may never even see the light of day, nonetheless I believe it can be exciting to get an early glimpse into the future of Web design and to hear feedback from anyone interested.

The Screen Fold API is an extension to the Screen interface that exposes an angle value, a posture attribute and a change event. It allows developers to identify which posture a device is in and rearrange content accordingly. Working in conjunction with the CSS spanning media queries, one could achieve very interesting layouts. The API identifies a set of predefined postures for common use cases as seen in the image below. (Do note that names or states are not final and we are already considering adding a ‘peek’ state and changing the ‘laptop’ moniker that doesn’t bode well for phones for example.)

1*QDYaRrLuEfPdUx-8_1vt5Q.png
diagram of the postures available in the Screen Fold API (laptop, flat, book, tablet, tent, no fold)

Use Cases

These postures respond to different use cases, as detailed in the explainer linked below, you might recognize some of this functionality already shipped in some native apps under the names of ‘flex mode’.

SamsungInternet/Explainers
See the latest draft of the specification here New types of mobile devices are appearing that have some sort of folding… github.com

These postures are not always what a developer might need so the API also allows us to directly query the angle values. The image below shows the proposed mapping between postures and angles.

1*eoAE6wVtS_jkRB4xIAKO4w.png
Mapping between postures and angles

Examples

There is a way to query these attributes both from JS and CSS. The former is through the screen object, the latter through handy CSS media queries. You can refer to an example below.

1*jBDFTZsrReoQdCQfqdxfEg.png


@media (screen-fold-posture: laptop) and (spanning: single-fold-horizontal){
   body {
     display: flex;
     flex-flow: column nowrap;
   }

   .videocall-area, .videocall-controls {
     flex: 1 1 env(fold-bottom);
   }
}

State of the API

If you’ve read up to here, it is very likely you find this interesting and are wondering what is the current state of the API? As I mentioned before, these are all early concepts. Fortunately, there is a specification draft and work under way at W3C. The Screen Fold API is expected to officially be a part of the Devices and Sensors Working Group and is already listed in the DAS WG roadmap.

1*Nti1pINXqEqlCgNpdi5CQQ.png
Devices and Sensors Working Group

The Screen Fold API
This document specifies an API that allows web applications to request the angular value to which a device with a… w3c.github.io

1*i8vZ-f2YvBoLxBmnx5i-4A.png
Samsung Internet Beta

Furthermore, at Samsung we have been experimenting internally with implementations for the Screen Fold API. We are committed to the Screen Fold API and hope to share an update and more news in the future. This is how a basic example for the Screen Fold API looks running on a Galaxy Z Fold2 device on a custom Samsung Internet build.

1*N8f45wKC_-t09vf5afYRMg.png
Screen Fold API behind a flag in Samsung Internet custom build

 

 

html {
    background-color: blue;
}

@media (max-screen-fold-angle: 120deg) {
    html {
        background-color: red;
    }
}

Finally, to enable you to jump into the fun and experimentation with the Screen Fold API, we have a very early version of a Polyfill and a settings widget for browsers that don’t support the specification draft or that do not fold.

1*iyfsRYjrUQiewKt0JJsmtA.png
Screen Fold Settings widget

Screen Fold Polyfill demo
The Screen Fold API allows you to know the angle and posture that the screen(s) of a device is in. The polyfill allows… w3c.github.io

Yes, I am aware there are compatibility issues in the settings widget at the moment in some browsers and I’m more than happy to welcome help 😉.

The Road Ahead

The journey to properly supporting the Web on foldable devices is just getting started. There are various efforts from different fronts and discussions in standards groups and among the community are under way. At Samsung, we are committed to continue the conversation and advance the implementation. We are looking into ways to bring experimental support for the API in the future and we are in talks with other vendors to make sure we have more implementations.

As technical co-editor of the specification, I am very interested in knowing what you think of the Screen Fold API and if you have any considerations of things we might have left out. Feel free to contact myself (@diekus) or Daniel Appelquist for any questions.

In the meantime, play with the polyfill and let us know your thoughts and designs, as they will help us better shape the folding future of the Web.

Acknowledgements

I’d like to finish this post with a huge thank you to Jo Franchetti. She has lent her support and her CSS expertize from the beginning with every weird idea that comes into my head. Additionally, even though in it’s infancy, this work is possible due to the great advice from my co-editor, Kenneth, and to the internal support from Daniel Appelquist, Heejin Chung and the amazing engineering team in Suwon, Baul and Kangil. ✨

1*kNL65VJsJ-2WA-Rgbyw6Xw.png

View the full blog at its source

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

  • Similar Topics

    • By STF News
      Foldable technology for mobile is a groundbreaking experience not only for users but also for developers. The presence of many form factors like immersive display, app continuity, Flex mode and UX optimization challenge developers to think outside of the box to adapt to this technology. In this blog, we discuss a checklist to provide a better understanding about the adaptation, modification and design changes that are required. So, let's go through each point, one-by-one.
      Large screen optimization
      Samsung foldable devices have two different physical screens: the cover display and the main display. As the cover display is much smaller than the main display, large screen optimization is one of the key areas of designing UX for foldable devices. In a nutshell, your app can utilize the extra space in the main display by showing more information. Just having a different UI design with the same information can do the trick of optimization as well.
      Large screen optimization of a note app
      To implement this scenario, define different layout files for each display using the alternate resources option. For example, if you want to define a different UI for the main display, create a new directory named layout-sw600dp under the res directory and then create an xml file named activity_main. Then add the layout code as required.
      UI is designed separately for the main display and the cover display
      Flex mode optimization
      In Galaxy Z series devices, Samsung introduced a new mode called Flex mode. This mode allows users to use apps while the book-like phone is partially folded. Creative design can really make your app stand out from others in Flex mode.
      Google Duo app in Flex mode in Galaxy Z series devices
      Using Google’s new Jetpack library, WindowManager, you can detect the current posture of a Galaxy Z series device and update the UI accordingly by following these steps:
      Step 1: Add the dependencies in the build.gradle.
      implementation "androidx.window:window:1.0.0-alpha01" Step 2: Define a WindowManager instance.
      val windowManager = WindowManager(this, null) Step 3: Register the DeviceState change listener. The listener notices changes in the device state (for example CLOSED, OPENED, HALF_OPENED).
      windowManager.registerDeviceStateChangeCallback( mainThreadExecutor /* Executor */, callback /* Consumer<DeviceState> */ ) Step 4: Write a callback function to check deviceState.posture to get the current posture of the device. If the posture is POSTURE_HALF_OPENED, the app UI gets updated for Flex mode.
      val callback = Consumer<DeviceState> { deviceState -> if (deviceState.posture == DeviceState.POSTURE_HALF_OPENED) { // Display is folded, show Split UX } else { // Display is not folded, show Full Screen UX } } Check out the CodeLab challenge on Flex mode for a more hands-on experience.
      App continuity
      While folding and unfolding the device, the app must prevent data loss thus ensuring its continuity. This is achievable by using the onSaveInstanceState() method. First, save the data to retain the current state with onSaveInstanceState().
      @Override public void onSaveInstanceState(Bundle savedInstanceState) { //Save the current state } Then, restore the data in the onCreate() function.
      @Override protected void onCreate(Bundle savedInstanceState) { if (savedInstanceState != null) { //restore the previous state } } Stopwatch app continuity while unfolding device
      To have a better understanding of implementing app continuity, see the CodeLab challenge on app continuity.
      Responsive UI layout
      To adapt to new form factors such as a diverse aspect ratio, Flex mode, multi-window, and pop-up window, utilize the following guidelines :
      Design a responsive UI layout for your app using ConstraintLayout. Define the activity of your app as resizable, to ensure the maximum compatibility of your app with both the cover display and the main display of the device. Set the resizableActivity attribute to true in Manifest.xml. <activity android:name=".MainActivity" android:resizeableActivity="true"> … </activity> Responsive layout of the Spotify app
      Cutout and punch hole of the main display
      The main display of a Galaxy Z Fold is covered by an area on the top-right side called the “L-cut” whereas the Galaxy Z Fold2 and Fold3 have a punch hole in the upper right side and the Galaxy Z Flip devices have a punch hole in the middle. Some portion of your app’s UI could be covered by the L-cut or the punch hole.
      Content is covered by the L-cut in a Galaxy Fold device in landscape mode
      To avoid such a scenario, depending on your UI content style, define a display cutout mode. For example, the content is letterboxed in landscape mode whereas it is rendered into the cutout area in portrait mode by default. Define the display cutout mode in the style.xml as shortEdges so that your content is rendered into the cutout area in both portrait and landscape modes.
      <item name="android:windowLayoutInDisplayCutoutMode">shortEdges</item> Display cutout in the default mode and the shortEdges mode, respectively
      Last but not the least, you can test your app on our device cloud, Samsung Remote Test Lab, to make sure you have implemented all the checkpoints discussed in this blog. You can also participate in our Codelab challenges to have a clear understanding of the implementation details.
      In this blog we have discussed about how to adapt your app for foldable devices. We hope it is a good guide for you to start with. You can also check out Samsung’s official documentation and reach out to our developer community if you have any queries.
      View the full blog at its source
    • By STF News
      Are your apps ready for the recently unveiled Galaxy Z Fold3? For your app to work seamlessly on foldable devices, it will need to correctly handle App Continuity and Flex Mode.
      With Remote Test Lab, you can test your apps on a real Galaxy Z Fold3. To begin, create an account or login on Samsung Developers, and then visit this link to access the Galaxy Z Fold3.
      Other recent devices, such as the Galaxy Tab S7+, Galaxy S21 Ultra, and Galaxy Note20 Ultra are also available on Remote Test Lab.
      View the full blog at its source
    • By STF News
      Last year, we told you that we are transitioning to an API-first SmartThings platform. As a part of this, we started investing deeply in improving stability and security. We are excited to announce the beta release of SmartThings Edge, a new architecture for hubs that uses device drivers and rules to execute hub-connected devices locally.
      What is SmartThings Edge?
      We are taking smart home events that would have happened on the SmartThings Cloud and moving them to your SmartThings Hub. SmartThings Edge uses Lua©-based device drivers and our Rules API to control and automate devices connected directly to a SmartThings Hub. This includes Zigbee, Z-Wave, and LAN devices as well as automations triggered by timers and other hub-connected devices using drivers. In the future, this will expand to include more protocols and features, like the new Matter standard.
      For users, edge processing means reduced latency. For partners and developers, SmartThings Edge also brings new benefits, including reduced cloud costs, an improved device permissioning model, and the ability to deploy updates to their drivers. Local device support is open to everyone—not just WWST partners—enabling easier device support and integration with standard capabilities.
      Hub-Connected devices are defined by a driver package that includes the fingerprints, components, preferences, and Lua source code. With this beta release, you can start testing features and providing us feedback before we start the platform migration. As a WWST partner, you can create your own driver package or add your device’s fingerprint to our predefined supported drivers.
      Building Hub-Connected Devices

      To get started, all you need is a SmartThings Hub with firmware version 000.038.000XX or greater and a Zigbee, Z-Wave, or LAN device. Visit the SmartThings Community to learn how to get started with the SmartThings CLI and building drivers.
      Rules and Automations

      As part of our June SmartThings app release, you may have noticed that some of your automations show a little 🏠. This means the automation is using locally-executing conditions and actions, such as time of day. As your devices begin using drivers and compatible rules, they will begin running on your hub, meaning higher reliability and faster response times.
      We have started expanding the Rules API to include new operations, including: risesAbove / dropsBelow, risesToOrAbove / dropsToOrBelow, and securityState. To learn more, visit these sample rules.
      Sharing and Distribution
      Previously, sharing device type handlers (DTH) required end-users to copy and paste the code for each individual DTH into their accounts. This could be confusing and time-consuming for users to ensure they had the latest updates installed. With SmartThings Edge, we are also introducing driver sharing for device testing and community distribution.
      OEMs can submit their drivers to SmartThings for WWST certification and distribution in the SmartThings catalog. However, hobbyists can now share custom devices with a simple link—no need to copy and paste code! Learn more about driver sharing and how you can start testing Edge drivers.
      Getting Started

      Along with this launch, we’ve also released a new Developer Documentation Portal.
      To learn more about building SmartThings Edge devices and automations, check out our new developer documentation portal. This includes a new Capabilities Reference Guide that clearly delineates between live, proposed, and (soon-to-be) deprecated capabilities. Visit the new Documentation Portal and the SmartThings Community for tutorials, code samples and more.
      Have feedback? Reach out to us on the SmartThings Community and share your projects with @SmartThings on Twitter!
      View the full blog at its source
    • By STF News
      Whether you are making a cryptocurrency exchange or a game on the Ethereum Blockchain, if you want to get more users for your Web Dapp, this article is for you.
      More web traffic now comes from smartphones than PCs. So, if you want to get more users for your Dapp, you need to take your application to the “Galaxies” (that is, Samsung Galaxy devices)! Thankfully, the Samsung Blockchain Platform (SBP) SDK has got you covered for all of your Blockchain needs.
      This article explores how to connect to a hardware wallet using SBP SDK and how to leverage that connection to sign your cryptocurrency transaction. You will learn how to use Cucumber Webview provided by SBP SDK to display your Web Dapp and leverage SBP SDK features. You will also explore how to send the signed transaction to a Blockchain network and receive payment. Let’s get started!
      Initialize the Samsung Blockchain Platform SDK
      Before using SBP SDK, make sure to include the required supporting libraries. To learn more about the required libraries and how to add them to the project, you can review the SBP SDK Getting Started guide.
      To begin, initialize the SBP SDK for our application. Running the following code segment initializes the SBlockchain object. You need to use this object for all further communication with the SBP SDK.
      try { mSBlockchain = new SBlockchain(); mSBlockchain.initialize(this); } catch (SsdkUnsupportedException e) { handlePlatformSDKUnsupportedError(e); } Connect to a hardware wallet
      Many Samsung Galaxy devices, such as the Galaxy Note20 and S20, already have a hardware wallet available: the Samsung Blockchain Keystore. You will connect to it in this demonstration. However, you can adapt the following code to connect to other supported hardware wallets, such as the Ledger Nano X, by simply changing the hardware wallet type.
      private HardwareWallet connectedHardwareWallet; ListenableFutureTask<HardwareWallet> connectionTask = mSBlockchain.getHardwareWalletManager().connect(HardwareWalletType.SAMSUNG, true); connectionTask.setCallback(new ListenableFutureTask.Callback<HardwareWallet>() { @Override public void onSuccess(HardwareWallet hardwareWallet) { connectedHardwareWallet = hardwareWallet; setupAccounts(connectedHardwareWallet); } @Override public void onFailure(ExecutionException e) { Log.e(LOG_TAG, "Connecting to Hardware Wallet failed."); Log.e(LOG_TAG, e.getMessage()); } @Override public void onCancelled(InterruptedException e) { Log.e(LOG_TAG, "Connecting to Hardware Wallet cancelled."); Log.e(LOG_TAG, e.getMessage()); } }); Once successfully connected to the hardware wallet, you can retrieve the accounts associated with it.
      Retrieve Ethereum accounts
      Using the SBP’s account manager, you can retrieve the list of accounts associated with a hardware wallet.
      AccountManager accountManager = mSBlockchain.getAccountManager(); List<Account> accountList = accountManager.getAccounts(connectedWallet.getWalletId(), COIN_NETWORK_INFO.getCoinType(), COIN_NETWORK_INFO.getNetworkType()); In order to get a list of accounts using SBP SDK, you need to specify the wallet ID of the hardware wallet connected, the coin type (cryptocurrency) you want to use (as of writing this article, only Ethereum is supported), and the desired network type (such as MAINNET or ROPSTEN).
      If the account list is empty, the account manager can generate a new account based on the connected hardware wallet, the specified cryptocurrency, and the network.
      accountManager.generateNewAccount(connectedHardwareWallet, COIN_NETWORK_INFO).setCallback( new ListenableFutureTask.Callback<Account>() { @Override public void onSuccess(Account newAccount) { defaultAccount = newAccount; showAccountAddressOnUI(defaultAccount.getAddress()); initializeWebView(); Log.d(LOG_TAG, "Account fetched."); } @Override public void onFailure(ExecutionException e) { Log.e(LOG_TAG, "Account fetching failed."); Log.e(LOG_TAG, e.getMessage()); } @Override public void onCancelled(InterruptedException e) { Log.e(LOG_TAG, "Account fetching cancelled."); Log.e(LOG_TAG, e.getMessage()); } }); After the account is retrieved, you can proceed to loading our Web Dapp using Cucumber WebView.
      Initialize Cucumber WebView
      The next step is to prepare the Cucumber WebView and enable it to make transactions through the Dapp.
      dAppWebView = findViewById(R.id.dapp_web_view); dAppWebView.addCoinServiceConnector(COIN_NETWORK_INFO.getCoinType(), ethereumCoinService, defaultAccount, transactionListener); To communicate with the blockchain network, use the CoinService interface of the SBP. The coin service enables us to retrieve the information needed for the transaction, such as nonce and gas prices. After all the information for the transaction has been retrieved, the coin service can help us upload the signed transaction to the blockchain network.
      Browser-Based Web Dapps built using web3JS nowadays use Metamask as their web3 provider. When our Dapp is loaded on the dAppWebView, the SBP SDK works as the web3 provider and forwards the web3JS-prepared transaction to a BaseOnSendTransactionListener event handler, which handles the transaction created by our Web Dapp.
      After preparing the transaction, a payment intent powered by the SBP SDK can be launched, which provides an interactive UI with the transaction details and a mechanism to select the preferred transaction speed.
      CoinService ethereumCoinService = CoinServiceFactory.getCoinService(MainActivity.this, COIN_NETWORK_INFO); BaseOnSendTransactionListener transactionListener = new OnSendEthereumTransactionListener() { @Override public void onSendTransaction(String requestID, EthereumAccount ethereumAccount, String toAddress, BigInteger value, String data, BigInteger nonce) { Intent paymentIntent = dAppWebView.createEthereumPaymentSheetActivityIntent(MainActivity.this, requestID, connectedHardwareWallet, toAddress, value, data, nonce); MainActivity.this.startActivityForResult(paymentIntent, 0); } }; dAppWebView.addCoinServiceConnector(COIN_NETWORK_INFO.getCoinType(),ethereumCoinService, defaultAccount, transactionListener); dAppWebView.loadUrl(MARKETPLACE_URL); Our Cucumber WebView (dAppWebView) is now ready to load our Web Dapp using its URL.
      dAppWebView.loadUrl("https://www.example.com"); When the marketplace has loaded, the SBP prompts the user to allow the Dapp to connect to our hardware wallet.
      The SBP’s payment intent UI enables the user to easily purchase items from the marketplace.
      Using the payment intent UI, the user can choose the transaction speed and send the transaction to their hardware wallet for signing. Once signing is done, the SBP SDK sends the transaction to the blockchain network for processing.
      Bonus: display the transaction ID
      In addition to enabling users to sign in and send a transaction with your Dapp, the SBP SDK can also retrieve the transaction ID.
      The payment intent returns the transaction Information, which can be parsed using the onActivityResult method, if desired.
      @Override protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) { super.onActivityResult(requestCode, resultCode, data); if (requestCode == 0 && data != null) { dAppWebView.onActivityResult(requestCode, resultCode, data); switch (resultCode) { case Activity.RESULT_OK: String transactionID = data.getStringExtra("txid"); Log.d(LOG_TAG, "TransactionId : " + transactionID); showAlertDialog("Transaction Successful with Id : " + transactionID); break; case Activity.RESULT_CANCELED: Log.d(LOG_TAG, "Transaction canceled by user."); break; default: Log.d(LOG_TAG, "Unknown Activity Result Code. Result Code: " + resultCode); } } } Conclusion
      With the SBP SDK, your Dapp is no longer confined to PCs. Both new and existing users can have your Dapp at their fingertips on Samsung Galaxy devices. It’s also easy to add support for new cryptocurrencies and hardware wallets to your Dapp; the SBP has you covered. Submit your application to the Samsung Galaxy Store and reach millions of new users today!
      Additional resources
      Download the Samsung Blockchain Platform SDK Example App
      More information on Samsung Blockchain Platform SDK
      View the full blog at its source
    • By STF News
      Looking to show off your design chops? You can now submit your portfolio to Samsung and apply to become a Galaxy Watch face or Galaxy Themes seller. The submission window is open October 14 - October 27.
      Galaxy Watch
      Check out our latest how-to blog and read up on the new process for watch face design reviews. Then head over to the submission page and complete your application.
      Galaxy Themes
      Watch our video for an overview of the process and what you need to prepare. Then download the Themes Submission Starter Kit and let the creative juices flow. Once you've assembled everything, complete your application here.
      While you're working on your portolfio, listen to Tony show Diego how to use Galaxy Themes Studio. It will livestream on Thursday, October 15 at 10am PT and will be available on YouTube for replay.
      Submit Now
      The application process is very selective, and only the best of the best are selected. Think you have what it takes? We can't wait to find out.
      View the full blog at its source
×
×
  • Create New...