A Drupal developer's guide to Progressive Web Apps

The following article is a companion to my presentation at Drupalcon and Drupalcamp covering Progressive Web Apps implementations.

Progressive Web Apps (PWA) have support from some of the top tech companies, including Google and Microsoft, with the common goal being "Web apps should be able to do anything iOS, Android, or desktop apps can." PWAs can add value to businesses at a variety of different stages. All projects have limitations, whether they be development resources, timeline, budget, or technical debt. Even with "unlimited resources," developing an app from a single codebase, using commonly known web technologies, allows for a more frictionless, sane release cycle.

Disclaimers:

• PWA is a collection of different techniques combined in a web browser to create an "app-like" experience.
• This information is from an architect's perspective for choosing and implementing various technologies that come together to build a product.
• Below is a high-level end-to-end outline of a path to launch a Drupal website on the app stores. Each section could be its own in-depth blog post.
• The techniques are written with Drupal in mind, but you can apply many of them to all web apps.

What is a PWA?

Image by:

(Alex Borsody, CC BY-SA 4.0)

Benefits of a PWA implementation:

• Increased Lighthouse score and SEO.
• Single codebase.
• Frictionless testing.
• Instant feedback loop for development cycles.
• Use of existing PaaS deployment workflows, including Acquia, Pantheon, Platform.sh etc.
• Use of web technologies that are a familiar skillset for a wide array of developers.
• Provides the only cross-platform development solution that delivers a full-fledged desktop experience.
• Offers unlimited options to customize a design without relying on a cross-platform framework's limited UI components.

This article covers some basic points for PWA deployment. There are many details to consider both at the architect and developer levels. The following topics are discussed:

• PWA minimum requirements and Drupal PWA module as a starting point.
• Publishing on app stores.
• Everything you need to know about making your PWA feel app-like.

PWA module on Drupal.org

The Drupal PWA module is a solution that generates a service worker for caching strategies and offline capabilities. Its secondary functionality also generates a manifest.json, so once installed, it will fulfill the basic requirements of a PWA out-of-the-box.

There is functionality in the module's service worker that provides unique solutions to Drupal-specific behavior, although you can also apply these solutions to apps outside of Drupal.

Image by:

(Alex Borsody, CC BY-SA 4.0)

Offline caching

Offline caching with a service worker is one of the functionalities that defines a PWA.

The following images summarize how a service worker acts as a proxy (sitting between the client and the internet/webserver) to intercept HTTP requests from the browser.

During the first request to the /about page, the browser reaches the network, and upon returning a 200 response from the server, the Javascript service worker calls cache.put() to store the HTML and all assets in the Cache API.

Image by:

(Alex Borsody, CC BY-SA 4.0)

On the second trip, the service worker bypasses the network completely and serves the page from the Cache API store in the user's browser, loading the page instantly. It can also load the page offline.

Image by:

(Alex Borsody, CC BY-SA 4.0)

The browser can precache pages to make them load instantly before the user visits them or even load offline before a visit. However, because in Drupal, the CSS/JS filenames change after compression, the solution must address the problem of identifying these assets before it can precache them via a service worker. It does this by internally requesting the URLs set in the admin panel and extracting assets from the DOM. This allows the service worker install event to fetch all CSS/JS and images from these documents to store in Cache API. The complete pages will then be viewable offline and load instantly, even if the user never visits them first.

Image by:

(Alex Borsody, CC BY-SA 4.0)

Image by:

(Alex Borsody, CC BY-SA 4.0)

Below, I fetch all the assets from the URLs set in the admin panel to inject later into the service worker precache assets array. In D8, I changed the request to use Drupal::httpClient(), which is the updated version of drupal_http_request() in D7 and is a wrapper for the PHP Guzzle library.

 foreach ($pages as $page) {
      try {
        // URL is validated as internal in ConfigurationForm.php.
        $url = Url::fromUserInput($page, ['absolute' => TRUE])->toString(TRUE);
        $url_string = $url->getGeneratedUrl();
        $response = \Drupal::httpClient()->get($url_string, array('headers' => array('Accept' => 'text/plain')));

This code matches all assets needed:

// Get all DOM data.
      $dom = new \DOMDocument();
      @$dom->loadHTML($data);

      $xpath = new \DOMXPath($dom);
      foreach ($xpath->query('//script[@src]') as $script) {
        $resources[] = $script->getAttribute('src');
      }
      foreach ($xpath->query('//link[@rel="stylesheet"][@href]') as $stylesheet) {
        $resources[] = $stylesheet->getAttribute('href');
      }
      foreach ($xpath->query('//style[@media="all" or @media="screen"]') as $stylesheets) {
        preg_match_all(
          "#(/(\S*?\.\S*?))(\s|\;|\)|\]|\[|\{|\}|,|\"|'|:|\<|$|\.\s)#ie",
          ' ' . $stylesheets->textContent,
          $matches
        );
        $resources = array_merge($resources, $matches[0]);
      }
      foreach ($xpath->query('//img[@src]') as $image) {
        $resources[] = $image->getAttribute('src');
      }
    }

Below, you can see the final result in the processed serviceworker.js file that is output in the browser. The variables in the service worker are replaced with the path to the assets to cache.

Image by:

(Alex Borsody, CC BY-SA 4.0)

Phone home uninstall

The module provides another clever piece of functionality—responsible cleanup when uninstalled. The module sends a request back to a URL created by the module. If the URL does not exist, it means the module has been uninstalled. The service worker then unregisters itself and deletes all related caches left on the user's browser.

// Fetch phone-home URL and process response.
  let phoneHomeUrl = fetch(PWA_PHONE_HOME_URL)
  .then(function (response) {
    // if no network, don't try to phone-home.
    if (!navigator.onLine) {
      console.debug('PWA: Phone-home - Network not detected.');
    }

    // if network + 200, do nothing
    if (response.status === 200) {
      console.debug('PWA: Phone-home - Network detected, module detected.');
    }


    // if network + 404, uninstall
    if (response.status === 404) {
      console.debug('PWA: Phone-home - Network detected, module NOT detected. UNINSTALLING.');
// Let SW attempt to unregister itself.
      Promise.resolve(pwaUninstallServiceWorker());
    }

    return Promise.resolve();
  })
  .catch(function(error) {
    console.error('PWA: Phone-home - ', error);
  });
};

Testing notes

Disable the module on dev as it provides an extra caching layer. Any changes pushed to production for CSS or other assets with cache first strategies should be followed by incrementing the service worker version to bust the cache.

You can find additional debugging steps for a service worker on this PWA module documentation page.

Using the Chrome console to remote debug on a mobile device is possible on Android and can be helpful.

2.x version

The 2.x and 7.2x versions port the service worker to Workbox, where you can set caching strategies. Here, setting caching strategies for different asset types and routes is simplified from about 30 lines of code using just the javascript Fetch API to about five lines. Some people may be resistant to libraries, but this is the direction Google is taking with PWAs.

Workbox caching strategies are similar to those in other caching layers such as Varnish. For example, by default, image assets and fonts are set to "cache first," so they are always served instantly. HTML would best be implemented as stale-while-revalidate.

Image by:

(Alex Borsody, CC BY-SA 4.0)

There is also functionality in Workbox, such as background sync, where a failed post request will retry upon coming back online.

Image by:

(Alex Borsody, CC BY-SA 4.0)

For more information on what a service worker can do and all the use cases where it may be helpful, check the W3 Service Workers Demo repo on GitHub.

Get your web app in the app stores

PWA builder is a web application powered by Microsoft where you input your URL and it generates everything you need to submit to the app stores.

For Android, it uses TWA, and for iOS, it wraps your web app in native SWIFT code using WebKit's WKWebView. These are techniques I have been using since 2013, way back when Drupal was a buzzy technology and being used by startups. Businesses that had mobile-optimized Drupal websites wanted them on the app stores. Before Android TWA, developers used Webview, and before WKWebView, there was UIWebView.

Recently PWA builder added a solution for iOS using WKWebView, which confirms my belief that this is the best option to get your PWA into the App Store. Maximilian Firtman also reveals this as the solution in his course "Creating Progressive Web Apps with Vue," which I purchased to see his answer to the problem.

The PWA module provides everything you need to run through PWA Builder:

• For Android, it creates a lightweight .apk/.aap using TWA to submit to the Play Store 800kb.
• For iOS, it wraps your website in WKWebView to submit to the App Store.

A live demo I put together of PWA builder is here. [[EDITORS - MISSING LINK]]

Android and TWA

The Google and Chromium teams are currently the strongest driving forces behind PWAs. Therefore, TWA is designed specifically to get your PWA into the Play Store. On the contrary, WKWebView is essentially a workaround not explicitly supported by Apple. However, WKWebView is extremely powerful, even though Apple doesn't advertise this or have much documentation on its capabilities.

Trusted Web Activity is essentially a Chrome process running in full screen with a status bar and loading screen. The thread is running in the same process as the Chrome app on your phone. For example, if you are logged in on your Chrome browser, you will be logged in on your TWA app. To clear up any possible confusion resulting from this, the TWA team has added a "toast," meaning the first time the user opens the app, a notification shows "Running in Chrome." This only happens the first time the app is installed. This annoyance is enough for some teams to ditch TWA and use the WebView class instead; however, Google discouraged this as you lose out on everything baked into the Chrome web browser.

The main points Google makes about using TWA are:

• Chrome is feature complete.
• Faster than Webview.
• Evergreen (always the up-to-date version of Chrome).

Additional useful functionality.

• Chrome handles frictionless OAuth requests.
• Share cookies, local storage, and saved settings with the preferred browser.

Below is a comparison chart of everything you get when using TWA instead of a Webview wrapper.

Image by:

(Alex Borsody, CC BY-SA 4.0)

Webkit: WKWebView

There are several considerations for publishing on the App Store. WKWebView is essentially a workaround and not a method implicitly endorsed by Apple to launch a native app. Some caveats come with this. The most important is to be mindful of Apple's minimal functionality guidelines.

From my experience, you will be approved if you do everything you can to make your web app "app-like" with useful functionality. Using the Webkit API to enhance your web app is another way to provide additional functionality outside of your website.

One technique is to set a cookie depending on the start_url. For example, add a parameter like myapp.com?ios_app and set a cookie to determine a separate stylesheet or customize logic.

Consider the following sample implementation.

Note: This technique should not be confused with Apple's limited add to homescreen support, which you usually hear about with Apple + PWAs. I won't cover this as it's not the experience a user would expect.

PWA builder provides the minimum functionality required to wrap a website in WKWebView for App Store submission. For features such as biometric or push notifications, you need a custom implementation of WKWebView.

In the graphic below, you can see the source files provided. You can then easily compile your app in XCode and submit it to the app store.

Image by:

(Alex Borsody, CC BY-SA 4.0)

PWA Builder provides:

• No Bounce when scrolling out of view with wKWebView.scrollView.bounces = false
• Service worker support
• Shortcuts URL capture
• Permitted navigation scopes
• Status bar customization
• Splash screen from manifest props
• iOS app awareness from JS code
• Mac Store support

A custom implementation of WKWebView can provide:

• Push notifications: Push notifications are possible by posting the device ID matched to the Drupal UID, which can be extracted from the URL /user/{uid}/edit, for example.
• Biometric: Biometric is implemented on all pages except for user/login and user/register, and the cookie max expiration is extended. Biometric is shown every time the app is closed and reopened.
• WKUIDelegate: Present native UI elements, such as alerts, inputs, or contextual menus.
• evaluateJavaScript(): Execute any Javascript. The possibilities here are endless.
• Password management using associated domains: Placing a public keypair in your /.well-known directory will allow your native app to trust your website and autofill passwords.

View the README.md of WKWebView+, a project I am working on that makes it easy to integrate this enhanced functionality into any iOS PWA.

Cons of WKWebView

Give the following considerations attention before implementing WKWebView:

• There is a paradigm shift in thinking required for frontend developers to debug a PWA correctly. Though it relies on web technologies, there is a learning curve.
• Without a native iOS developer, certain features are not possible to implement. However, WKWebView+ was designed to solve this.
• Though the outlook for Apple and PWAs looks positive, as usual, you are at the mercy of the next Safari release.

Moving forward

Many of the features available with TWA are only available on Chromium-based browsers. Webkit mobile/WKWebView lags. This lag includes push notifications, "add to home screen," and overall web browser standards. Maximilian Firtman's blog is currently one of the best resources for a summary of the updates in the latest Safari, even if they were not announced in the release notes.

The optimistic outlook is that WKWebView is based on the open-source project Webkit, and there is a collaboration among the developers that work on both Chromium and WebKit. Anyone can create an issue and pull request. Often, features already implemented in Chrome have patches submitted to Webkit that do the same.

Make it app-like

Websites that took all the right vitaminsA PWA is essentially a collection of web technologies that combine to make your web experience app-like, as if the website "took all the right vitamins." Below I have identified points that make up a good PWA:

• UX/UI: Visual problem solving is at the core of making your website feel like an app. A great CSS developer with an eye for design and detail, such as animations, input/font sizes, and scrolling issues, is essential.
• Stay current with app-like enhancements: Keeping frontend code updated and compatible across WebKit/Chrome requires research and periodic updates, particularly when a new version of the iPhone is released.
• Implement expanded web capabilities: The Chromium team constantly improves the browser experience. This can be tracked in Project Fugu, the overarching web capabilities project. The closest thing there is to comprehensive documentation on PWAs is on webdev.
• Page speed: I covered caching with a service worker, but there are countless other technologies and techniques.

Some examples of app-like enhancements include using HTML/CSS/JS technologies commonly available to web developers, and then making them frictionless to implement, test, and deploy. You can find a good example of a web application using many of these suggestions here.

Suggestions include:

• Javascript touch events: Disable pinch zoom and add swipe/multitouch gestures.
• CSS:
  • Minify/optimize CSS and apply Lighthouse suggestions.
  • "App-like" input/font sizes and make sure everything fits in the viewport; make it visually look like an app.
  • Tactful use of preloaders.
• Utilize cookies: Set cookie based on app start URL.
• HTML attributes:
  • autocomplete="username"
  • autocomplete="current-password"
  • autocomplete="one-time-code" - WebOTP API
  • input type="tel"
• Ajax API (Drupal specific), Websockets, or SPA framework.
• iPhone specific suggestions:
  • Status bar on iPhone X and Viewport meta tag.
  • Apple-specific meta tags.

Image by:

(Alex Borsody, CC BY-SA 4.0)

Wrap up

PWA brings together different techniques to create an app-like experience in a web browser. I outlined an approach to PWA implementation for a Drupal site, but other options are certainly available with similar designs. What implementations of PWA might help your organization's user experience?

View the README.md of WKWebView+, a project I am working on that makes it easy to integrate this enhanced functionality into any iOS PWA. 

Ionic the spiritual successor to Cordova is a popular framework that also utilizes WKWebView to build native iOS.