The Relationship

Let’s travel down memory lane. Take a moment to think back when you had your first date. Stop! Just kidding. That’s outside the scope of this book. But, do take a moment to bring up memories of when you bought your first cell phone. For some of us, that journey was a bit longer than others. Even though mine was some time ago, I clearly remember the exact year, the model, and reason why I bought it in the first place.

The year: 1997. College.

The model: Motorola StarTAC, 2G GSM. 4 x 15 character, monochrome graphic display.

The reason: Cool factor! A flip phone, and the smallest cell phone available. I could send and receive calls, SMS, and store up to a 100 contacts. And I could show it off ever so smoothly when I clamped in onto my belt. Back then, that was all of the functionality I could ever want! It was love at first sight!

The Break-up

Well, today, that phone is no longer with me, sad to say. I’ve had to keep up with the times and the technology. Since that StarTAC, I’ve gone through an extensive amount of mobile phones. I believe the count total now is eight. That number seems reasonable with the rate at which people get a new mobile phone today -– a new phone every 18 months.

Today, my mobile needs of having a device to assist me managing my life require greater interactive control and highly visible functionality on a powerfully, crisp and color display. That original 4 x 15 monochrome graphic display used today, would be quite limiting and unsatisfying.

I’m not "everyman"

Despite my mobile needs, they’re not everyone’s. Mobile design is never about us. It’s about all the other people who are using a range of multiple devices, with varying needs in limitless contexts.

Here are the things we must consider relating to this Screens, Lights & Sensors chapter in order to create an enriching user experience while considering everyone.

Context of Use

One of the mobile design principles centered around this book is that mobile devices must work in all contexts. They must function properly and behave appropriate in variety of environments. Let’s consider the following contexts that can affect the way we interact with the mobile display.

Outdoors

The outdoors is the most complicated environment to design around. It’s highly unpredictable, constantly dynamic, and uncontrollable. External stimuli such as bright sunlight, cloudy days, moon light, darkness, street lights aren’t controlled by the user. We can’t just switch on and off the sun or blow the clouds away. All of these stimuli can make it more difficult to view the screen affecting the way our users interact with the device.

Indoors

The indoor environment may be more predictable, less dynamic, and controllable than the outdoors, but it is still highly complicated. External stimuli present may be natural light from windows, doors, skylights, or generated light from bulbs: florescent, incandescent light, LED, halogen, high-pressure sodium lamps.

Both

Mobile users are constantly transient which changes their environment. They may be using their device outside in daylight to read the news, while then walking inside a public building with dim lighting.

This change affects the amount of time it takes our rod receptors in the back of our retina to respond to light differences. The greater difference between the two environments, the longer it will take our eyes to appropriately adjust. This affects our ability to quickly detect and identify details in that time – such as text, small images, and even controls.

Imagine this scenario: A user takes his mobile smartphone into a movie theater. The lights inside are very dim. He wakes his phone to check the time. The screen display is excessively bright causing him to look away. This is not just a screen display technology issue, it’s a UI design problem.

Here are two solutions:

  1. Have the automatic brightness sensor picks up the ambient light around you and the phone (not your face, which typically occurs). It recognizes the low level of light and automatically dim’s the screen to an appropriate level to minimize eye strain and maximize viewability.

But not all users like automatic brightness on, partly as it can rapidly drain the device’s battery.

  1. Provide immediate access to brightness controls. Rather than have them buried in a system setting, consider using the physical keys (ex. volume) that can open up a menu to control the display settings. These physical controls have eyes-off functionality, and can be interacted with, without even looking at your device.

Displays

Mobile devices displays can range in size, resolution, and pixel density (ppi). As a mobile designer and developer, it’s helpful that you are become familiar with these differences so you can make appropriate decisions throughout the design process. Depending on the requirements of the project, you may be designing for one particular device or multiple devices with varying displays.

Screen sizes

Display sizes vary with the type of mobile device. They are measured diagonally from the display’s corners. Typically, smartphones have larger screen sizes then feature phones. A common misconception with mobile devices is that the screen sizes limit their ability to see the screen.

However, people automatically adjust the visual angle between the device and our eyes. Entirely aside from phones being hand-held, they get moved to the right distance to see things correctly.

Broadly speaking, video display ends up occupying the same field of view regardless of the device size. If we watch a movie on a 46” HDTV, we will distance ourselves enough so we are able to see the entire TV in our visual field. That visual angel can be similar when we view a movie on a phone. This is why people actually do watch video on phones, and will continue to do so.

Other factors that affect how we automatically adjust our visual angle to see the screen include:

Screen resolutions

The screen resolution is determined by the total number of physical pixels on a screen. Note: The actual size of a pixel varies across devices as well as they type of technology being used in the display. So it can be very misleading when people say use a minimum touch target size of 44 px. For more information on proper use of touch target size, refer to the General Touch Interaction Guidelines found in the Input & Output Section.

Below are common screen resolutions (pixels) across mobile devices: feature phones, smartphones, and tablets.

Small- 128, 176, 208, 220

Medium - 240, 320

Large - 320, 360, 480+

Tablet – 600, 800, 768, 1024

Pixel density

Pixel density (ppi) is based on the screen resolution. It equals the number of pixels within an area, such as a square inch (units do vary, so refer to numbers you find very carefully).

A screen with a lower pixel density has fewer available pixels spread across the screen width and height. A screen with a higher density has more pixels spread across the same area.

When designing for mobile displays, it’s important to be aware of the targeted device’s pixel density. Images and components that are designed in a particular height and width in screen pixels, appear larger on the lower density screen and smaller on the higher density screen. Objects that become too small will affect legibility, readability, and detail detection.

Mobile Display Technologies

Mobile devices use a range of display devices. Some devices may use multiple types of hardware within a single device. Each technology can serve a unique purpose in functionality – backlight, primary display, flashing indicators.

As a designer and developer, it’s important to understand that each of these technologies have limitations such as affecting the device’s battery life, lifespan, creating glare, or restricting orientation modes. Therefore, we need to create UIs that can maximize the user experience around these limitations. Below are descriptions of some common types.

OLED and AMOLED displays are lit-pixel displays (without a backlight). White text on black will, unlike a backlit display, use much less power than black text on white. This may be a key design consideration for those devices.

Retro-reflectivity

Retro-reflective displays include a reflective layer behind the backlight layer. Ambient light coming into the display is dispersed across the surface and reflected back to provide passive, power-free illumination of the display content. This can be used to save power, to provide some view of the display when the device is in sleep mode, or to boost the performance of the display in bright light; instead of fighting the sun with high-power backlight, it is simply reflected back.

In some displays, this is the primary or only method to illuminate the display, and there is no backlight. In this case, the layer is located immediately behind the backplane.

Transflective layers are similar in concept, but use a transmissive/reflective layer which can be placed in front of the backlight. The key problem with these two technologies is that high-density color screens with touch overlays limit the amount of light passing through. Placing the reflective layer as far forward as possible increases the efficiency.

This works only for pixel masking displays, such as LCDs. Lit-pixel displays, such as OLED, cannot use it as they have no backlight -- all pixels are opaque and must provide their own illumination. Although ePaper uses the concept, some technologies also cannot use it directly, and the reflectivity is integral to the "white" component of the display.

Display illumination on devices with reflective layers must take this into account to provide the most suitable illumination, and to extend battery life when not needed. In sleep states, the lock screen must continue to display useful information since it can be read at any time.

Input Overlays

If the display includes a touch or other input function, this is generally attached as a separate layer on top of the display screen. This is a problem in two ways. First, every layer over the display plane adds depth, which increases the parallax; this

Second, nothing is truly transparent. Stacking several, with different indices of refraction as well, can significantly reduce the brightness the user is able to perceive. Any touch screen will be dimmer than the same hardware without a touch screen.

Touch and other input layers are frequently sold as an integrated package by component manufacturers. There are now, therefore, solutions such as Super AMOLED, which allow the sensing layer to be thinner and more optically integrated than separately-developed components.

Sensors

Another important concept that will be discussed later in this chapter are location based services used by operators and GPS satellites. These services identify tracking sensors in your mobile device to determine your location with varying degrees of precision and accuracy. Location services can provide an enriching user experience for a multitude of reasons.

They can work tangently with your devices OS and existing applications to trigger notifications or create appropriate device state changes without compulsory user interaction.

Example: You are driving to the airport to catch a flight. Your mobile “Itinerary travel application” has you scheduled for a 3:00pm departure at ORD.

As you get within a particular range of the airport, location services identify your current location and proximity to the airport. Sending that information you’re your Itinerary application, a notification pops-up mentioning the latest departure time, and any occurring gate changes.

Screens from the Motorola StarTAC, Sanyo SCP-6400, Samsung Rant and Nokia N8, showing evolution from low-resolution, monochromatic LCD through OLED. Over time, screens have increased in size, color depth and resolution. Interfaces and interactions have always pushed the requirements of new screens, and taken good advantage of the technologies.

Low power screens, or low-power modes, must make tradeoffs in display color or detail. LED screens, such as the Casio Gz'One external display and the N8 sleep screen, use less power due to only lighting some pixels. ePaper, as shown in part with the Sony Touch, only requires power when changing the pixels, but can only show one color at moderate resolution.

Screen settings for the Garmin GPSmap60 are opened along with volume, when the power button is tapped. The Samsung Galaxy-S uses a typical settings panel, buried under the settings, but also ships with a settings widget on the home pages. Changing the brightness on the Samsung SPH-630 featurephone happens on a page covered in color and grayscale bars, to help determine the appropriate level.

Patterns for Screens, Lights & Sensors

The patterns within this chapter describe how Screens, Lights, and Sensors can be used to increase visual and device performance, provide additional notifications, and incorporate location base services with native applications. The following patterns in the chapter will be discussed.

LED– Notice of status should have a visual component that does not require use of the primary display.

Display Brightness Controls – Displays are the most critical output on smartphones. A method to control the display settings must be easily accessible, easy to use, and allow for manual adjustments.

Orientation – Devices must allow for orientation changes to display content in the manner most readable and comfortable to the user.

Location – Availability of location based service, and the actual location of the handset must be easily and accurately communicated.