Differences between revisions 30 and 44 (spanning 14 versions)
Revision 30 as of 2011-03-01 13:28:00
Size: 10561
Editor: shoobe01
Comment:
Revision 44 as of 2013-03-13 16:36:15
Size: 16047
Editor: shoobe01
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
A method must be provided to input text that is simple, natural, requires little training and can be used in any environment. You must provide a text and graphic input method that is simple, natural, requires little training and can be used in any environment.

'''Pen Input''' requires hardware to be dedicated to the input method. When designing for pen devices, you must consider what input methods will need to be adjusted to work best with them. Many pens have hover states, so this would need to be added to existing touch designs.
Line 6: Line 8:
Pen interfaces can be an excellent alternative to keypads or touch screens when touch is unavailable due to environmental conditions (gloves, rain), when the device is used while on the move, or just to provide additional precision and obscure less of the screen. Pen input used to be the norm for touch-sensitive screens, when relatively low-resolution resistive was the norm; capacitive touch screens have moved to direct touch and multi-touch inputs, but pens as an add-on to touch are making a slow resurgence. Writing is a function inherently performed by the use of a tool. Until the development of typing, this was exclusively done with the stylus, then the pen and pencil.
Line 8: Line 10:
Text input is particularly suited to pen devices, especially for populations (or cultures) with little or no experience with keyboard entry, for no standard keyboard for the data type or for the above mentioned environmental conditions. In situations where only one hand may be used to type, character entry by pen can often be must faster that the use of a virtual keyboard. Pen interfaces can be an excellent alternative to keypads or touch screens when touch is unavailable due to environmental conditions (gloves, rain), when the device is used while on the move, or just to provide additional precision and obscure less of the screen. Pen input used to be the norm for touch-sensitive screens, when relatively low-resolution resistive input was the norm; capacitive touch screens have moved to direct touch and multi-touch inputs, but pens as an add-on to touch are making a slow resurgence.
Line 10: Line 12:
Other systems, including newly-developed capacitive screens, can use specialized pens to isolate touch input, allowing the user to more naturally interact with the device by resting their hand on the screen. Mode switching is also available, either automatically or manually, to Text input is particularly suited to pen devices, especially for populations (or cultures) with little or no experience with keyboard entry, when there is no standard keyboard for the data type, or for the above mentioned environmental conditions. In situations where only one hand may be used to enter the text, character entry by pen can often be much faster that the use of a virtual keyboard.

Other systems, including newly-developed capacitive screens, can use specialized pens to isolate touch input, allowing the user to more naturally interact with the device by resting their hand on the screen. Mode switching is also available, either automatically or manually, to allow either input method.
Line 18: Line 22:

* '''Word entry''' – An area is provided where users may input gestural writing, as they would with a pen on paper. Printed characters are the most common, but script styles should also be supported. The device will use the relationships between characters to translate the writing to complete words, based on rules associated with the language, and will automatically insert spaces between words. The input language, and sometimes a special technical dictionary, must be loaded.
 * '''Word entry''' – Provide an area where users may input gestural writing, as they would with a pen on paper. Printed characters are the most common, but script styles should also be supported. The device will use the relationships between characters to translate the writing to complete words, based on rules associated with the language, and will automatically insert spaces between words. The input language, and sometimes a special technical dictionary, must be loaded.
Line 25: Line 27:
All available input modes should be made available to the user at all times. The user will generally have to select which mode to use, so a method of switching must be provided. This does not preclude development of a hybrid approach, or even simply appropriately selecting which mode is most suitable to the current situation. If a gestural shorthand is available, it should be offered as a part of the natural character input method; both will be accepted, and the user may change the input method per character. All available input modes should be made available to the user at all times. The user will generally have to select which mode to use, so you will have to provide a method of switching. This does not preclude development of a hybrid approach, or even simply appropriately selecting which mode is most suitable to the current situation. If a gestural shorthand is available, it should be offered as a part of the natural character input method; both will be accepted, and the user may change the input method per character.
Line 27: Line 29:
In certain cases, such as where the note taking must be as fast as possible, or the user is combining images, formulas and other data with words, a mode or application may be provided that will store the gestures without live translation. Later, the user may select all or part of the gestures to translate. This behavior should follow the pattern covered here when similarities arise, in order to preserve a consistent interface. Portions will, of course, be different. In certain cases, such as where the note taking must be as fast as possible, or the user is combining images, formulas and other data with words, you may provide a mode (or a whole application) that will store the gestures without live translation. Later, the user may select all or part of the gestures to translate. This behavior should follow the pattern covered here when similarities arise, in order to preserve a consistent interface. Portions will, of course, be different.
Line 34: Line 36:
When a field that can accept character entry becomes in focus, the input panel should open to the last used mode. This mode may be a virtual keyboard, but as the modes may be switched between, the principle applies.
Line 36: Line 40:
To provide sufficient space for the user to write, scrolling must be allowed for within the input panel. When the panel is filled, the system may automatically scroll, or the user may be allowed to scroll manually instead, to enable review and correction. Avoid dynamically adding additional space to the input panel, as this will just obscure more of the screen. To provide sufficient space for the user to write, you must allow scrolling within the input panel. When the panel is filled, the system may automatically scroll, or the user may be allowed to scroll manually instead, to enable review and correction. Avoid dynamically adding additional space to the input panel, as this will just obscure more of the screen.
Line 38: Line 42:
WHEN DONE??? WINDOWS MAKES YOU ACCEPT THE INPUT, LIVE TYPING MAKES SOME SENSE THOUGH ALSO. PRESENT BOTH. At least two methods are available for actions when input is completed.
 * '''Manual''' – The user completes their entry in the input panel, and presses a button to load the content into the field. The next field may be entered, or the form submitted.
 * '''Automatic''' – The system loads characters or words into the input field as they are completed, or after a brief pause to allow the user to make immediate corrections. This may be used to eliminate the need for scrolling the input panel, as it only needs to support one long word, at most.
Line 40: Line 46:
Some provision must be made for functions which are not visible characters, such as line feeds, ok/enter, backspace and tab. These should be keys visible as part of the input panel. See the '''[[Keyboards & Keypads]]''' pattern for additional information on key placement; the handwriting area of the input panel may be considered equivalent to the character keys, in this sense. Mode switches may either commit all completed characters and words to the input field, or switch them to the new entry mode. As discussed in the '''[[Cancel Protection]]''' pattern, never discard user entered information.
Line 42: Line 48:
Gestures may also be supported for the user to more quickly input key functions. However, these will require learning, so should be used as a secondary method in most cases; continue to provide keys for these functions as well. See the appropriate section for more details on gestures. You must make some provision for functions which are not visible characters, such as line feeds, OK/Enter, backspace and tab. These should be keys visible as part of the input panel. See the '''[[Keyboards & Keypads]]''' pattern for additional information on key placement; the handwriting area of the input panel may be considered equivalent to the character keys, in this sense. Place these other keys in the proper relationship to each other so the user may find them easily.

'''[[On-Screen Gestures]]''' may also be supported for the user to more quickly input key functions. However, these will require learning, so should be used as a secondary method in most cases; continue to provide keys for these functions as well.
Line 46: Line 54:
...Win7 gesture to insert corrections, etc. A robust method for correction must be designed into the system. Do not require the user to manually delete and re-write, but build in approval steps, or gestures to correct existing phrases, even when loaded to the input field. A good path is to allow easy conversion from word to character input, to allow correction of characters that could not be recognized correctly. Avoid training, but do make the system learn the user's input method and accept new words in the dictionary so accuracy improves over time.
Line 52: Line 60:
For very small screens, the input panel may take up essentially the entire screen. In this case, special consideration must be taken to display the text already entered in order to provide context to the user. {{attachment:PenInput-Parallax.png|Parallax for input on screens manifests as a perceived offset between the surface where contact is made and the actual display area. As shown in the illustration, this is because protective overlays, the sensing panel and other materials cause a gap between the two. When viewed straight on, they line up and there is no problem. However, users will not always view directly perpendicular to the screen, and will notice the difference with pen inputs. This is especially troublesome on larger devices, like tablets and kiosks, as the user cannot view all of the screen straight on at the same time. |align="right"}}

For very small screens, the input panel may take up essentially the entire screen. In this case, special consideration must be taken to display the text already entered in order to provide context to the user.
Line 56: Line 66:
... icon to open panel...


Any delays in translation or conversion
should be indicated with a '''[[Wait Indicator]]''' so the user is aware the system is still working. Besides the usual risk of the user thinking the system has hung, they may believe the phrase cannot be interpreted, and will waste time trying to correct it.
Any delays in translation or conversion, especially for '''[[Autocomplete & Prediction]]''' methods, should be indicated with a '''[[Wait Indicator]]''' so the user is aware the system is still working. Besides the usual risk of the user thinking the system has hung, they may believe the phrase cannot be interpreted, and will waste time trying to correct it.
Line 63: Line 70:
Cursors must continue to be provided within any input areas on the screen, regardless of the method used. Within the handwriting input panel, a small dot or crosshair should be used to indicate the current pen position. When the pen leaves the area, this cursor must disappear. Be sure to continue providing a cursors within any input areas on the screen, regardless of the method used. Within the handwriting input panel, a small dot or crosshair should be used to indicate the current pen position. This is crucial due to miscalibration and parallax. Both are always present in screen-input devices. For touch devices, the finger is so large the errors are generally not obvious, but pens are very small so reveal these errors or other phenomena.
Line 65: Line 72:
When the pen leaves the input area, this cursor will revert to the normal cursor. When the pen leaves the screen and cannot be detected, the cursor should entirely disappear. Do not use the desktop metaphor of cursor always present, as that is based on the pointing device used. Mice are "relative" devices, and always provide input from the last location, so the cursor is critical. Pens (and fingers) are "absolute" devices, and point wherever they are placed on the screen.

If a hardware '''[[Directional Entry]]''' key is provided which can move the cursor, this is again a relative pointer, and the cursor may re-appear, from it's last location, when this input is detected..
Line 67: Line 77:
Be careful using a pen for large stretches of input on kiosks or where the user cannot rest his hand or arm. The "gorilla arm" effect can impose fatigue or even repetitive stress injury. Allowing the user to rest their hand or arm on the input device, as well as more casual use or regular changes in the type of entry can avoid this.
Line 76: Line 88:



----
Next: '''[[Mode Switches]]'''
----
= Discuss & Add =
Please do not change content above this line, as it's a perfect match with the printed book. Everything else you want to add goes down here.

== It's a stylus ==
Since I've already seen Samsung fanbois (who knew?) discussing the Galaxy Note and saying "it's not a stylus, it's an S-pen, which does so much more..." and always like forum jerks, let's make that clear: It's not. A stylus is a stylus. Or pen. I don't much care which term you use. Right now (really, I type with the Wacom pen in my fingers) I am holding a stylus with pressure, height, tilt and three optional buttons. It's still just a stylus. Your brand name doesn't make it something else, so cut it out.

== History of Handwriting Recognition ==
Buried in a very long article about Palm OS is a very long intro to handwriting recognition. Neat stuff.
http://www.osnews.com/story/26838/Palm_I_m_ready_to_wallow_now/page2/
Line 78: Line 106:

The pen input editing controls on Windows 7 (Windows 8 prototypes are very similar). You can use gestures to correct entry, all with the pen. Useful.
{{attachment:6378874745_655184fd65.jpg|The pen input editing controls on Windows 7 (Windows 8 prototypes are very similar)}}
http://www.flickr.com/photos/shoobe01/6378874745/sizes/m/in/photostream/

== Make a new section ==
Just like this. If, for example, you want to argue about the differences between, say, Tidwell's Vertical Stack, and our general concept of the List, then add a section to discuss. If we're successful, we'll get to make a new edition and will take all these discussions into account.

Problem

You must provide a text and graphic input method that is simple, natural, requires little training and can be used in any environment.

Pen Input requires hardware to be dedicated to the input method. When designing for pen devices, you must consider what input methods will need to be adjusted to work best with them. Many pens have hover states, so this would need to be added to existing touch designs.

Solution

Writing is a function inherently performed by the use of a tool. Until the development of typing, this was exclusively done with the stylus, then the pen and pencil.

Pen interfaces can be an excellent alternative to keypads or touch screens when touch is unavailable due to environmental conditions (gloves, rain), when the device is used while on the move, or just to provide additional precision and obscure less of the screen. Pen input used to be the norm for touch-sensitive screens, when relatively low-resolution resistive input was the norm; capacitive touch screens have moved to direct touch and multi-touch inputs, but pens as an add-on to touch are making a slow resurgence.

Text input is particularly suited to pen devices, especially for populations (or cultures) with little or no experience with keyboard entry, when there is no standard keyboard for the data type, or for the above mentioned environmental conditions. In situations where only one hand may be used to enter the text, character entry by pen can often be much faster that the use of a virtual keyboard.

Other systems, including newly-developed capacitive screens, can use specialized pens to isolate touch input, allowing the user to more naturally interact with the device by resting their hand on the screen. Mode switching is also available, either automatically or manually, to allow either input method.

Pen input devices may be used for multiple types of interfaces, and will often be the primary or only method for the device. This pattern covers only character and handwriting input and correction. Although any pen input panel should also support a virtual keyboard mode, this mode is covered under the Keyboards & Keypads pattern.

Word entry with pens will appear in an input panel, occupying part of the screen. Options such as the mode switches and special characters appear as buttons within this panel. Translation candidates appear beneath the written word, and may be explicitly accepted or automatically loaded after a pause.

Variations

Pen input for text entry may fall into one of two modes:

  • Word entry – Provide an area where users may input gestural writing, as they would with a pen on paper. Printed characters are the most common, but script styles should also be supported. The device will use the relationships between characters to translate the writing to complete words, based on rules associated with the language, and will automatically insert spaces between words. The input language, and sometimes a special technical dictionary, must be loaded.

  • Character entry – The input panel is broken up into discrete spaces in which individual characters may be entered. This may be accomplished by simply dividing up the word entry area with ticks or lines, or may present a series of individual input panels. The device will translate individual characters, and will generally disregard the relationships between adjacent characters. The user is responsible for spelling words correctly, and adding space between words. The language must be set correctly for the fastest recognition, but alternatives within the language family will also be available (e.g. all accented characters for any roman language).

A subsidiary version of the character entry mode uses a shorthand, gestural character set instead. This is not required for most languages with current input and recognition technologies, but may have advantages for certain specialized applications, or for the input of special characters or functions.

All available input modes should be made available to the user at all times. The user will generally have to select which mode to use, so you will have to provide a method of switching. This does not preclude development of a hybrid approach, or even simply appropriately selecting which mode is most suitable to the current situation. If a gestural shorthand is available, it should be offered as a part of the natural character input method; both will be accepted, and the user may change the input method per character.

In certain cases, such as where the note taking must be as fast as possible, or the user is combining images, formulas and other data with words, you may provide a mode (or a whole application) that will store the gestures without live translation. Later, the user may select all or part of the gestures to translate. This behavior should follow the pattern covered here when similarities arise, in order to preserve a consistent interface. Portions will, of course, be different.

The finger may also, on some devices, be used to input gestural writing without a pen. Many of the principles outlined in this pattern will apply, but the size of the finger may obscure the screen and make this sub-optimal.

Letter input divides the input panel into discrete entry areas. Each character displays a translated candidate. One way to handle correction is for optional translations to be offered in a list. If the best guess is incorrect, the user may open the list and select another option.

Interaction Details

When a field that can accept character entry becomes in focus, the input panel should open to the last used mode. This mode may be a virtual keyboard, but as the modes may be switched between, the principle applies.

Whenever the user makes a gesture in the input panel, the path taken is traced as a solid line in real time, simulating pen on paper. As soon as a word (or for character entry, a character) is recognized, it should be offered as a candidate. The translated characters and words are the same as the candidate words as discussed in Autocomplete & Prediction. See that pattern for additional details on presentation, the interaction to select alternatives and on user dictionaries.

To provide sufficient space for the user to write, you must allow scrolling within the input panel. When the panel is filled, the system may automatically scroll, or the user may be allowed to scroll manually instead, to enable review and correction. Avoid dynamically adding additional space to the input panel, as this will just obscure more of the screen.

At least two methods are available for actions when input is completed.

  • Manual – The user completes their entry in the input panel, and presses a button to load the content into the field. The next field may be entered, or the form submitted.

  • Automatic – The system loads characters or words into the input field as they are completed, or after a brief pause to allow the user to make immediate corrections. This may be used to eliminate the need for scrolling the input panel, as it only needs to support one long word, at most.

Mode switches may either commit all completed characters and words to the input field, or switch them to the new entry mode. As discussed in the Cancel Protection pattern, never discard user entered information.

You must make some provision for functions which are not visible characters, such as line feeds, OK/Enter, backspace and tab. These should be keys visible as part of the input panel. See the Keyboards & Keypads pattern for additional information on key placement; the handwriting area of the input panel may be considered equivalent to the character keys, in this sense. Place these other keys in the proper relationship to each other so the user may find them easily.

On-Screen Gestures may also be supported for the user to more quickly input key functions. However, these will require learning, so should be used as a secondary method in most cases; continue to provide keys for these functions as well.

Some pens will have buttons, which the user can activate to bring up contextual menus or which may be programmed to perform other functions. These are not usually noticed so should be considered like gestures, and are only shortcuts to functions which can be found otherwise.

A robust method for correction must be designed into the system. Do not require the user to manually delete and re-write, but build in approval steps, or gestures to correct existing phrases, even when loaded to the input field. A good path is to allow easy conversion from word to character input, to allow correction of characters that could not be recognized correctly. Avoid training, but do make the system learn the user's input method and accept new words in the dictionary so accuracy improves over time.

Presentation Details

Writing is entered into an input panel that appears as a part of the screen. For small screens, this should always be a panel docked to the bottom of the viewport, and should automatically open when an input field is placed into focus. For larger screens, the input panel may be a floating area displayed contextually, adjacent to the current field requiring input. This panel must always be below the input field, to prevent obscuring the information.

Parallax for input on screens manifests as a perceived offset between the surface where contact is made and the actual display area. As shown in the illustration, this is because protective overlays, the sensing panel and other materials cause a gap between the two. When viewed straight on, they line up and there is no problem. However, users will not always view directly perpendicular to the screen, and will notice the difference with pen inputs. This is especially troublesome on larger devices, like tablets and kiosks, as the user cannot view all of the screen straight on at the same time.

For very small screens, the input panel may take up essentially the entire screen. In this case, special consideration must be taken to display the text already entered in order to provide context to the user.

When input is completed, the input panel should disappear to allow more of the page to appear.

Any delays in translation or conversion, especially for Autocomplete & Prediction methods, should be indicated with a Wait Indicator so the user is aware the system is still working. Besides the usual risk of the user thinking the system has hung, they may believe the phrase cannot be interpreted, and will waste time trying to correct it.

If the pen can be detected as a unique item, the input panel may be placed anywhere on the screen. Otherwise the panel must always be placed to avoid accidental input, usually along the bottom of the viewport. Resistive and most capacitive screens will perceive multiple inputs and may not be able to tell the difference between the pen and the user's hand.

Be sure to continue providing a cursors within any input areas on the screen, regardless of the method used. Within the handwriting input panel, a small dot or crosshair should be used to indicate the current pen position. This is crucial due to miscalibration and parallax. Both are always present in screen-input devices. For touch devices, the finger is so large the errors are generally not obvious, but pens are very small so reveal these errors or other phenomena.

When the pen leaves the input area, this cursor will revert to the normal cursor. When the pen leaves the screen and cannot be detected, the cursor should entirely disappear. Do not use the desktop metaphor of cursor always present, as that is based on the pointing device used. Mice are "relative" devices, and always provide input from the last location, so the cursor is critical. Pens (and fingers) are "absolute" devices, and point wherever they are placed on the screen.

If a hardware Directional Entry key is provided which can move the cursor, this is again a relative pointer, and the cursor may re-appear, from it's last location, when this input is detected..

Antipatterns

Be careful using a pen for large stretches of input on kiosks or where the user cannot rest his hand or arm. The "gorilla arm" effect can impose fatigue or even repetitive stress injury. Allowing the user to rest their hand or arm on the input device, as well as more casual use or regular changes in the type of entry can avoid this.

The space taken by input panel must be considered by the page display. Do not allow important information to be obscured by the panel. For example, do not make it so the panel must be closed to scroll to the bottom of a form so the submit button may be activated. Consider the input panel to be a separate area instead of an overlay so items may be scrolled into view.

Handwriting recognition can be very fast and may even be approximately as error prone as typing on a keyboard. However, correction methods and other parts of the interface can serve to make the overall experience very much slower. Use the methods described here to reduce clicks and provide automatic, predictive and otherwise easy access to mode changes.

Respect user-entered data, and do whatever is possible to preserve it. Use the principles outlined in the Cancel Protection pattern whenever possible. Especially when user input must be committed before being populated, do not allow gestures to be discarded by modal dialogues or other transient conditions.

Separate entry areas are generally no longer required technically so are not suggested unless there is some specific technical or interface reason that the screen cannot or should be used. If only one screen or portion of the screen is to be used for pen input, assure it is as optimized as possible for this task. Generally, this area may be dedicated to input, and take on all the behaviors of the input panel as described above, when in pen input mode.


Next: Mode Switches


Discuss & Add

Please do not change content above this line, as it's a perfect match with the printed book. Everything else you want to add goes down here.

It's a stylus

Since I've already seen Samsung fanbois (who knew?) discussing the Galaxy Note and saying "it's not a stylus, it's an S-pen, which does so much more..." and always like forum jerks, let's make that clear: It's not. A stylus is a stylus. Or pen. I don't much care which term you use. Right now (really, I type with the Wacom pen in my fingers) I am holding a stylus with pressure, height, tilt and three optional buttons. It's still just a stylus. Your brand name doesn't make it something else, so cut it out.

== History of Handwriting Recognition == Buried in a very long article about Palm OS is a very long intro to handwriting recognition. Neat stuff. http://www.osnews.com/story/26838/Palm_I_m_ready_to_wallow_now/page2/

Examples

http://www.visionobjects.com/en/myscript/text-input-applications/myscript-stylus-mobile/key-benefits/

The pen input editing controls on Windows 7 (Windows 8 prototypes are very similar). You can use gestures to correct entry, all with the pen. Useful. The pen input editing controls on Windows 7 (Windows 8 prototypes are very similar) http://www.flickr.com/photos/shoobe01/6378874745/sizes/m/in/photostream/

Make a new section

Just like this. If, for example, you want to argue about the differences between, say, Tidwell's Vertical Stack, and our general concept of the List, then add a section to discuss. If we're successful, we'll get to make a new edition and will take all these discussions into account.

Pen Input (last edited 2013-03-13 16:36:15 by shoobe01)