Gestures on Mobile Devices
The introduction of the Multi-Touch technologyon mobile devices has changed the way people interact with these devices by introducing the gestures. The gesture metaphor allows a user to interact with the device with "natural" gestures that require the movement of the fingers on the screen.
Tap: a single, brief touch on the screen within a bounded area and back up off the screen again.
Double Tap: two quick taps within a bounded area.
Touch and Hold: a single finger down within a bounded area for a defined period of time.
Swipe: touch the surface with a sweeping motion; or a single finger down moved rapidly in any direction and ends with the finger lifted up.
Drag: a single finger placed down and moved across the screen in any direction; it ends when the finger is lifted from the screen.
Pinch: two fingers down within separate bounded areas followed by the fingers moving closer together (pinch) or further apart (stretch).
Rotate: a motion with two fingers, placed like at the edges of a circumference, that move simultaneously describing a circle.
(point your mouse or tap on the icons to get more info about gesturs)
Whereas embossed buttons, changing cursors and changing targets indicate possible interactions in desktop interfaces, the corresponding indicators for gestures and hence interactions are often missing in mobile devices. As a result, users are often unaware of less obvious interaction possibilities, and may therefore never notice certain functionalities, possibly becoming frustrated because they do not find the way to accomplish their tasks. We argue that gesture hinting is a means to solve this problem.
| CAUSES | gestures are not really "natural" | is not always obvious and easy to understand which gestures are available |
| PRIMARY CONSEQUENCES | users may miss possible way of interacting and new functionalities | users can be frustrated if they don't find a way to accomplish their tasks |
| SECONDARY CONSEQUENCES | unpleasant user experience | lead the user to renounce or leave the app |
Current Solutions
Gesture hinting in current applications is made trough different methods. None of these examples could be applied in a general framework that can help the user with gesture hinting through all the systems and apps. However some examples show that these methods are really good in some situations.
In the following figure is presented a table that summarizes the good and bad aspects fo current solutions for gesture hinting in mobile devices. It follows a list of examples of current gesture hinting solutions in mobile apps.
This project doesn't have the aim to find a gesture hinting solution that will replace all these methods. These methods have reason to exist, and the solution proposed in the thesis should work along with them. The advantage with the proposed solution is that in any moment, wherever the user is, he can ask for gesture hinting in a quick and dynamic way.
Standard Components
The standard components are elements that have a specific meaning, and appear the same also cross applications. The consistency through the all system and in non-native applications makes them easy to remember for the users. Once the user learns them, he knows which kind of interactions is able to do with them each time that they appear on the screen. Examples of standard components are:
- the cropped images that show to the user that he can swipe to see more content (used in all three OSes);
- the small dots (iOS) or thin highlighted line (Android) used to indicate pagination, hinting to the user to swipe.
Android, Apple and Windows are the main three companies leader in mobile devices; each of them has his own standard components, many of them are however similar, with subtle differences.
Standard Components in Android (i.e. small thin blue line in the homescreen indicating the pagination, cropped figures and text). 
Standard Components in iOS (i.e. small thin blue line in the homescreen indicating the pagination, cropped figures). 
Standard Components in Windows Phone (i.e. cropped figures and text). Written Instructions
Written instructions are one of the most common gesture hinting techniques used in the applications. There are several different ways to propose written instructions to the users. We can describe this different ways along two axes: the prominence and the persistency. The prominence is the characteristic for the written instructions to be in foreground or in the background. The persistency is related with the time; the written instructions can be persistent or transient (only the first time you use the app, for the first seconds of use, pop-up when new items are revealed).
From left to right some examples of: the written instruction of Sudoku2 at the beginning of the app, the persistent written instruction in the background of the Logic Box app, the transient written instructions in the background from TopTruckLite app. 
These screenshots caught from Cut the Rope app shows how to use pop-up written instructions in the background to give instructions to the users for the new items on the screen. Animations
Animations are a very good example of gesture hinting. With subtle animations we can attract the focus of the user and hint him the available gesture. The animations allow also to showing to the user how to perform the gesture. Therefore this technique is more intuitive for the user, because he has just to mimic the gesture shown by the animation.
On the left the slide to unlock animations from Apple devices: on the top iOS 6 and on the bottom the redesign for iOS 7. The second image is from the Mini Golf app, showing an animation of a finger tapping the redo button. The third image shows the swipe gesture over the fruit, this is the action required to the user for play with Ninja Fruit app. On the right the animation for cut the rope in the homonym app. Videos & Tutorials
Some apps use videos or tutorials to describe the tasks that the user can accomplish with the app and show to the users which kind of actions they are supposed to do. The videos and tutorials are usually presented at the beginning, the first time that the users open the app. They are a short introduction to the app. The videos are not interactive, while the tutorials can be also interactive (teaching step-by-step how to interact with the app). The tutorials lead the user through the basic steps that allow she to accomplish the main task of the app. Usually they should be short and don’t require too much time and effort for the user. A big difference is the interactivity of the tutorial.
On the left the introduction screen of miniDraw app shows the possibility for the user to see a video tutorial, the video is presented in the right image. 
On the left the introduction screen of miniDraw app shows the possibility for the user to see a video tutorial, the video is presented in the right image. The next figure shows the interactive tutorial of the Mini Golf app. The tutorial starts only for the first game of the player, and helps the player step-by-step. The tutorial is interactive, since the written instructions that appear in the comic show only one action per time. For example, the first comic says “Place your ball on the starting pad” and an animation with a hand that taps the pad appears, showing to the user where he should tap. Only once the user tapped on the pad the tutorial show the new information. The same happens for the other actions. In this way the task is simple to understand for the user, the gestures are easier to remember and retain (since the user have to accomplish them correctly to skip to the next action), the user can perceive a good feeling since he is successfully completing the tasks.
On the left the introduction screen of miniDraw app shows the possibility for the user to see a video tutorial, the video is presented in the right image. Hoping for the Best
Another common approach is to just show or indicate the most important gestures in the interface, hoping that users will stumble across other functions during use.
Alchemy Concept
The Alchemy concept Lundgren and Hjulstrom suggests a possible approach to the gesture hinting that could be used as a framework through different apps and OSes. Alchemy is presented not as a perfect solution, but rather as a proof of concept to explore and discuss issues related to gesture hinting. Alchemy had at that point in time not been fully implemented and thus never been tested with users. Since a first part of the project is focused on the implementation and user testing of Alchemy concept, the next paragraph presents the concept and exposes some early pros and cons related to it, as stated by the authors.
“In short, our solution is to overlay semi-transparent gesture hints on top of the interface – any interface – which means that the design (graphics and code) can be used together with any application”
- Lundgren and Hjulstrom (2011).
The Alchemy solution proposes a dynamic gesture hinting system that use a set of semi-transparent icons that appear under the finger of the user, on top of the interface, that show which gestures are available for the target object under the user’s finger. For a detail description of Alchemy read the paper Alchemy: Dynamic Gesture Hinting for Mobile Devices.
To summarize and give a first overview of the pros and cons identified in the first analysis of the Alchemy concept I prepared the following table, where I listed the main design decisions described in the article and for each I explain some pros and cons.
| Characteristics | Pros | Cons |
|---|---|---|
| Touch and Hold | - familiar for users | - occlusions - already used in apps and OSes |
| Icons Surround the User's Finger | - close to the focus | - occlusions - combined icons - size and color - partially visible close to the edges |
| Idiomatic Icons | - flexible - extensible |
- less intuitive - require time to be learned |
| Combined Icons | - small space - fast once learned |
- increase the cognitive workload - require time to be lerned - specifically designed for combined icons - not extensible for more gestures |
The main issues with Alchemy concern the occlusion problem and the use of combined icons. The touch and hold gesture, used to activate the gesture hinting mode, is familiar to the users, but has the two important disadvantages. The user holds his finger on the target and therefore occludes part of the screen. The gesture is already used in apps and OSes. The icons surround the user's finger, therefore the information is displayed close to the user's eyes focus. However this means that the finger occlude part of the icon. The choice of size and color could be problematic since the background could be not uniform. If the finger is close to one edge of the screen the icon can be only partially visible.
With Alchemy it is necessary using combined icons in order to show many available gestures at the same time. This will save space, but will require more effort for the users to understand the icons. The use of combined icons force the design of icons that could be combined together, making the concept less extensible for more gestures. Using idiomatic icons we have more freedom to design icons, and the icon set can be easily extended. However the downside is that the icons are less intuitive and need to be learned. For these reasons I started to explore new alternative ways to solve the gesture hinting problem.