Usability
Study of Handwriting Recognition System for Text Entry in PDA’s With Respect to
Handedness in Adult Users
Erman Uzun
Department of Computer Education and Instructional
Technology
Middle East Technical University, Ankara, Turkey
Kurþat Cagiltay
Department of Computer Education and Instructional
Technology
Middle East Technical University, Ankara, Turkey
Abstract:
This paper explores whether handedness has any effect on
usability score and writing speed of users on Graffiti. This study was carried
out on twenty Middle East Technical University students with the age range of
20-28 who have no prior experience on writing with Graffiti. Ten of them named
themselves as left-handed and ten of them named themselves as a right-handed.
Data collected from these users with two activities. No significant correlation
was found between handwriting speed of users on paper and handwriting speed of
users on Graffiti.
Keywords: Handedness, Usability, Graffiti, Handwriting
recognition
1.
Introduction
PDAs
(Personal Digital Assistant) are handheld devices that allow you to store
information with computer capabilities. These devices can be viewed as small
desktop computers that can work on users palm with its increasing power.
However, its small size, small display, limited input capabilities and many
other things restrict the improvements on these devices. One may argue that, using touch-sensitive display
makes these devices more ergonomic, because users do not need many keys and
buttons over these devices. Users can use the system with limited keys, and
great functionality of the touch sensitive display. Unfortunately, there are
still some problems about the text input functions of these devices.
The major
problem for handheld devices is text entry because of their small screen size.
This is an important issue because it is estimated that in 2005 approximately 1
trillion text messages were sent (Gsmworld, 2007). This clearly reveals the importance
of the text entry into the mobile devices. There are two approaches related to
text entry methods on handheld devices. One of them is based on adapting the
user habit on PCs to the handheld device by using virtual keyboard. This
keyboard layout design is same as the QWERTY keyboard design, and users quickly
adapt to use virtual keyboard, because using metaphors that represents users’
PC experiences can reduce users’ effort to learn the mobile device (Jeong &
Lee, 2006).
The other and the dominant approach
highlight the requirement of new design approaches for the special needs of
mobile devices (Jeong & Lee, 2006). With respect to text entry methods on
mobile devices new design approach is handwriting recognition systems. In these
systems two basic approaches come into prominence. First one is handheld device
tries to recognize users handwriting by perceiving the whole character as a
picture and then convert those to a character (Vuori, 1999 ;
Shanthi & Duraiswamy, 2007). This is called as offline recognition systems (Vuori,
1999 ; Shanthi & Duraiswamy, 2007). However, those
systems are not very compatible with the immediate interaction because of its
powerful computing needs.
In the second one, users movement while
they are writing the character, interpreted during writing, and this system is
widely used in mobile computing called online character recognition (Vuori,
1999). One of the most successful handheld computer producers is PalmOS about
handwriting recognition systems. In this system users try to adapt his/her
handwriting to predetermined standards. It is like learning to write with a new
alphabet. The underlying reason of this achievement is the easy-to-use
pen-based text entry method called Graffiti. You can write characters with one
stroke over a special area of the Palm devices touch sensitive displays. The
findings of MacKenzie and Zhang (1997) reveal that after 1 minute of
instruction users can write 86%, after 5 minutes of instruction users can write
97% accurately.
According to the research results of
Költringer and Grechenig (2004), although virtual keyboard’s error rate is
lower and the writing speed is faster than graffiti 2, users state that they
prefer Graffiti2 to virtual keyboard. Because they believe that Graffiti2 is
more intuitive, closer to handwriting, and the subjects expected to gain speed
and accuracy by training. Actually, the most important difference of this new
system is that user can write some characters (I, K, T and X) with two stroke
develop.
As it was stated by MacKenzie and Soukoreff
(2002) users' writing speed on standard keyboard is in the range of 20 - 40
words per minute (wpm). On the other hand, handwriting speeds are commonly in
the 15 - 25 wpm range. As it was cited by Zingale and Kudrick (2005), Bohan stated
that users can write 16-18 wpm on handwriting recognition systems on PDAs.
According to the research results of Költringer and Grechenig(2004),
although virtual keyboard’s error rate is lower and the writing speed is faster
than graffiti 2, users state that they will prefer Graffiti2 to virtual
keyboard. Because they believe that Graffiti2 is more intuitive, closer to
handwriting, and the subjects expected to gain speed and accuracy by training.
Moreover, MacKenzie and Khang (2001) emphasized one point of view that touch
typing speed cannot be transferred to touch tapping as expected, because a
fast-typist cannot transfer his/her skills to the touch tapping. The underlying
reason of that situation is that touch typing is a “highly learned motor act”,
on the other hand touch tapping is mostly “visual oriented” (MacKenzie &
Khang, 2001). For that reason, QWERTY layout design does not preferred by
expert users because of its deficiencies. Therefore, MacKenzie and Khang (2001)
define touch tapping as “hunt-and-peck”.
The more handwriting recognition systems
recognize word properly, and increase writing speed, the more they get usable. In
usability testing one important point should be considered during the studies
is handedness. Devices seem to have symmetric shape, but that does not mean
that it is usable for all left-handed or right-handed users equally. As it was
stated by Inkpen et al.(2006), in PDAs stylus is
located in the right side, buttons and thumbwheels are also located for the
easy use of right-handed users. Although, right-handed users constitute 90% of
the users, a good design should consider all kinds of people and their needs (Schneiderman,
1997). Therefore, this situation necessitate that the left-handed users adapt
themselves to use right-handed designed devices. As it was cited by Inkpen et
al. (2006), up to now many researches reveal that handedness plays an important
role on general aiming (Flower, 1976), pen-based pointing tasks (Kabbash,
MacKenzie, & Buxton, 1993), and the speed in selecting items (Hancock &
Booth, 2004). Inkpen et al.(2006) proved that
left-aligned scrollbars reveals significant advantage for left-handed users.
Actually a simple design change for left-handed users has enhanced their task
accomplishment skills 20-25 %.
The goal of this experimental study is to
investigate the effect of handedness on handwriting recognition systems and to
determine the whether there is a difference among those with respect to
usability scores and writing speed. The percentage of left-handed person is
approximately 10-13 (Raymond et. al, 1996). However, most of the time designers
concentrated on the majority of the user and neglect minorities like
left-handed users. Thus, it can be determined that whether the usability of handwriting
recognition systems will change among left-handed or right-handed.
2.1.
Research
Questions
Twenty students from Middle East Technical
University (METU), Turkey, took part in the study voluntarily. The age of
participants range from 20 to 28, and nine of them are female. Ten of them were
right-handed, and ten of them were left-handed. None of them had any prior
experience on handwriting recognition systems of handheld computers.
In this
study, Palm m130 and two camera recording systems were used. One camera system
recorded users’ handwriting the other recorded the screen of the
Palm while they were writing. All users used the same Palm in a real office
environment (with 160x160 pixel touch sensitive screen and Palm OS 4.1).
First of
all, paper based section handwriting activity was initiated. In this part users
supposed to write 5 short and 1 considerably long text. They wrote under their
text as appropriately as possible with their handwriting. They had opportunity
to use eraser, because in the palm they also have opportunity to fix their
mistakes. After finishing paper based handwriting test, participants informed
about how to write on a Palm device for ten minutes. A small program that works
on PalmOS was used in this short instruction session. Apart from that, in this
instruction users were informed about how to hold device and how to write. This
short instruction ended with a short test activity to adapt users to real test
environment in the Palm and video recording apparatus. In the study, Memopad
application of PalmOS was used as a tool for accomplishing tasks that were
given in the study. Five short sentences and one considerably long text were
given to each participant, to write them into their device as fast and as
accurate as possible. Some of these tasks were taken from the study of Fletwood
et al. (2002), or inspired from the tasks given by Költringer and Grechenig
(2004). Other tasks were determined with the light of the capabilities of
handheld devices, and these all words can be used in daily tasks in a handheld
device. After the sessions the researcher conducted semi-structured interviews
with the participants to learn their satisfaction. After the completion of all
these procedures, in order to assess users’ usability level system usability
scale (SUS) were filled.
In this paper usability of the system was
evaluated under three heading. As it was determined by ISO, usability
evaluations should be based on:
·
Effectiveness
(success rate of the accomplishing their task goals)
·
Efficiency
(performance of the user with respect to time, effort, cost)
·
Satisfaction
(reactions of the users to the application)
4.1.Efficiency
In the
paper based handwriting activity participants supposed to write 428 characters
(with no space). These texts were chosen as the users possibly used in this
handheld devices. These text also include some punctuation characters and
special characters such as “:”, ”,” , ”/”, “@”. The
purpose of this activity is to assess the writing speed of the users, there is no sufficient information about handwriting
speed of the users for this kind of special activity. Without considering
handedness, their task accomplishment time is 231.5 sec. On the other hand,
left-handed users’ task accomplishment time is 249.3 sec, and right handed
users task accomplishment time in paper based handwriting activity is 213.7
sec. When we look at the character per minute (cpm) data of all users, it is
110.9 cpm. Left-handed users achieve 103.1 cpm, and right-handed achieve 120.2
cpm.
Although
there is a difference in the hand writing speed of the users in order to
determine whether there is a significant difference, Independent sample t-test
were done. After checking the assumptions of t-test, analysis of the data were
done and yields that there is a significant difference between left-handed and
right-handed users handwriting speed (t(18)=2.195.
p=.041).
Table 1. Comparison of task accomplishment time Paper based and Palm based handwriting
|
|
|
Paper-based (sec) |
Palm-based (sec) |
|
|
|
||
|
All Users |
231,5 |
846,8 |
|
|
Left-Handed Users |
249,30 |
860,8 |
|
|
Right-Handed Users |
213,70 |
832,8 |
|
Other test
environment is PalmOS and the tool is Graffiti. In this activity participants
supposed to write 277 character 6 texts. 5 short text and one considerably long
texts were given to users. In this task “space” and
“enter” viewed as a character because in order to make these actions user
should draw specific characters to the screen. Without considering handedness,
their task accomplishment time is 846.8 sec. On the other hand, left-handed
users’ task accomplishment time is 860.8 sec, and right handed users task
accomplishment time in paper based handwriting activity is 832.8 sec. When we
look at the character per minute (cpm) data of all users, it is 19.63 cpm.
Left-handed users achieve 19.31 cpm, and right-handed achieve 19.96 cpm. In
order to control whether there is difference in the handwriting speed of users
with respect to handedness, independent sample t-test were done. All the
assumptions of t-test were checked. This analysis yields that there is no
significant difference between left-handed and right-handed users handwriting
speed in Graffiti (t(18)=.275.
p=.786)
Table 2. Comparison of Paper-based and Palm-based handwriting
|
|
|
Paper based (cpm) |
Palm based (cpm) |
|
|
|
||
|
All Users |
110,93 |
19,63 |
|
|
Left-Handed Users |
103,01 |
19,31 |
|
|
Right-Handed Users |
120,17 |
19,96 |
|
4.2.Effectiveness
In all
these two activities users have opportunity to correct their errors. However,
in Palm some users cannot write all characters in the given tasks. Therefore, this gives the error rate of the activity.
In palm-based activity total error is 41. This means that approximately 99.3%
of the tasks were done by the users after ten minutes of instruction in novice
adult users. This reveals consistent results with the MacKenzie and Zhang
(1997). Because they highlight that after 1, and 5 minutes of instruction users
can write 86%, 97% accurately. When we consider handedness in error rate,
left-handed users error rate is 0.65 %, and
right-handed users error rate is 0.83 %. When we analyze errors that users made
and video records of the palm based handwriting activity, most of the users
have problems in writing punctuations.
4.3.Satisfaction
Users’
satisfactions were collected by semi-structured interviews. With respect to
answers that users gave, most of the users do not satisfied with this system.
Only 6 users express their satisfaction with the system (3 of them
right-handed, 3 of them left-handed). Although these users satisfied with the
system, their satisfaction mainly depends on the novelty effect of the system. Moreover,
one user express that
“I like and
satisfied with the system, it is very funny...”.
On the
other hand, users that were not satisfied with the system express the problems
of the system. Different users have problems in writing different characters
but most of the user highlights the complexity of writing punctuations.
“Each time I
wrote in the same way in ‘:’ but the system cannot recognize it… so it has some
inconsistencies…”.
Another
inconsistency that effect the satisfaction of the users is the writing style of
different character is different some of them can be written in capital letter
some of them not.
“I have problems to remember
some characters because some of them written in capital letter some of them
not. This really confused me…”.
Moreover,
some users have problems in writing one stroke. They want to write some
characters like “B”, “D” in two stroke. However, users
can write numerical characters much more easily. Nonetheless, most of the users
were not satisfied with the system because of the reasons stated above, and
users percentage of that they satisfied with the system were only 30 %.
4.4.Total usability ratings
In order to
learn overall usability score of the users System Usability Scale. It is a ten
item Likert type usability assessment scale. The calculation of this scale were
done as Brooke(2007) explained. Therefore the total
score of usability rating will range from 0-100. As a result of this study,
total usability mean is 56.5. However, interestingly when handedness is
considered, there is a difference in the usability score of the users.
Left-handed users give 62.9 usability score to Graffiti, and right-handed users
give 50.1. Actually there seems to be a difference between left-handed and
right-handed user’s usability scores. However, in order to determine whether
there is a significant difference or not, independent sample t-test were done.
This analysis yields that there is no significant difference between
left-handed and right-handed users’ usability scores for Graffiti (t(18)=1.238.
p=.231).
Writing skills is shaped from the early
childhood many people have different writing style. According to Weiss and
Parush (2004), handwriting is a complex human activity that entails an
intricate blend of cognitive, kinesthetic, and perceptual-motor components. Today
with the developing technologies there are many different ways to write such
as:
·
Writing
with a pen based system with hand writing
·
Using
hard keyboard
·
Using
soft keyboard
·
Sound
recognition.
Although there is an increasing motivation
to write with a hard keyboard, pen based writing systems are indispensable.
Especially, writing on handheld devices is still an important problem for
designers, and the most appropriate way to solve this problem seems to be
pen-based systems. In this research, Graffiti handwriting recognition system
that is used on Palm devices was analyzed. However, handedness causes no
significant difference on users’ hand writing efficiencies. Moreover, when
correlation of palm-based activity and paper-based activity analyzed, there
seems to be any correlation. Therefore, handwriting speed does not have any
correlation with writing with Graffiti. For instance, the fastest writer on
Graffiti could achieve 35 cpm, it is far more faster
than other users. Although his/her handwriting on paper was in the middle
position (97 cpm), his/her writing speed on Graffiti was the best. Apart from
that he/she has no error in the Graffiti. In order to understand underlying
reason of his/her great achievement on the system a special interview session
were done. After this interview, this user express that writing on this device
is very simple, some of the characters are similar to the technical drawing
letters and he/ she correlate his/her achievement on his skills on technical
drawing skills. Therefore, the user who has experience on technical drawing or
drawing like activities may achieve better results in handwriting recognition
systems. When camera records of the writing activity on graffiti were analyzed,
this situation can be seen more clearly, because, the fastest writer can draw
characters of the Graffiti quickly and sharply. However, low achievers writing
reveal that their hands shake and for that reason the system could not
recognize what they wrote, and they have to try to write same character several
times.
Although different user faced with
different problems on writing different characters, there is
some common problems on writing punctuation letters. Especially users have
problem in making internet related tasks writing ‘://’. Punctuation letters is
written after the point and it needs a combination of two tapping.
Apart from that, users have problems on
remembering some letters like G, H, N. The users
explain this situation with the complexity of the alphabet. Some characters
were written in capital letters, but some were not. This really confused users
mind, when video records of the Palm-based activity analyzed most of the users
cannot write these letters in the first trial. However, with the Graffiti2
there is not any improvement in this problem. It is still a usability problem,
and this situation should be investigated more deeply.
Graffiti handwriting system is not usable
for left-handed novice users. The usability score of Graffiti for left-handed
users is 63. With ten minutes of training they can perform all tasks with
99.35% accuracy, in average 860 second time, and only 30% of the left-handed
users satisfied with the system. Similarly, Graffiti handwriting system is not
usable for right-handed novice users. The usability score of Graffiti for
right-handed users is 50. With ten minutes of training they can perform all
tasks with 99.5% accuracy, in average 833 second time, and only 30% of the
left-handed users satisfied with the system. There is a difference in
left-handed and right-handed users’ handwriting time with Graffiti and
usability scores. However, this is not statistically significant difference.
Therefore, with this information obtaining from this study handedness has no effect
on handwriting studies in pen-based systems in novice adult users.
For further studies, different pen-based
handwriting systems should be investigated with usability perspective with
including Turkish letters, and in these researches both left and right handed
users can be participated. Apart from that, there is a great gap in writing
style of Turkish students. These styles may affect their writing speed and
quality, and these differences may effect pen-based
computing skills of the users.
References
Brooke
J. (nd.). SUS - A quick and dirty
usability scale. Retrieved 01, 05, 2006, from
www.usabilitynet.org/trump/documents/Suschapt.doc
Cheng, S., & Y. Macredie, R. D. (2005). The assessment of usability of electronic
shopping: A heuristic evaluation. International
Journal of Information Management, Vol. 25, pp. 516-532.
Fleetwood, M. D., Byrne, M. D., Centgraf, P., Dudziak,
K. Q., Lin, B. and Mogilev, D. (2002). An evaluation of text
entry in Palm OS – Graffiti and the virtual keyboard. Proc. of the Human
Factors and Ergonomics Society 46th Annual Meeting, pp. 617-621.
Inkpen,
K., Dearman, D., Argue, R., Comeau, M., Fu, C., Kolli, S., Moses, J., Pilon,
N., & Wallace, J. (2006). Left-handed scrolling for
pen-based devices. International
Journal of Human-Computer Interaction. Vol. 21, No. 1, pp. 91-108.
Jeong, S., & Lee, K. (2006). The effects of experience with a PC on the usability of a mobile
product. Retrieved 11, 27, 2006, from
www.idemployee.id.tue.nl/g.w.m.rauterberg/conferences/CD_doNotOpen/ADC/final_paper/361.pdf
Költringer, T., &Grechenig, T. (2004). Comparing the Immediate
Usability of Graffiti 2 and Virtual Keyboard. In
Extended Abstracts of the ACM Conference on Human Factors in Computing Systems,
pp. 1175-1178.
MacKenzie, I. S., & Soukoreff, R. W. (2002). Text entry for mobile computing: Models and
methods, theory and practice. Human-Computer Interaction,
Vol. 17, pp.147-198.
MacKenzie,
I. S., & Zhang, S. (1997). The immediate usability of
Graffiti. Proceedings of Graphics Interface '97, pp. 129-137.
Toronto: Canadian Information Processing Society
MacKenzie,
I. S., & Zhang, S. Z. (1999 b) The design and
evaluation of a high performance soft keyboard. Proceedings
of the ACM Conference on Human Factors in Computing Systems - CHI '99,
pp. 25-31. New York: ACM.
MacKenzie, I. S., & Zhang, S. Z. (2001). An empirical
investigation of the novice experience with soft keyboards. Behaviour & Information Technology. Vol. 20, pp. 411-418.
MacKenzie, I. S., Zhang, S. X., & Soukoreff, R. W.
(1999 a). Text entry using soft keyboards. Behaviour &
Information Technology, 18, 235-244.
Raymond,
M., Pontier, D., Dufour, D., & Moller, A. P. (1996). Frequency-Dependent
Maintenance of Left Handedness in Humans. Proceedings: Biological
Sciences, Vol. 263, No. 1377, pp. 1627-1633.
Schneiderman, B. (1997). Human
Factors of Interactive Software. In Designing the User Interface:
Strategies for Effective Human-Computer Interaction ,
pp. 1-37, Addison-Wesley.
Shanthi N. and Duraiswamy K. (2007). "Performance Comparison of Different Image Sizes for Recognizing
Unconstrained Handwritten Tamil Characters using SVM," Journal
of Computer Science, 3 (9): 760-764.
Vuori
V. (1999). "Adaptation in On_line Recognition of Handwriting," M.S.
thesis, Helsinki University of Technology, Helsinki.
Zingale
C., Ahlstrom V., & Kudrick B. (2005). Human Factors Guidance for the Use of Handheld, Portable, and
Wearable Computing Devices. Retrieved 11, 27, 2006, from
http://www.tc.faa.gov/its/worldpac/techrpt/ct05-15.pdf