Automatic Glossing for Second Language Reading Michael Walmsley The University of Waikato Hillcrest Road, Hamilton
[email protected] ABSTRACT Research in second language (L2) acquisition suggests extensive reading (ER) as an effective learning strategy. ER involves learners reading lots of easy L2 text. Classroom ER programs use graded readers—simple books written specifically for L2 learners. However, the availability of graded readers is limited by the expense needed to create them. An alternate approach is to read L2 news articles with Computer software providing glosses, i.e., first language translations of words learners click on while reading. Since manual glossing of text is tedious, this project is exploring approaches for automatic glossing.
Categories and Subject Descriptors K.3 [Computers and Education]: Computer Education – computer assisted instruction (CAI).
General Terms Algorithms, Human Factors.
Keywords Natural language processing (NLP), machine translation, computer-assisted language learning (CALL), hypertext glossing.
1. INTRODUCTION Second language (L2) acquisition requires a significant investment of time and effort. Extensive reading (ER) is an effective technique that involves fast-paced reading of large quantities of easy, interesting text. ER is a good way to learn new vocabulary and enhance existing vocabulary knowledge. Studies also demonstrate improved reading and writing ability as well as improved attitude to L2 reading as a result of ER. A key factor for successful ER is that learners choose what to read, and stop reading if a text is uninteresting or difficult. Dictionary use slows reading and should be avoided. Instead, learners should guess the meaning of key unknown words from context, and skip those that do not inhibit a general understanding of the text. The mental effort required to guess the meaning of unknown words promotes vocabulary learning, as long as the learner guesses correctly. Research suggests that learners can read without assistance, and achieve good comprehension when they are familiar with at least Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. NZCSRSC’11, April 18-21, 2011, Palmerston North, New Zealand.
98% of the words in a text [1]. Carver [2] indicates that with 98% coverage reading is difficult but comprehensible, 99% coverage makes reading comfortable, but close to 100% is required for easy reading. The exact value of the vocabulary threshold depends on the ability of individual learners to skip unknown words that don't affect the understanding of a text, and to infer the meaning of important unknown words from context. The threshold also varies between texts, depending on the richness of the context surrounding unknown words. Studies by Nation et al. estimate that learners require knowledge of 5,000 word families 1 to achieve 98% coverage of novels written for native English speaking teenagers [3] and approximately 9,000 word families for adequate comprehension of adult novels [4]. It follows that authentic text—text written for native speakers—is inappropriate for unassisted ER by all but the most advanced learners. For this reason, many educators advocate the use of learner literature, that is, stories written specifically for L2 learners, or adapted from authentic text [5]. For learners of English, there are over 40 graded reader series, consisting of over 1650 books with a variety of difficulty levels and genres [6]. However, the time and expense in producing graded readers results in high purchase costs and limited availability in languages other than English and common L2‘s like Spanish and French. At a cost of £2.50 for a short English reader in 2001 [7] purchasing several thousand readers to cater for a school wide ER program requires a significant monetary investment. More affordable options are required, especially for schools in developing nations. Day and Bamford [8] recommend several alternatives when learner literature is not available. These include children's and young adult books, stories written by learners, newspapers, magazines and comic books. Some educators advocate the use of authentic texts in preference to simplified texts. Berardo [9] claims that the language in learner literature is ―artificial and unvaried‖, ―unlike anything that the learner will encounter in the real world‖ and often ―do not reflect how the language is really used‖. Berardo does concede that simplified texts are ―useful for preparing learners for reading 'real' texts.‖
2. ASSISTED READING Due to the large proportion of unknown vocabulary, beginner and intermediate learners require assistance when using authentic text for ER. Two popular forms of assistance are dictionaries and glossing. There are pros and cons of each approach.
1
A group of words that share the same root word, e.g., run, ran, runner, runs, running.
2.1 Dictionaries Dictionary use can improve comprehension and incidental vocabulary learning when reading difficult texts [10]. When looking up ambiguous words—those with multiple meanings— learners have to determine the correct meaning from context [11]. The disambiguation process promotes vocabulary learning in the same way that guessing from context does, however, with a limited set of possibilities learners are less likely to guess incorrectly when using a dictionary. Furthermore, Holley and King [12] describe the 'rehearsal' effect, the idea that learners vocally or sub-vocally repeat words they look up in a dictionary. The rehearsal of the words also facilitates vocabulary learning. The downside of paper dictionaries is that slow lookup breaks the flow of reading [10], [13]. If learners are given access to dictionaries they will often use them excessively [14]. Day and Bamford recommend an outright ban on dictionary use for ER novices. This approach helps students develop the skills of skipping unimportant unknown words and guessing the meaning of important unknown words from context [8]. As learners begin to understand the purpose of ER they can be given the freedom to use dictionaries to determine the meaning of unfamiliar words that appears several times in a text.
2.2 Glosses Glosses are short L1 or L2 definitions that appear next to difficult words, in the margin, or at the end of a text.1 As with dictionary use, gloss access enables learners to comprehend more difficult texts than with unaided reading. Gloss access has the additional advantage of being faster than dictionary lookup [15], resulting in less disruption to the flow of reading. Dictionary use is particularly problematic for sentences containing multiple unfamiliar ambiguous words. Single word/phrase glosses alleviate the need for learners to disambiguate the contextual meaning of words with multiple senses. Studies indicate that addition of glosses to ER texts, can increase comprehension [16-18] and incidental vocabulary learning [19], [20]. Furthermore, surveys show that readers prefer glossed over un-glossed texts [19], [21]. Determining the optimal conditions for glossing is difficult, as it depends on a number of interacting factors such as:
Learner ability and text difficulty [17], [22], [23];
Proportion of glossed words [24];
Gloss language—L1 or L2 [25-27].
An example complication is the mixed-results of studies comparing the effectiveness of L1 versus L2 glosses [28]. The effect of gloss language interacts with learner ability; L1 glosses are more effective for beginners, but L2 glosses are more effective for advanced learners [29]. Glossers of paper-based texts face the challenge of deciding what proportion of words to gloss. If too many words are glossed the reading process will be slowed by distracting glosses of known words. Conversely, if too few words are glossed the learner must resort to the less time-efficient methods of dictionary look-up and guessing from context, for unknown un-glossed words. An optimally glossed text only contains glosses of unknown words that can not be easily guessed from context. Since, no two learners share identical vocabulary knowledge [30], optimal glossing is dependant on the ability of each individual reader.
The effects of other aspects of gloss presentation have also been investigated. Cheng and Good compared three types of glosses, L1 glosses with L2 example sentences, L1 marginal and L1 in-text glosses [19]. Jacobs [21] indicated that learners prefer marginal glosses or in-text glosses over glosses at the bottom of the page or the end of the entire text. Another issue to consider is that the speed advantage of glosses is a disadvantage if a learner is trying to learn the meaning of glossed words. Simply viewing a gloss requires less mental effort than contextual guessing or dictionary lookup. With less mental effort the learner is less likely to remember the meaning of the word [31]. As a trade-off between contextual guessing and single word glosses, Hulstijn proposed the use of multiple-choice glosses, where learners are presented with the correct gloss as well as one or more distracters [32]. Studies indicate that multiple-choice glosses are more beneficial to vocabulary learning than single word glosses [33-35]. However, multiple-choice glosses cause problems with comprehension if learners guess incorrectly. A study by Gettys, Imhof and Kautz suggests that providing L1 dictionary form glosses promotes a deeper level of processing than regular single word glosses, hence increasing vocabulary learning [36]. Take for example the Spanish sentence: Fue a las tiendas. (He went to the shop) Regular glosses for fue and tiendas would be went and shops. Dictionary form glosses would be (verb) to go and (noun) shop.
3. ELECTRONIC DICTIONARIES Electronic dictionaries provide advantages over their paper-based counterparts—most notably increased lookup speed. This section describes pros and cons of several forms of electronic dictionaries.
3.1 Hand-held electronic dictionaries Hand-held electronic dictionaries are available for many common L1-L2 pairs. They package several large paper-based dictionaries, thesauri and encyclopaedias into purpose-built devices that fit in the palm of ones hand. These devices allow for faster lookup than paper-based dictionaries, but are expensive ($100– $1000NZ) and are available in fewer languages.
3.2 Software dictionaries Software dictionaries are available for portable devices such as PDAs and mobile phones. Software dictionaries are available at no cost for many language pairs. However, they are generally less user friendly than purpose-built electronic dictionaries. Both electronic and paper-based dictionaries are particularly difficult to use for learners who are learning to read the L2 character set.
3.3 Online Dictionaries Online translation dictionaries are provided by a number of web-sites, such as Google Dictionary 2 , which provides English
2
http://www.wiktionary.org
translations for 26 different languages 3 . Online dictionaries provide fast look-up speed when reading electronic texts, because learners can copy and paste unknown words into a search box. This technique is particularly efficient when the learner is learning the character set and spelling conventions of the L2. Denshi Jisho 4 is a Japanese online dictionary that allows learners to search for Japanese words by typing with the English alphabet; words are automatically converted to Japanese characters before the search begins.
translations. The dictionaries are based on Wiktionary7 and other free dictionaries, augmented with contributions by Lingro users. Other commercial web sites like Ultralingua.net and Wordchamp provide similar functionality [39]. Gymn@zilla is a similar system developed at several research institutions in the UK [40]. Hypertext builder 8 is a research system for learners of English, French and Spanish. It provides audio pronunciation, concordance examples and dictionary definitions [41]. PopJisyo 9 and Rikai 10 provide instant dictionary lookup for Japanese and Chinese web pages (PopJisyo also supports Korean). As well as displaying the meaning of a particular word, the system also shows the meaning of the constituent characters. iFinger 11 is a popular commercial system that provides instant access to both dictionary entries and additional reference information when reading web pages. Other systems like eLecture12—online lecture delivery software—provide translation dictionary look-up as a supplementary feature. Glosser-RuG [42] is a desktop application designed to support Dutch speakers reading French texts, but can be extended to support other languages. In addition to translations, the system provides morphological analysis, part-of-speech disambiguation and example sentences from. The WELLS project implemented a web-based version of Glosser-RuG called Glosser-WeB [42].
Figure 1 The Rikaichan Firefox extension
4. HYPERTEXT GLOSSING
Plug-ins and extensions provide rapid access to translations and definitions while reading text in a web browser. For example, Rikaichan 5 (see Figure 1) displays English translations when users hover over unknown words in a Japanese text.
CAER systems provide access to glosses, by allowing learners to view translations, definitions, and synonyms of words they click on. A quantitative meta-analysis of 32 studies concluded that CALL glossing facilitates reading comprehension more than paper-based glossing due to greater flexibility [43]. Overglossing—glossing too many words—is alleviated by displaying glosses only when learners request them. Under-glossing— glossing too few words—is alleviated by giving learners access to glosses for every single word in a text. Increased gloss access increases comprehension.
One downside of online dictionaries is the need for internet access. However, the increasingly widespread availability of webenabled portable devices, coupled with increasing coverage of 3G networks and wi-fi, make online dictionaries a serious competitor to specialised hand-held dictionaries.
3.4 Online Reading Web Applications Computer-assisted ER websites provide hypertext dictionaries, displaying dictionary entries when learners click or hover over unknown words while reading. Additionally, some systems display concordance examples. Studies show that computermediated dictionary access increases reading speed [37], comprehension and vocabulary acquisition from ER [38]. The Lingro website 6 provides instant dictionary lookup for 11 languages. Learners enter the URL of a web page they want to read. Lingro displays a modified version of the page, in which learners can click unfamiliar words to view a list of possible
3
Computer systems that do not take advantage of the flexibility at offer will likely produce no learning effect. For example, in one students read a 568 word glossed text in electronic and paper form. In both formats the same 21 words were glossed. The paperbased text used marginal glosses; the computer-based version displayed glosses in the top-right corner of the screen. The results of this study were that both groups performed equally well. This is not surprising, since both groups had access to the same number of glosses in a similar format [44]. Another CALL glossing meta-analysis demonstrated a medium effect for reading comprehension gains and a large effect for
7
http://www.wiktionary.com
8
http://www.lextutor.ca/hypertext/ http://www.popjisyo.com
As at the 10/12/2010
9
4
http://www.jisho.org
10
http://www.rikai.com
5
http://www.polarcloud.com/rikaichan
11
http://www.ifinger.com
http://www.lingro.com
12
http://www.umediaserver.net/electure/
6
incidental vocabulary learning [45]. The vocabulary learning effect was particularly large for intermediate and advanced learners, with only small gains for beginners.
different language versions of the text [49]. This means that appending English glosses to Spanish Bible verses requires an electronic copy of both the English and Spanish Bibles.
Abuseileek [46] investigated the effect of gloss position in a hypertext reading system. The results indicate that marginal glosses are preferable over glosses at the bottom of the page, or the end of the text. Glosses appearing in a pop-up window in the margin were inferior to less conspicuous marginal glosses.
The SGT algorithm utilizes the chapter-verse structure of the bible. To gloss the Spanish word templo, the algorithm performs the following steps:
4.1 Assisted Glossing Manual preparation of glossed texts is a laborious task. Hence, several systems have been developed to assist teachers in creating glossed texts. For example, the Guided Reading hypermedia shell [47] makes it easy for teachers to annotate texts with both audio and visual glosses. The LitGloss website from the University of Buffalo provides similar functionality [48]. jGloss 13 is a system that assists teachers in appending in-text glosses to Japanese texts. Teachers decide which words to gloss, and then select the correct gloss from a list of possible translations. The system also assists teachers in attaching furigana to kanji learners may be unfamiliar with.
1.
Search the Spanish Bible and create a list of verses containing templo and other words in the same word family, such as templos.
2.
Count the frequency of word families occurring in the corresponding English verses.
3.
For each English word family in the verses calculate the ratio fv / ft, where fv is the frequency count in the selected verses, and ft is the frequency in the entire bible. The word family with the ratio value closest to 1 is the primary translation. In the case of templo it will be temple.
4.
Remove the English verses that contain the primary translation and repeat steps 2 and 3 on the remaining verses to reveal any secondary translation.
4.2 Automatic Glossing
Step four is necessary because some words have more than one translation. For example, the Spanish word pan translates to both bread and loaf in the English version of the Bible.
Automatic glossing aims to further alleviate the manual work required to gloss texts. This section describes three approaches.
4.2.3 Reading Assistant Project
4.2.1 Second Language Reader Second Language Reader 14 (SLR) is a web-based application that aids L2 reading. Learners select from pre-created texts or upload their own. The system provides several reading tools. Glosses are provided via the public Google Translate (GT) API. When learners click an unknown word in a text, the single word is sent to the GT API, which returns the statistically most common translation. The same gloss is displayed regardless of the context in which the word appears. For example, the SLR will always gloss the English word set with the Spanish word conjunto which refers to a noun meaning group, squad or mathematical set. Google Dictionary lists 81 Spanish translations for the word set. SLR incorrectly glosses set when it is used to mean something other than group, squad or mathematical set. SLR also displays Google Translations of entire paragraphs when learners click in the left margin next to a paragraph. SLR also provides a button that links to ImTranslators text-to-speech software15, which allows learners to listen to a text. ImTranslator provides speech synthesises for 8 languages.
4.2.2 Statistical Glossing Tool The Statistical Glossing Tool (SGT) was developed by the British & Foreign Bible Society. The SGT uses a language independent approach to automatically gloss the Bible, using two
13
http://jgloss.sourceforge.net
14
http://open.byu.edu/projects/readers/L2/L2.htm
15
http://text-to-speech.imtranslator.net
The Reading Assistant Project16[50] programmatically analysed the frequency of words in a corpus of English-language medicalrelated articles. Frequency counts in the genre specific collection were compared to frequency counts in a general corpus, in order to automatically extract a list of 8,000 technical words. A single medical related Japanese translation was manually specified for each word. This list of words and corresponding translations is called the RAP dictionary. The RAP software is used to automatically gloss arbitrary medical related text using the RAP dictionary. Some of the words in the RAP dictionary have non-medical related usages with different Japanese translations. For example, heart is translated shinzō when referring to the organ involved in a heart attack, but is translated kokoro when referring to a kindhearted person. The RAP software will incorrectly gloss nonmedical-related usage of terms in the RAP dictionary. Due to this high propensity for errors, teachers are required to manually check that glosses are appropriate for the context in which they appear. The other major drawback is the manual work required to generate the translations in the RAP dictionary. However, it may be possible to automate the translation process using an approach similar to that of the SGT project. The approach could use machine translation to create a sentence aligned parallel corpus of technical articles. This corpus would be used in place of two different language versions of the Bible. In future research we may investigate the effectiveness of this approach.
16
http://lsd.pharm.kyoto-u.ac.jp/en/service
5. OUR SYSTEM To our knowledge, no automatic-glossing systems use word-sense disambiguation. Recent studies have described the great benefits that would accrue from a system that accurately glosses texts with the best translation for ambiguous word [51]. This project is exploring approaches to do so.
5.1 Overview We are developing a web-based application called Flax-Reader, which includes functionality for automatic glossing of L2 texts. Server
1. Request for L2 reader from URL
2. Extract text 3. Create glosses
4. Return L2 reader with embedded glosses
Client
Figure 3 Process for creating a glossed L2 text The process for creating glossed texts (see Figure 3) is as follows: 1.
A user requests the creation of a glossed L2 reader from an online news article by sending the URL of the article to the Flax-Reader server.
2.
The server retrieves the web-page for the news article and extracts the article contents from the raw html using a series of regular expressions.
3.
The text is automatically glossed using the GT API17.
4.
A HTML page containing the article text is returned and displayed on the server. Glosses are embedded in span tags and displayed instantly using JavaScript when the learner clicks an unknown word.
Figure 2 is a Spanish news article displayed in Flax-Reader. The reader has clicked the left mouse button on the word diciembre, revealing the gloss in December. When the learner releases the left mouse button the gloss is hidden.
5.2 Statistical Machine Translation GT is a state-of-the-art statistical machine translation engine that supports 59 languages18. This section gives a brief introduction to machine translation (MT) and how GT works. MT is the use of computers to automatically translate one natural language into another. Since the 1950s MT systems used various rules-based approaches, which incorporate a detailed set of rules developed by linguists to translate between two languages. However, increases in computer processor speeds have led to the emergence of statistical machine translation (SMT) as the dominant approach [52]. The statistical approach treats MT as a machine learning problem. Translation models are trained on a large body of parallel corpus, which consists of documents translated by humans into multiple languages. Google‘s translation models are based on billions of sentences of multilingual content freely available on the web. Major sources include United Nations documents, which are translated into six languages, and the European Union documents which are available in 20 languages. Google‘s also trains a language models from trillions of words of mono-lingual corpus also harvested from the web [53]. Increasing the size of the corpus used to generate the models increases the accuracy of the translations. The sheer volume data that Google bases its models on led to first placing in the NIST test of 20 MT engines in 2005 [52].
Pass me the red wine please. Word generation with translation model Pásame
el
tinto
vino
por
favor
Word ordering with language model
Pásame el vino tinto por favor. Figure 4 Model of the statistical machine translation process The SMT process can be modelled as the two step process illustrated in Figure 4. Step one uses the translation model generate a set of translationally equivalent target language words from a sequence of source language words. Step two places those sentences in the order that makes the most grammatical sense, using the language model for the target language. Figure 2 FlaxReader reading GUI
17
See http://www.google.com/research/university/translate
18
As at 31/12/2010
5.3 Google Translate Research API With the GT Public API a client sends a chunk of text and a target language code, and receives the resulting translation. In addition to the translation, the Research API returns word-alignment data. Word-alignments link words and phrases in a source text with their counterparts in the translation. Access to the GT Research API is available for university research only. In order to gain access to the Google Research API one must send a research proposal and wait for approval from Google. The Research API is accessed by sending an HTTP GET request to the following URL: http://translate.google.com/researchapi/translate? sl=en&tl=es&q=Hello+world!&align=1
In this example the tl parameter specifies that the target language is Spanish, and the sl parameter specifies a source language of English. The q parameter indicates the query text is ‗Hello world!’, and align=1 is used to request word alignments. The XML result for the request is as follows (ellipsis replaces unimportant details in the response):
case; their respective glosses are FlaxReader, ayudas and Segundo. In some cases, a single word in the translation may correspond to two or more words in the source text. In the example, the translated word idioma is aligned to both language and learning in the source text. In this case the two words in the source text are combined, and glosses as the two-word phrase language learning. In other cases, multiple words in the translation correspond to a single word in the source text. In the example both un and idioma align with language, and both aprender and idioma align with the source word learning. These words are also combined so that the phrase language learning is glossed with the Spanish phrase aprender un idioma. Figure 5 illustrates the gloss that corresponds to each word or phrase in the example sentence. FlaxReader
ayudas
segunda
aprender idioma
FlaxReader aids second language learning. Figure 5 Glosses displayed by FlaxReader
http://translate.google.com/researchapi… 2006-09-05T23:59:37.924Z Translation Feed Hello world!
Access to the GT Research API in Java code is simplified by using a Java client library provided by Google.
http://translate.google.com/… 2006-09-05T23:59:41.500Z Translation ¡Hola, mundo!
By utilizing this tool, Flax-Reader can easily support reading in most of the 59 languages supported by GT. Unfortunately, the word alignments do not work with Japanese and Chinese texts. The position attribute in the word alignments correspond to the start of words in the original text. However, since Japanese and Chinese contain no spaces between words, the alignment positions are zero. For example, the following result is obtained when translating a Japanese sentence with no commas:
The data of interest in the XML response is contained in the gt:alignment elements. The response contains a sequential list with at least one element for every word in the translation. The position attribute specifies the character position in the source text to which words in the translation corresponds. In the example code ‗¡Hola,‘ corresponds to an index position of 0 in the source text, which matches the word Hello; ‗mundo!‘ corresponds to the word beginning at position 6, which is ‗world!‘. The following excerpt for a translation of a complete sentence demonstrates some of the issues with using word alignments for glosses: FlaxReader aids second language learning. FlaxReader ayudas aprender un segundo idioma.
Glossing a text is straightforward for words that are aligned to one translated word that has one alignment entry. In the example sentence FlaxReader ayudas and segundo are examples of this
5.4 Issues
この例文は動作しません。 This sentence does not work.
Another issue to consider is that the quality of the glosses is dependant on the quality of the translation engine. GT works best with languages like French and Spanish for which it has large parallel corpus from which to build its models.
6. EVALUATING AUTO GLOSSING This section outlines our plan for evaluating the use of the GT Research API in automatically glossing L2 text. This evaluation will investigate both the quality of generated glosses, as well as the impact of poor glosses on learners.
6.1 Gloss Quality This evaluation will compare glosses generated with GT Research API with glosses generated using the approaches described in section 4.2. None of these approaches will provide the best glosses for an arbitrary text all of the time. The evaluation will
determine the proportion of words accurately glossed by each method for:
Low-, mid- and high-frequency vocabulary;
Different parts-of-speech;
Different text genres;
Different text types, e.g., literary versus non-literary texts.
Where possible, analysis will be automated with software, to minimise manual gloss inspection. Automation may include marking ambiguous words for manual inspection. Additional work may investigate the effect of the following two improvements to the automatic glossing algorithms: 1.
2.
Words with multiple possible translations can be glossed by using the L1 translation that covers the most senses of the L2 word. Alternately, the first translation in a dictionary can be used, since it is often the most common. NLP part-of-speech tagging can be applied to filter translations that correspond to an incorrect part-of-speech.
6.2 Impact of Errors The second part of the evaluation will conduct user studies to answer the following questions: How do incorrect glosses comprehension and enjoyment?
impact
learner's
7. CONCLUSION This project has developed an approach to automatic glossing that uses the Google Translate Research API. The approach has been integrated into the FlaxReader web-application. Initial user testing of FlaxReader supports the use of GT Research API glossing approach for language pairs for which GT works well. The evaluation described in section 6 will attempt to quantify the value of this approach in relation to others.
8. REFERENCES [1]
[2]
[3]
[4]
reading
How easily do learners recognise poor glosses? How helpful is providing dictionary lookup as a back-up?
[5] [6]
How often do learners access poor quality glosses when reading articles on the internet? Low-frequency words are generally less ambiguous than high-frequency words. Do beginner learners (those with less knowledge of highly ambiguous high-frequency words) access incorrect glosses more frequently than advanced learners?
[7]
Articles with a high proportion of ambiguous words and long sentences are likely to be difficult to gloss. Can these problematic articles be algorithmically identified?
[10]
In this evaluation, learners will be asked to read a text and mark words with glosses that seem incorrect or confusing. Learners will then be asked affective questions about their experience in reading glossed texts generated using the various approaches. Other evaluation ideas include: Expert evaluation rating the quality of individual glosses on a four point scale and suggesting alternatives to poor glosses.
[8] [9]
[11] [12]
[13]
Expert evaluation of manually produced glosses compared to automatic glosses.
[14]
6.3 User Studies
[15]
The literature recommends texts with at least 95% familiar vocabulary for unassisted ER, but provides no threshold for ER with glosses. A user study to fill this gap would involve subjects reading several short texts and rating perceived difficulty. The proportion of unknown words in each text could be estimated by having learners click unfamiliar words after reading. Words for which they accessed a gloss would also be considered unfamiliar. Perceived difficulty and reading time would be correlated with gloss access and self-reporting of the number of unfamiliar words.
[16]
[17]
[18]
B. Laufer and G. C. Ravenhorst-Kalovski, ―Lexical threshold revisited: Lexical text coverage, learners‘ vocabulary size and reading comprehension,‖ Reading in a Foreign Language, vol. 22, no. 1, p. 1, 2010. R. P. Carver, ―Percentage of unknown vocabulary words in text as a function of the relative difficulty of the text: Implications for instruction,‖ Journal of Literacy Research, vol. 26, no. 4, pp. 413–437, 1994. D. Hirsh and P. Nation, ―What vocabulary size is needed to read unsimplified texts for pleasure?,‖ Reading in a foreign language, vol. 8, pp. 689–689, 1992. I. S. P. Nation, ―How large a vocabulary is needed for reading and listening?,‖ Canadian Modern Language Review/La Revue canadienne des langues vivantes, vol. 63, no. 1, pp. 59–82, 2006. I. S. P. Nation, Teaching ESL/EFL reading and writing. Taylor & Francis, 2008. D. R. Hill, ―Graded readers in English,‖ ELT Journal, vol. 62, no. 2, pp. 184 -204, Apr. 2008. D. R. Hill, ―Survey. Graded readers,‖ ELT Journal, vol. 55, no. 3, pp. 300 -324, Jul. 2001. R. R. Day and J. Bamford, Extensive reading in the second language classroom. Cambridge Univ Pr, 1998. S. A. Berardo, ―The use of authentic materials in the teaching of reading,‖ Reading, vol. 6, no. 2, 2006. S. Luppescu and R. R. Day, ―Reading, dictionaries, and vocabulary learning,‖ Language Learning, vol. 43, no. 2, pp. 263–279, 1992. M. Bensoussan, ―Dictionaries and tests of EFL reading comprehension,‖ ELT Journal, vol. 37, no. 4, p. 341, 1983. F. M. Holley and J. K. King, ―VOCABULARY GLOSSES IN FOREIGN LANGUAGE READING MATERIALS,‖ Language Learning, vol. 21, no. 2, pp. 213-219, 1971. Y. Okuyama and H. Igarashi, ―Think-Aloud Protocol on Dictionary Use by Advanced Learners of Japanese,‖ The JALT CALL Journal, vol. 3, no. 1, pp. 45–58, 2007. C. Prichard, ―Evaluating L2 readers‘ vocabulary strategies and dictionary use,‖ Reading in a Foreign Language, vol. 20, no. 2, p. 2, 2008. V. J. Leffa, ―Making foreign language texts comprehensible for beginners: An experiment with an electronic glossary,‖ System, vol. 20, no. 1, pp. 63–73, 1992. J. N. Davis, ―Facilitating effects of marginal glosses on foreign language reading,‖ Modern Language Journal, vol. 73, no. 1, pp. 41–48, 1989. Y. F. Hong, ―The Effect of L1 Vocabulary Glosses on EFL Elementary School Students' Reading Comprehension and Reading Process,‖ 2006. G. M. Jacobs, Second language reading recall as a
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
function of vocabulary glossing and selected other variables. University of Hawaii, 1991. Y. H. Cheng and R. L. Good, ―L1 glosses: Effects on EFL learners ‗reading comprehension and vocabulary retention,‖ Reading in a Foreign Language, vol. 21, no. 2, pp. 119–142, 2009. J. H. Hulstijn, M. Hollander, and T. Greidanus, ―Incidental vocabulary learning by advanced foreign language students: The influence of marginal glosses, dictionary use, and reoccurrence of unknown words,‖ Modern Language Journal, vol. 80, no. 3, pp. 327–339, 1996. G. M. Jacobs, P. Dufon, and F. C. Hong, ―L1 and L2 vocabulary glosses in L2 reading passages: Their effectiveness for increasing comprehension and vocabulary knowledge,‖ Journal of Research in Reading, vol. 17, no. 1, pp. 19–28, 1994. G. M. Jacobs, ―What Lurks in the Margin: Use of Vocabulary Glosses as a Strategy in Second Language Reading.,‖ Issues in Applied Linguistics, vol. 5, no. 1, pp. 115–37, 1994. E. E. Joyce, ―Which Words Should Be Glossed in L2 Reading Materials? A Study of First, Second, and Third Semester French Students' Recall.,‖ Pennsylvania Language Forum, 1997. J. Luo, A study of the effects of marginal glosses on the reading comprehension of intermediate-level college students of French. Pennsylvania State University., 1993. F. L. Bell and L. B. LeBlanc, ―The language of glosses in L2 reading on computer: Learners' preferences,‖ Hispania, vol. 83, no. 2, pp. 274–285, 2000. M. H. Ko, ―Glosses, comprehension, and strategy use,‖ Reading in a Foreign Language, vol. 17, no. 2, pp. 125– 143, 2005. R. C. Palmer, A Comparison of the Effect of Glossed Selfinstruction Reading Materials and Traditional Teacher Fronted Instruction. Indiana University of Pennsylvania., 2003. M. Yoshii, ―L1 and L2 glosses: Their effects on incidental vocabulary learning,‖ Language Learning & Technology, vol. 10, no. 3, pp. 85–101, 2006. N. Miyasako, ―Does text-glossing have any effects on incidental vocabulary learning through reading for Japanese senior high school students,‖ Language Education & Technology, vol. 39, no. 1, pp. 1–20, 2002. T. Saragi, I. S. P. Nation, and G. F. Meister, ―Vocabulary learning and reading,‖ System, vol. 6, no. 2, pp. 72–78, 1978. J. H. Hulstijn and B. Laufer, ―Some empirical evidence for the involvement load hypothesis in vocabulary acquisition,‖ Language learning, vol. 51, no. 3, pp. 539– 558, 2001. J. H. Hulstijn, ―Retention of inferred and given word meanings: Experiments in incidental learning,‖ Vocabulary and applied linguistics, pp. 113–125, 1992. N. Nagata, ―The effectiveness of computer-assisted interactive glosses,‖ Foreign Language Annals, vol. 32, no. 4, pp. 469–479, 1999. S. Rott, J. Williams, and R. Cameron, ―The effect of multiple-choice L1 glosses and input-output cycles on lexical acquisition and retention,‖ Language Teaching
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44] [45]
[46]
[47]
[48] [49]
[50]
[51]
[52] [53]
Research, vol. 6, no. 3, p. 183, 2002. Y. Watanabe, ―Input, intake, and retention,‖ Studies in Second Language Acquisition, vol. 19, no. 03, pp. 287– 307, 1997. S. Gettys, L. A. Imhof, and J. O. Kautz, ―ComputerAssisted Reading: The Effect of Glossing Format on Comprehension and Vocabulary Retention,‖ Foreign Language Annals, vol. 34, no. 2, pp. 91–99, 2001. R. Aust, M. J. Kelley, and W. Roby, ―The use of hyperreference and conventional dictionaries,‖ Educational Technology Research and Development, vol. 41, no. 4, pp. 63–74, 1993. J. Li, ―Learning vocabulary via computer-assisted scaffolding for text processing,‖ Computer Assisted Language Learning, vol. 23, no. 3, pp. 253–275, 2010. J. W. Leloup and R. Ponterio, ―Vocabulary Support for Independent Online Reading.,‖ Language, Learning & Technology, vol. 9, no. 2, pp. 3–8, 2005. O. Streiter, J. Knapp, and L. Voltmer, ―Gymn@ zilla: Language learning with the Internet,‖ in Proceedings of TALC 2004 The sixth Teaching and Language Corpora conference, 2004. T. Cobb, ―Computing the vocabulary demands of L2 reading,‖ Language Learning & Technology, vol. 11, no. 3, pp. 38–63, 2007. D. A. Dokter, ―From Glosser-RuG to Glosser-WeB,‖ AlfaInformatica, University of Groningen. Retrieved from the WWW in November, 1998. A. M. Taylor, ―CALL-Based versus Paper-Based Glosses: Is There a Difference in Reading Comprehension?.,‖ CALICO Journal, vol. 27, no. 1, p. 14, 2009. M. A. Bowles, ―L2 Glossing: To CALL or Not to CALL,‖ Hispania, vol. 87, no. 3, pp. 541-552, Sep. 2004. L. B. Abraham, ―Computer-mediated glosses in second language reading comprehension and vocabulary learning: A meta-analysis,‖ Computer Assisted Language Learning, vol. 21, no. 3, p. 199, 2008. A. F. Abuseileek, ―Hypermedia annotation presentation: learners‘ preferences and effect on EFL reading comprehension and vocabulary acquisition,‖ CALICO Journal, vol. 25, no. 2, pp. 260–275, 2008. A. Martinez-Lage, ―Hypermedia technology for teaching reading,‖ Technology enhanced language learning, pp. 121–163, 1997. J. W. LeLoup and R. Ponterio, ―ON THE NET LiTgloss,‖ 2007. J. D. Riding, ―Statistical glossing, language independent analysis in bible translation,‖ Translating and the Computer, vol. 30, 2008. H. Ohtake et al., ―Development of a genre-specific electronic dictionary and automatic gloss-embedding system.‖ S. Alessi and A. Dwyer, ―Vocabulary assistance before and during reading,‖ Reading in a Foreign Language, vol. 20, no. 2, pp. 246–263, 2008. D. Geer, ―Statistical machine translation gains respect,‖ Computer, vol. 38, no. 10, pp. 18–21, 2005. F. J. Och, ―Statistical machine translation: Foundations and recent advances,‖ Tutorial at MT Summit X, Phuket, Thailand, 2005.
