Annotated Bibliography in On-line Character ...

Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers

Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers (DOI: 10.13140/2.1.3018.8322) This is a posting of a bibliography on gesture user interfaces, on-line character recognition (a.k.a. dynamic character recognition, a.k.a. pen computing), and related topics, including both hardware and software. I am posting it as a service to those with interest in the field. It may also be of special interest to anyone investigating any of the flood of patents in the areas of digitizer tablets, character recognition, tablet-PC GUIs, and multi-touch computing (such as the iPhone and Touch iPod). It covers the time period from approximately 1891 (first electronic tablet) through 1914 (first handwriting-recognition input system) to the present day. This compilation is copyright © Jean Renard Ward, 1992, 1996, 2003, 2005, 2008, 2009, 2011, 2013, 2015. Permission is hereby given to link to this list, or to cite it provided that notice of the source is given as stated below, that the full URL of this link is included, and also that you use my correct full name "Jean Renard Ward" is included with any such reference or link. Permission is hereby given to reproduce any diagrams or photographs in this collection for any purpose (to the extent that the materials are not subject to copyright by persons other than myself), including for the purposes of confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement: "Source: Annotated Bibliography in Touch/Pen Computing and Handwriting Recognition by Jean Renard Ward". The bibliography is broken up into several web pages, for convenience in printing: 1891 to 1984 to 1991 to 2001 to 2011 to Pictures

1970 1971 to 1975 1976 to 1980 1981 to 1983 1985 1985 to 1986 1987 to 1988 1989 to 1990 1993 1994 to 1995 1996 to 1998 1999 to 2000 2003 2004 to 2005 2006 to 2007 2008 to 2010 2013 2014 to 2015 not yet catalogued and Videos

When was Pen Computing invented? Check this out: Notes on the "unknown" history of Pen Computing (from a talk given to the Boston Computer Society in 1992) Dan Bricklin has posted a video of the presentation on youtube.com: http://www.youtube.com/watch?v=4xnqKdWMa_8. Note that there is a tendency for major vendors to re-name Pen Computing ("Tablet PCs", "Touch PCs", "Organic User Interfaces" are a few such names), which tends to obscure the historical record a bit. Who invented Pen Computing? You may have read something somewhere that indicated some organization like Microsoft Nestor Apple Newton MicroPad IBM PenCept GO Corporation Rand Corporation Linus PenPoint CIC Wacom GRAIL Casio Slate Graffiti ... had invented pen computing / tablet computing. After skimming through some of the references above, you might start to think some of those claims - particularly those by large corporations - could be slightly exaggerated: for

http://rwservices.no-ip.info:81/biblio.html[7/14/2015 9:45:52 PM]


example, inventing an improvement on spelling correction [Viterbi67] is not the same as inventing the first graphical word-processing program. Which is not to say that a particular system did not do some very interesting new things within the context of pen computing, such as [CarrR91a] and [PenCept85f]. Note: I find the very early references to technology interesting. For example, Handwriting recognition by a machine in real time, with a user writing with a stylus, goes back to before World War I: United States Patent 1,117,184, November 17, 1914 [GoldbergHE1914] United States Patent 1,311,384, July 29, 1919 [DrewWF1919] Optical character recognition (from a printed page) goes back to before the days of computers: United States Patent 1,838,398, December 29, 1931 [GoldbergE31a] Electronic Tablets were invented in the 19th century: United States Patent 461,472, October 20, 1891 [Gray1891b] United States Patent 491,347, February 7, 1893 [Gray1893b] (Note: Elisha Grey is best known in history as the person who may have invented the telephone before Alexander Graham Bell, but lost the patent dispute in a famous and controversial court decision) "Electronic ink" (the pen-computing kind, not the electronic display) is older than you think: "The RAND Tablet: A Man-Machine Graphical Communication Device", 1964 [DavisMR64b] (Note: This is the earliest published use of this term in pen-computing, AFAIK). Language knowledge being used in character recognition: The early references to the "Viterbi" algorithm are interesting to compare with more recent inventions on glyphs and context. [Viterbi67] So you think your recognition software is pretty good? ... look at this. Another interesting slant is the many claims of "near perfect" handwriting recognition algorithms in the earliest days of pen computing, such as [Simek67] and [DimondTL57a] (Note: At one of my pen-computing jobs in the late 1980s, we joked that Dimond had achieved 97% "perfect" handwriting recognition in 1957, and that the technology had been getting about 5% better every decade since: the trend seems to have continued unbroken, like some odd form of Moore's Law.) You can find almost anything using one of the major search engines on the Internet these days. However, here are some links to other pages with talking about the history of pen computing. (Please note that links expire, so these may not all be up-to-date.) Images of early PenComputing products (a work in progress -- I have lots more in my attic). A talk on the "unknown" history of Pen Computing given to the Boston Computer Society in 1992. Schomaker's MIAMI group in the "Esprit" project includes a detailed taxonomy and another set of references relating to pen computing. Andreas Hennig has posted another (shorter) bibliography relating to pen computing. Nottingham Trent University has a research group in this area. Gary Perlman set up an extensive on-line bibliography on human-computer interaction. Bill Buxton has been working on haptic user-interfaces at various institutions for a long time. You can find his most recent work at www.billbuxton.com The "Pen Report" has references to many of the commercial developments in one of the more recent pencomputing "crazes". It is listed as part of the AMUG Newton page, one of many user-group pages on the Web. There have been some original papers posted in odd places, such as this one by Scattolin Patrice. Of course, there is information on Wikipedia, but you should be cautious that the information can be biased towards a particular organization. Thank you,

EMail: [email protected]



Mirrors of this page can be found at: http://ruetersward/biblio.html http://users.rcn.com/rwservices/biblio.html http://rwservices.no-ip.info:81/biblio.html Back to homepage for Rueters-Ward / Zurück zur Heimatseite Rüters-Ward (Copyright © 20150711 21:31:40 EDT)


(1891..1970) History of Pen and Gesture Computing

History of Pen and Gesture Computing: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers References from the approximate years 1891 to 1970. This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015. Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement: Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward

Most of these are in my personal collection, either as physical items or electronic files. I can be contacted at [email protected]. Copyright © Tue Jul 14 21:08:40 EDT 2015 . Mirrors of this page can be found at: http://www.ruetersward.com/pens/biblio70.html http://users.rcn.com/rwservices/pens/biblio70.html http://rwservices.no-ip.info:81/pens/biblio70.html (Generally most up-to-date) penhist (Prev page)

Main Page

(Next page) biblio75

[ARRL36a] American Radio Relay League, staff "The Radio Amateur's Handbook, 14th ed.", American Radio Relay League, 1936 Amateur radio electronics and theory [ARRL56a] American Radio Relay League, staff "The Radio Amateur's Handbook, 33th ed.", American Radio Relay League, 1956 (hardcopy book) Amateur radio electronics and theory [ARRL68a] American Radio Relay League, staff "The Radio Amateur's Handbook, 45th ed.", American Radio Relay League, 1968 (hardcopy book) Amateur radio electronics and theory [AbmaJS61a] Abma, John S.; Mason, Lawrence, J.; and Rice, David Reagan "Optophone", United States Patent 3,007,259, November 7, 1961 Reading devices for the blind: Optophones (auditory phones) known in the art. Probe with rollers on tip (for scanning text) and set of photocell sensors, sensors directly connected to autitory tone generator. [AlbergaCN67a] Alberga, C.N. "String similarity and misspellings", CACM, Vol 10 No 5, May 1967, pp 302-313 Context string correction, spelling correction: review of several algorithms, some of which turned out to be trash [AndersonRH68a] Anderson, Robert H. "Syntax-Directed Recognition of Hand-Printed Two-Dimensional Mathematics", in "Interactive Systems for Experimental Applied Mathematics", pp 436-459, Klever, M. and Reinfelds, J., editors, Academic Press, New York, 1968. Also Proc. ACM Symposium on Interactive Systems for Experimental Applied Mathematics, 1967, pp. 436-459 Refers to min/max extents, "typographical" centroid of characters; User interface for mathematical symbol parsing, 2-D symbol parsing, sketch input [AndersonRH70a] Anderson, R.; Bator, R; Gagan, R.; Meads, J.; and Metrick, L. "Interactive Specification of Data Displays", NASA Contractor Report CR-1628, June 1970 Graphical display language: refers to a Symbol recognizer rather than a handwriting recognizer. Mentions Ledeen recognizer from a private communication at Harvard, 1968. Describes both fixed and

http://rwservices.no-ip.info:81/pens/biblio70.html[7/14/2015 9:45:54 PM]


adaptive/trainable recognizers (a bit naive about trainability). List set of 8 symbols (gestures?) for command input in one application, plus two others: e.g. Scrub/scratch-out for erase. [Applicon70a] Applicon Incorporated (Harry Lee) "Applicon CAD System with Trainable Hand-Drawn Symbol Recognition (video)", Applicon - Video available on YouTube, 2015 Marketing demonstration video of Applicon CAD/CAM system, ca. 1970. Shows gestures on tablet: alpha gesture for zoom out, free-hand electronic ink lines straightened (sketch recognition). Macro definition with "teach" mode (records commands)., assign to any of fixed set of recognized gestures. [Applicon70b] Applicon Incorporated (Harry Lee) "Applicon Design Assistant System (video)", Applicon - Video available on YouTube, 2015 (three parts) Longer demonstration video of Applicon CAD/CAM schematic/architecture drawing system, ca. 1970. Display is viewport onto larger virtual drafting table. Mentions use of gestures on tablet allow user to focus attention on display. Triangle gesture, inverted V (caret), Z gesture for zoom out, C gesture to center, R gesture (with second tap stroke) for resistor component, select gesture is diagonal line gesture on object. N, gestures. "Teach" mode assigns commands (command + X/Y position ) to gestures with location. Multilayer drawings for different classes of objects / views/ PC board layers. "Graphical entities" (objects), can be represented on multiple levels: editing definition changes all instances in drawing (compare with Smalltalk?). Snap-to grid. Macro-commands by single "stroke" (gesture) or single keyboard key. Text entered from keyboard, but can be edited with stylus gestures (no handwriting recognition). Shows drafting-table-size digitizer tablet (compare with tabletop?), electromechanical digitizer, for digiting existing drawings (movable keypad referred to as keyboard interface only). Dual displays: one for "world view" (Compare with Telesis) [ArkadevAG67a] Arkadev, A.G. and Braverman, E.M. "Computers and Pattern Recognition", translated from Russian by W. Turski and J.D. Cowan, Thompson Book Company, Washington, D.C., 1967 Papers on one character "invading the space" of another, compare to Write 1950 book on Arabic Numerals. Description of Perceptrons, early neural-net technology. [AttneaveF54a] Attneave, F. "Some informational aspects of visual perception", Psychology Review, Vol 61 No 3, 1954, pp 183-193 Cited in Teh89 , O'Callaghan70, cited in Lipscomb91 as Atteneave Features: Points of high curvature (corners) are dominant points in human pattern recognition of characters, not local maxima/minima for recognition [Baecker69] Baecker, Ronald M. "Picture-driven Animation", Proc. AFIPS Joint Computer Conference, May 14-16 1969, SJCC, pp 273-288 Sketch animation system: uses gestures? [BakisR68a] Bakis, Raimo; Herbst, Noel M.; and Nagy, George "An Experimental Study of Machine Recognition of Hand-printed Numerals", IEEE Trans. Systems Science and Cybernetics, Vol SSC-4 No 2, July 1968, pp 119-132 Statistical adaptive classifier for handwriting: takes vectors of features, trains self to data base by finding best (lowest error confusion) weighting Data base collections: most are too small ("greenhouse data sets"), therefore low accuracy Formal training of user not acceptable for handwriting recognition 7000 and 30000 character collections from handwriting "back-room" forms not large enough Pre-processing of handwriting before recognition: scaling, shear normalization (tilt), Using too many feature measurements can lower performance, since the probability is higher that some features are of low effectiveness Gives actual/typical substitution errors from data collection on handwriting recognition [BeckerFK60a] Becker, Floyd K. "Electrographic Transmitter", United States Patent 2,925,467, February 16, 1960 resistive sheet telautograph/telewriting tablet with stylus, non-linear distortions from discrete terminals corrected by using multiple terminals along the edge, and generating a signal potential at each one that matches the linear voltage gradient. Storage tube display of both local writing, and of remote writing by another user, for collaborative work. Compare with later whiteboard systems? [BernsteinMI64] Bernstein, Morton I. "Computer recognition of on-line, hand-written characters", Memorandum RM-3753-ARPA, The RAND Corporation, Santa Monica, California October 1964 Also position and size invariant (no punctuation): Mentions that he does nothing on stroke segmentation Eight-direction chain direction codes (rotationally invariant), with endpoints for rotation-dependent added Features are eight-direction chain codes, quantized segment length, exhaustive table of types of sequences (like Penverter SME types?), followed by eight directions for end-point relative positions [BernsteinMI67a] Bernstein, M.I. "An On-Line System for Utilizing Hand-Printed Input Progress Report", System Development Corporation, Santa Monica, California, Report No NASA-CR-86108; TM/L/3052/001/00, 18 December 1967 Handwriting recognition software and user-interface: Appendix B refers to using spelling dictionary to do character segmentation / stroke parsing, and corner detection (and not just local minima/maxima?) [BernsteinMI67b] Bernstein, M.I. "Flow charts for recognizing hand printed characters on-line in real time operation", NASA Technical Report NASA-CR-86108; TM L-3052 001 00, Dec 18, 1967 (abstract only)



Handwriting recognition [BernsteinMI68a] Bernstein, M.I. and Howell, H.L "Hand-Printed Input for On-Line Systems: Final Report for Phase I", System Development Corporation, TM-(L)-3964/000/00, Santa Monica, California, April 1968. (abstract only) Trainable recognition: touch-display / electronic ink, by rear-projecting image onto digitizer tablet. Suggests free-form (2-Dimensional) input, not text entry is most useful application: implies that gestures are the useful application over handwriting recognition. [BernsteinMI68b] Bernstein, Morton I., and Howell, H.L. "Hand-Printed Input for On-Line Systems", System Development Corporation, TM-3937/000/00, Santa Monica, California, April 1968 (Abstract only). User interface: suggests handwriting best used for free-form/two-dimensional input, not linear text. [BernsteinMI68c] Bernstein, Morton I. and Williams, T.G. "A two-dimensional programming system", IFIP Congress, Edinburgh, Scotland, August 5-10, 1968. pp C84-C89 Cited in MillerGM69 on mathematical/flow-chart user interface parsing [BernsteinMI68d] Bernstein, Morton I. "A Method for Recognizing Handprinted Characters in Real Time", in Pattern Recognition, proceedings of IEEE Workshop on Pattern Recognition, L.N. Kanal, editor, Thompson Press, Washington, D.C., 1968, pp 109-113 Smoothing, filtering, corner detection features, minima and maxima of character, types of component strokes (chain codes?) Handwriting tablet data rate: 2.5 msecs (400 points/sec.) [BernsteinMI69a] Bernstein, M.I., and Williams, T.G. "A Two-Dimensional Programming System", Proceedings 1968 IFIP Congress, North Holland Pub. Co, page 586, 1969 [BernsteinMI69b] Bernstein, M.I. and Howell, H.L "Hand-Printed Input for On-Line Systems: Final Report for Phase I", System Development Corporation, NASA Contractor Report CR-1284, March 1969 Trainable recognition: touch-display / electronic ink, by rear-projecting image onto digitizer tablet. Suggests free-form (2-Dimensional) input, not text entry is most useful application: implies that gestures are the useful application over handwriting recognition. Recognition uses 8-direction chain codes, zone recognition (and corner detection?) [BernsteinMI70a] Bernstein, M.I. "Software for Interactive Graphics Input", in Computer Handling of Graphical Information, Richard D. Murray, Editors, 1970, SPSE Press, pp 28-40 Claims digitizing tablet invented by Dr. Herbert Teager of MIT around 1960, before Rand tablet Describes different tablet filter pre-processing algorithms: square-window, diamond-window, circular-window, doublesquare, etc. Handwriting recognition with electronic ink on a RAND tablet/display combination, 1970. [BlackwellFW67a] Blackwell, Frederick W. "An On-Line Symbol Manipulation System", Proceedings of the 22nd National ACM Conference, Thompson Book Company, Washington, D.C., 1967, pp 203-209 Mathematical user interface for symbol-manipulation such as algebra: appears to use a keyboard for input, predecessor to Blackwell70 which was two-dimensional hand-written notation [BlackwellFW69a] Blackwell, Frederick W. and Anderson, Robert H. "An On-line Symbolic Mathematics System using Hand-Printed Two-Dimensional Notation", Proc. ACM '69 24th national conf., pp. 551-557, . Also RAND Memorandum RM-6018-PR, 1970 User-interface on handwriting and mathematics: parser, as well as recognition, can be extended by user (new operators, new syntax?) using Anderson68 Some user-interface questions on mathematics left open: how to show intermediate results; very large expressions online manipulation of symbolic mathematical formulae: user hand-prints formulas, in ordinary two-dimensional notation [BledsoeWW59a] Bledsoe, W.W. and Browning, J. "Pattern recognition and reading by machine", Proceedings of the East Joint Computer Conference, pp 225-232, December 1959 Had spelling dictionary with probabilities of words; compare to later context/dictionary solutions for handwriting recognition [BledsoeWW66a] Bledsoe, W.W. and Browning, J. "Pattern recognition and reading by machine", in Pattern Recognition, L. Uhr, editor, Wiley, New York, 1966, pp 301-306 Cited in Srihari83 : same as Bledsoe59? [Bonner67] Bonner, R.E. "Segmentation Method and Apparatus", United States Patent 3,344,399, assigned to International Business Machines Corporation, New York, New York, September 26, 1967 Uses eight-vector-direction chain codes Claims related to curve-following [Brenner70] Brenner, A.E. and deBruyne, P. "A Sonic Pen: A Digital Stylus System", IEEE Trans. Computers, June 1970, pp 546..548 Sonic digitizer using plane microphones: three-dimensional digitizer: states 1mm resolution. [BrownCD70] Brown, Curtis Del "A Real Time Handprinted Character Recognizer", M.S. Thesis, M.I.T, June, 1970 Mentions Graham as standard set of handwriting styles (for Morse code) intended to avoid conflicts in form handwriting recognition using properties of curvature and chain codes reviews of Teitelman "sequence" recognizer, Groner at Rand corporation chain-code recognizer, Graham at Sylvania [BrownRM64] Brown, R.M. "On-line Computer Recognition of Handprinted Characters", IEEE Trans. Electronic



Computers, Vol EC-13 No 12, December 1964, pp 750-752 alphanumeric and mathematic symbols, uses term "dynamic character recognition", stroke-direction recognition using zones. Cited in Litvin82, FoleyJD84, FoleyJD82. Used zone-based input device of conductive zones, compare with stylator (not a real tablet). [BushV45] Bush, Vannevar "As We May think", The Atlantic Monthly, July 1945 Vannevar Bush essay that introduced the concept of the Memex, a hyperlinked information archive. Other references show handwriting (recognition?) input, and a telautograph or digitizer tablet. May have been anticipated by Goldberg Patent in 1931. [Calabi69] Calabi, L. and Hartnett, W.E. "A family of codes for the correction of substitution and synchronization errors", IEEE Trans. Information Theory, Vol IT-15 No 1, pp 102-106 Cited in Kruskal83 String correction, spelling correction distance [CaldwellSH59a] Caldwell, S.H. "The Sinotype - a machine for the composition of Chinese from a keyboard", Journal of the Franklin Institute, Vol 267 no 6, June 1959 Chinese Kanji input keyboard, using one keyboard stroke per character stroke. Notes that two types of strokes (vertical / horizontal) very high frequency (50%) in Chinese writing, with diagonal 75%: keys duplicated for either hand. No computer processor, all discrete hardware components, binary codes implemented electromechanically. [CameronSH67a] Cameron, S.H. et al "DIALOG: A Conversational Programming System with a Graphical Orientation", CACM, Vol 10 pp 349-357 DIALOG - Illinois Inst Tech, 1966. Electrosketch Interactive math/geometry using graphics tablet: cited for setting up command/translation areas on a digitizer tablet in Eichen73. Dynamic/virtual keyboard, autocompletion of typed (handwritten?) commands. [CarlsonG66a] Carlson, Gary "Techniques for replacing characters that are garbled on input", Proc. SJCC, 1996, pp. 189-192 Dictionary connection of miss-recognized (or miss-typed) characters using digrams and trigrams. [CaseyR66a] Casey, R. and Nagy, G. "Recognition of printed Chinese characters", IEEE Trans. Electronic Computers, Vol EC-15, February 1966, pp 91-101 Cited in Agui79 [ChampionML31a] Champion, M.L. and Gregg, J.R. "Gregg handwriting", Gregg publishing company, 1931; Bibliography by T.R. Davis 1931 reference to shorthand writing styles? [ChengG68] Cheng, G., Ledley, R.S.; Pollock, D.; and Rosenfield, A. "Pictorial pattern recognition", Thompson Book Company, Washington D.C., 1968 Cited in O'Callaghan70 [CheslerLG67a] Chesler, Leonard G. and Turn, Rein "The Application of On-Line Graphical Techniques for Programming and Operating a "Moving Network" Monitoring Display", RAND Corporation Memorandum RM-5183PR, 1967 GUI description for RAND Graphic Input Tablet for creating a real-time moving network display of spacecraft checkout operations [ChodrowMM66] Chodrow, M.M., Bivona, W.A. and Walsh, G.M. "A Study of Hand-Printed Character Recognition Techniques", Rome Air Development Center Technical Report No RADC-TR-65-444, February 1966 Cited in MunsonJH68 as showing handwriting recognition in 1965 [ChristensenC67a] Christensen, Carlos "An Example of the Manipulation of Directed Graphs in the AMBIT/G Programming Language", Proc. ACM Symposium on Interactive Systems for Experimental Applied Mathematics, 1967, pp. 423-435 Visual programming. Compare with LabView, other AMBIT references for discussion of user interface on touchscreen. [ChuangPC70a] Chuang, P.C. "Recognition of Handprinted Numerals by Two-Stage Feature Extraction", IEEE Trans. on Systems Science and Cybernetics, april 1970, pp. 153-154 OCR static handwritten character recognition using two stages of feature extraction: low-level (line thinning, local coordinates of points, second state is branching and union (crossing?) points. [ClarkDW70a] Clark, D.W., Connell, David B., Dimeo, Michael P., Opitz, Bruck K., Sammon, John W. jr., and Sanders, Jon H. "Handprinted Character Recognition Techniques", NTIS Report AD 876 875, RADC Technical Report RADC-TR-70-206, October, 1970 (abstract only) On-line handprinted/handwritten character recognition, constrained to Standard Character Format. Claims 99% correct recognition (without rejection). [ClemensJK65a] Clemens, Jon Kaufman "Optical Character Recognition for Reading Machine Applications", Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1965 Optical character recognition, generates artificial stroke sequence similar to on-line character recognition. Cited in MunsonJH68 as showing handwriting recognition in 1965



[ClowesMB68a] Clowes, M.B. "Transformational grammars and the organization of pictures", paper presented to the PISA Conference on Automatic Interpretation and Classification of Images, Pisa, Italy, August, 1968 Cited in O'Callaghan70 [ColemanML67a] Coleman, M.L. "Design for a text editor", Memo CCP 238, Carnegie-Mellon University, Pittsburgh, Pennsylvania, 1967 See also Coleman69 user-interface on gestures / handwriting symbols [ColemanML69a] Coleman, Michael L. "Text editing on a graphic display device using hand-drawn proofreader's symbols", from Pertinent Concepts in Computer Graphics: Proceedings of the 2nd University of Illinois Conference on Computer Graphics, Faiman, M. and Nievergelt, J., editors, University of Illinois Press, Urbana, Illinois, 1969, pp 282-290. Also in M. Faiman and J. Nievergelt (Eds.), Pertinent Concepts in Computer Graphics User interface: gesture/command symbols for proofreading/editing. User-interface/gesture nine symbols for text: crossing out to erase, insert, transpose, move, remove spaces / close up, insert space, scroll up, scroll down, delete. Features for gestures / handwriting proofreading marks: closure. User-interface: text entry via voice, tolerate higher error rate on voice recognition if gesture-editor is convenient. [CraneHD60a] Crane, H.D. "Sequence Detection Using All-Magnetic Circuit", IRE Trans. on Electronic Computers, Vol EC-9, No 2 June 1960, pp 156-160 Appears to be early version of SRI direction/accelerometer pen by Crane, with hard-wired circuits to respond to specific sequence of chain-codes. [CrookMN63a] Crook, M.N. and Kellogg, D.S. "Experimental Study of Human Factors for a Handwritten Numeral Reader", IBM Journal of Research and Development, January 1963, pp 76 ..78 Cites G.G. Neal Wright. 90% correct recognition for somewhat constrained handwriting recognition on groups of subjects, no details on the type of recognition. Data scanned optically OCR. Subject were: students, sales clerks, sales clerks. Worst recognition success was with high-school males. [CurryJE69a] Curry, James E. "A Tablet Input Facility for an Interactive Graphics System", Proceedings IJCAI 1969, AFIPS/ACM International Conference on Artificial Intelligence, pp 33..40, May 1969 Trainable "symbol recognizer" for gestures for the TX-2 computer. Cited in RubinSM94 and Buxton05: Compare with Ledeen recognizer, gesture input? Example is a graphical editor for analog circuits (transistors, resistors, etc.) [DamerauFJ64a] Damerau, Fred J. "A technique for computer detection and correction of spelling errors", CACM, Vol 7 No 3, March 1964, pp 171-176 Context via a spelling dictionary: cites to earlier work to 1960 [Danna69] Danna, S.R. "Signature Identification Instrument", United States Patent 3,480,911, November 25, 1969, assigned to Conetta Manufacturing Co. Incorporated, Stamford, Connecticut. Signature verification using force/pressure [DavisMR64a] Davis, M.R. and Ellis, T.O. "The RAND Tablet: A Man-Machine Graphical Communication Device", AFIPS Fall Joint Computer Conference #26, part 1, 1964, Spartan Books, Baltimore, Maryland, pp 325-331 Early reference to RAND tablet: Gray code electronics. See also historical notes in 2008 Ware paper. User interface/ ergonomics: says separate tablet and display not a problem for user after a couple of minutes, even for signatures: even has advantage that hand does not obscure drawing. See also interview notes with Uncapher User interface: earliest reference to electronic ink by name. Gestures? RAND Tablet digitizer was capacitive-coupled/electrostatic. [DavisMR64b] Davis, M.R. and Ellis, T.O. "The RAND Tablet: A Man-Machine Graphical Communication Device", Rand Memorandum RM-4122-ARPA, August 1964: ARPA Order No. 189-61 Early reference to RAND tablet [DayLF1910] Day, L.F. "Alphabets Old and New", B.T. Batsford, Limited, London, 1910 Examples of alphabet/character styles from historical sources: includes some examples of Greek, Coptic etc. alphabets, mostly ancient styles of Roman alphabet. [DeHaanWR66a] DeHann, William R. "The Bell Telephone Laboratories automatic graphic schematic drawing program", Proc. of DAC '66 SHARE design automation project Early CAD / schematic sketching program: input is by sketch recognition on coordinate paper, rather than touchscreen or lightpen digitizer [DeringerD68] Deringer, D. "The Alphabet", Funk and Wagnalls, New York, 1968 Background on historical variability of the written alphabet [Deutsch67] Deutsch, P. and Lampson, B.W. "An Online Editor", CACM, Vol 10 No 12, 1967, pp 793-799 See also Coleman69 on gesture user-interface. QED keyboard-command text editor had two-letter/twocharacter macros/"gestures" for inserting stored phrases and text. [DevoeD67a] Devoe, D. "Alternatives to Handprinting in the Manual Entry of Data", IEEE Trans. Human Factors in Electronics, Vol HFE-8 No 1, January 1967, pp 21-32 Cited in FoleyJD84. States that handprinting on forms (keypunch sheets) first step in data entry, regardless of final data entry (keyboard, mark sense forms, also compared with handwriting -- script?)



[DevoeD68a] Devoe, Donald D., and Graham, Donald N. "Evaluation of Handprinted Character Recognition Techniques", Sylvania Electronics Systems Final Report No F-6171-1; RADC-TR-68-103, May 1968 (Abstract only). cited in Goodale83 User interface for handwriting: visual feedback has no effect on writing style, but is essential for editing and correcting Handwriting recognition: user perception of adaptive versus constrained is about the same (adaptive itself is constraining?) [DeweyG23a] Dewey, G. "Relative Frequency of English Speech Sounds", Harvard University Press, Cambridge Massachusetts, 1923 Cited in Srihari83: on spelling/context correction [DickensonWE60] Dickenson, W.E. "A Character-Recognition Study", IBM Journal of Research and Development, July 1960, pp 335..348 Simulation (no real data) for recognition for ten "specially designed digits", using X-Y scans. Special digit shapes for OCR are intentionally rectangular. [DimondTL57a] Dimond, T.L. "Devices for reading handwritten characters", Proceedings of Eastern Joint Computer Conference, pp 232-237, December 1957 Early paper on handwriting character recognition: Stylator (stylus translator), a zone-based tablet, which was earlier than RAND digitizer tablet, which was full Cartesian coordinates. Describes cartesian-coordinate grid for character recognition, zone-based recognizers. [DimondTL58a] Dimond, Tom .L. "Experimental device for reading handwritten numbers", Electronic Equipment, Vol 6 No 1, January 1958, pp 6-7 Stylator zone-based handwriting recognition input device [DimondTL63a] Dimond, T.L. "Machine Reading of Handwritten Characters", United States Patent 3,108,254, October 22, 1963, assigned to Bell Telephone Laboratories, New York, New York. Early patent on five-zone/sector recognition. Constraints: user writes around two vertically-oriented dots with a stylus. See also Stylator. [DixonNF65] Dixon, N.F. "Subliminal Perception: The Nature of a Controversy", McGraw-Hill, 1965 (Date approximate): threshold for recognition in humans will vary [DoyleW60a] Doyle, W. "Recognition of sloppy, hand-printed characters", Proceedings of Western Joint Computer Conference, 1960, pp 133-142. Original date 10 December 1959, Revised 2 May 1960 Cited in Genchi68 early OCR handwriting recognition for sloppy uppercase handprinting. Notes that adapting/training to real-world data can lead to different fit / evaluation than comparing from ideal representation. Feature extraction, then scoring. [DrewWF1919] Drew, W.F. "Electric Keyboard for Calculating Machines, Type Writing Machines, and the Like", United States Patent 1,311,384, July 29, 1919 on a keyboard terminal that uses a stylus to touch conductive key contacts: relate to digitizer technology, or handwriting input with a stylus. Cited in Irland64. [DreyfussH59a] Dreyfuss, Henry (Associates) "H1 Anthropometric data - Adult Male seated at Console", Henry Dreyfuss & Associates 1959. The Measure of Man and Woman: Human Factors in Design. Whitney Library of Design, 18 East 50th Street, New York 22, New York. Ergonomic figures for human reach, console/desk design [DreznerSM66a] Drezner, Stephon M., Gatto, O.T. and Wisneiwski, T.D. "Report on a Demonstration Test of a Computer-Assisted Countdown", RAND Corporation Research Report RM-5005-NASA, 1966 Describes user interface for a prototype stylus GUI, that enables the executive (user?) to act online with a simulation of the countdown: he can make changes to the script or to the simulation [DudaRO68a] Duda, R.O. and Hart, P.E. "Experiments in the recognition of hand-printed text: part II -- context analysis", Proceedings of Fall Joint Computer Conference, 1968, Thompson Books, Washington, D.C., pp 11391149 Context in handwriting (handprinted) recognition on Fortran coding sheets. Applying syntactic rules (GOTO 5 more likely than GOTO S), limited use of semantics (variable names unlikely to appear only once in a program), syntax of mathematical expressions. Cited in Tappert's bibliography, FoleyJD82. [DunnRankinP68a] Dunn-Rankin, Peter "The Similarity of Lower-Case Letters of the English Alphabet", Journal of Verbal Learning and Verbal Behavior, Vol 7, pp 990-995, 1968 Human recognition performance on lower-case letters. Analysis of critical/distinguishing features of letters: compare with Kuklinski and Shillman [DunnRankinP68b] Dunn-Rankin, Peter and Leton, Donald A. "Hierarchical Grouping Procedures for Comparing Indices of Letter Similarity", Perceptual and Motor Skills, Vol 27, 1968, pp. 457-468 Artificially/theoretically determine similarity (not a confusion matrix) of lower-case characters to determine which character pairs had similar feature. Contrasted with empirical results (with human subjects) matching which pairs characters were similar: similar results. [DunnRankinP68c] Dunn-Rankin, Peter; Leton, Donald A.; and Shelton Velma F. "Congruency Factors Related to Visual Confusion of English Letters", Perceptual and Motor Skills, Vol 26, 1968, pp. 659-666



Experimental study of which features of lowercase letters children learning to read are most reliable, which cause confusion. Break (O and C) cause little confusion, but rotation (e.g. p, b, d; n, u) is major cause of confusions. Simple comparison of similarity using transparent overlays good predictor of confusion matrix of pairs of characters. [DydykRB70] Dydyk, R.B., and Kalra, S.N. "Recognition of handprinting", Proceedings of 2nd International Conference on Systems Science, January 1969, pp 333-336 Cited in Blatt88: 60% character recognition rate [EarnestLD62] Earnest, L.D. "Machine recognition of cursive writing", in "Information Processing", C. Cherry, editor, Butterworths, London England, 1962, pp 462-466 Cited in MacDonald 1966, Tappert 1980. [EdenM61a] Eden, Mark and Halle, M. "The characterization of cursive writing", Information Theory, C. Cherry, editor, Butterworth, London, 1961, pp 287-299 Cited in Sayre73 [EdenM62a] Eden, Mark "Handwriting and pattern recognition", IRE Trans. Information Theory, Vol IT-8 No 2, pp 160-166, 1962 Cited by Ehrich78 as saying: 33 basic stroke segments in cursive writing [EdenM64a] Eden, Mark and Mermelstein, P. "Experiments on computer recognition of connected handwritten words", Information and Control, Vol 7., pp 255-270, 1964 Yasuhara quotes his paper along with some Eden studies of cursive effects on handwriting styles: first author is Mermestein (see file) [EdenM68a] Eden, Mark "Other Pattern Recognition Problems and Some Generalizations", in "Recognizing Patterns", edited by Paul A. Kolers and Murray Eden, Massachusetts Institute of Technology Press, Cambridge, Massachusetts 1968, pp 196-225 Any successful recognition machine will ultimately use principles derived from or relevant to the operation of living systems (implementation may differ) Circa 1968: Predictive value of most theoretical models for character recognition is near zero. Validation (testing) of theory has been very meager. (Note: Hype). General theory of pattern recognition based on human recognition and processes, not statistical models. Pragmatic pattern recognition (Murray Eden): depends onpurpose of classification. Why statistical methods for recognition will not hack it. Problems of sample bias and feature weighting in statistical/adaptive recognition. [EdisonTA1880a] Edison, T.A. "Electric-lamp", United States Patent 223,898, January 27, 1880 Original Thomas A. Edison electric light bulb. Carbon thread filament. [EdwardsAW64a] Edwards, A. Wood and Chambers, Robert L. "Can A Priori Probabilities Help in Character Recognition?", Journal of the ACM, Vol. 11, No. 4 (October, 1964), pp. 465-470 Using statistics on letter pairs as context, slight (but significant) improvement in OCR character recognition (using simple recognizer). Compare with Viterbi algorithm. [EinsteinA30] Einstein, Albert and Szilard, Leo "Refrigeration", United States Patent 1,781,541, November 11, 1930 U.S. Patent for German patent on a refrigerator by Albert Einstein and Leo Szilard: no moving parts. Not the original Swiss patent with Einstein's first wife Mileva. [EllisTO66a] Ellis, T.O. and Sibley, W.L. "On the Development of Equitable Graphic I/O", The RAND Corporation, Report P-3415, Santa Monica, California, July 1966 NTIS citation index: User interface with handwriting [EllisTO68a] Ellis, T.O., and Davis, M.R. "Digital computer and graphic input system", United States Patent 3,399,401, assigned to the United States of America, August 27, 1968 Cited in Tappert's patent list of 1986 Patent on putting a handwriting digitizer in front of a separate display in one system. Capacitive coupling digitizer tablet using wire grid Digitizing tablet with capacitively coupled stylus to X/Y grid, using Gray code encoding of position on on-line/dynamic handwriting recognition. Having touch surface separate from display screen advantageous: user adapted quickly, and display not occluded or obscured by hand in the way (integration of touchscreen and display) Cited in Fox88: on annotation drawing/handwriting [EllisTO68b] Ellis, T.O., and Sibley, W.L. "On the Problem of Directness in Computer Graphics", The RAND Corporation, Report P-3697, March 1968 Commentary to accompany a film (!!) entitled "The GRAIL Project". See also RAND history by Ware, 2008 System depends heavily on real-time symbol recognition: Functions can be invoked by pressing "virtual buttons" on the tablet. Bulk of paper is CRT display pictures. Precursor to work later named "direct manipulation"? Possible example of gesture commands? [EllisTO69a] Ellis, T.O., Heafner, J.F., and Sibley, W.L. "The GRAIL Project: An Experiment in Man-Machine Communications", The RAND Corporation, RM-5999-ARPA, Santa Monica, California, September 1969 Bernstein70 cited this on user-interface for flow-chart symbols User-interface: un-highlight/dim commands



not valid at the moment Pen-computing GUI for creating and editing flowcharts Refers to a video of the handwriting recognition system in operation from 1968 Squiggle/rubout erase gesture. Flow-chart input on Rand Tablet, recognize rectangles, triangles, lines. Note: file folder also contains later references to the video of the Grail gesture/handwriting user interface, such as Alan Kay's talk. [EllisTO69b] Ellis, T.O., Heafner, J.F., and Sibley, W.L. "The Grail Language and Operations", The RAND Corporation, RM-6001-ARPA, Santa Monica, California, September 1969 Context sensitive (position) for manipulating on flowchart symbols vs. adding characters Handwriting user interface: multiple character erasure mark, caret gesture for insertion, writing on a "viewing window" Electronic ink / gesture user interface: has list of basic editing functions for handwriting (replacement, placement, insertion, deletion, erasure). Handwriting recognition for alphanumerics, flowchart parts, erase mark (zig-zag), special symbols. Virtual buttons in a handwriting/sketch input user interface, compare with Kaplow. Shows "scribble" (scratch out) gesture/character for erasure recognition, symbolic ink ("electronic ink") tracks Mathematical expression, flow-chart, and text editing (insert, move, delete, line detection, replacement) user interface with handwriting [EllisTO69c] Ellis, T.O., Heafner, J.F., and Sibley, W.L. "The Grail System Implementation", The RAND Corporation, RM-6002-ARPA, Santa Monica, California, September 1969 User interface for handwriting: display divided into different virtual areas Mathematical expression and text editing user interface with handwriting (early gestures?) Data structures to denote properties implied by the picture, plus positional information to relate stylus to the other forms Operating system description: garbage collection (or the avoidance thereof), real-time interrupt processing, etc. [EllisTO69d] Ellis, T.O. et al "Grail System", c2.com/cgi/wiki/wiki.cgi?GrailSystem, fetched 2008 Wiki page (with links) on the Grail system and graphical programming languages: see other Ellis references 1969 [Engelbart70a] Engelbart, Douglas C. "X-Y Position Indicator for a Display System", United States Patent 3,541,541, November 17, 1970 Original (?) patent on a mouse, using two rotary wheels at right angles connected to rheostats [EnglishWK67a] English, W.K., Engelbart, D.C., and Berman, M.L. "Display Selection Techniques for Text Manipulation", Trans. of IEEE, Vol HFE-8, page 5-ff. Text selection UI with different devices: light pen (stylus digitizer), Grafacon (commercial version of Rand Tablet digitizer), joystick in rate and absolute modes, knee-control level, SRI "mouse". Based on two years actual experience. Mouse generally considered best. Cited in NewmanWM68a: as the original SRI "mouse", for a user-interface [EvansSutherland70a] Evans and Sutherland Computer Corp. "Evans and Sutherland Line Drawing System Model 1 System Reference Manual", Evans and Sutherland Computer Corporation, November 1, 1970, U0800-1-1 Early graphical system: Windows and Views, 3D projections, drawing instructions similar to Turtle. Uses tablet input (LDS-1) [FBI64a] Federal Bureau of Investigation ""Louie Louie" (The 60's Song) Phonograph Record "Louie Louie" Distributed by Limax Music", U.S. Federal Be FOIA copy of FBI file (redacted) and lab reports on investigation of the lyrics of "Louie Louie". [FederJ65a] Feder, J. and Freeman, H. "Segment fitting of curves in pattern analysis using chain correlation", NTIS report AD619525, March, 1965 Cited in DavisLS77 on segment fitting for shape recognition: feature is match of chain codes along a segment [FingerFW47a] Finger, Frank W. and Spelt, David K. "The Illustration of the Horizontal-Vertical Illusion", Journal of Experimental Psychology, Vol 37, pp 243-250 Cites earlier references for horizontal/vertical perception, back to 1912, 1893. [FittsPM54a] Fitts, Paul. M. "The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement", Journal of Experimental Psychology, Vol 47, pp. 381-391, 1954 Fitts' Law [Flannery1918] Flannery, J.R. and Dodds, E.I. "Means for Displaying Illuminated Words or Characters", United States Patent 1,285,098, assigned to Flannery Bolt Company, Pittsburgh, Pennsylvania, November 19, 1918 An electric pencil moved over contacts on a digitizer board to trigger display of characters with lamps [FluhrFR65a] Fluhr, F.R. "Light-Pencil Coordinate Positioner", NRL Memorandum Report 1592, March 1965, U.S. Naval Research Laboratory, Washington, D.C. Light-pen (?) device for use with radar display screens: functions by retrace insertion, generating marker at point of light-pencil on screen. [Frank69] Frank, A.I. "Data Entry System", United States Patent 3,487,371, assigned to ScanData Corporation, Norristown, Pennsylvania, December 30, 1969 Patent on real-time display of DCR input (?). Shows that final (throughput) error rate is 0%, if user can make corrections Real-time display makes DCR device self-teaching/adapting to train user immediate



feedback [FreemanH61a] Freeman, H. "On the encoding of arbitrary geometric configurations", IRE Trans. Electronic Computers, Vol EC-10, 1961, pp 260-268 Segment matching, for recognition of figures by segments [FreemanH70a] Freeman, H. "Boundary encoding and processing", in Picture Processing and Psychopictorics, B.S. Lipkin and A. Rosenfeld, editors, New York, Academic Press, 1970, pp 241-266 Cited in Pavlidis80, Shu89 [FreibergerH63a] Freiberger, H. and Murphy, E.F. "Reading devices for the blind: an overview", Human Factors in Technology, McGraw-Hill, 1963, pp. 299-314, earlier version in IRE Trans. HFE-2, March 1961, pp. 8-19 Cited in Rosenfeld as translating characters to sounds, but not character recognition. Optical probe: photoelectric cell an buzzer for blind to determine altitude of sun 1938. Reinvented multiple times by persons not aware of prior work. optophone, 1918 optical "recognizer" emitting tones for patterns of light and dark. [FreibergerH66a] Freiberger, H. and Murphy, E.F. "Reading machines for the blind", Science, No 152, April 29, 1966, pp. 679-680 Cited in Rosenfeld as translating characters to sounds, but not character recognition. Reading machines/assistive technology for the blind. Early reading machine for digits, prototype used remote server over telephone connection. Mauch Laboratories reading machine: compare to later Kurzweil reading machine. [FriedmanSR68] Friedman, Symour R; Campbell, Douglas A; and Fehskens, Leonard G. "MOSAIC - The Improved Editing of Scientific Text by Handdrawn Commands and Data: A Technique for RAND Tablet and CRT Display", Technical Report ESD-TR-68-422 Inforonics Inc, Cambridge MA, 14 Oct 1968 (abstract only) Nugent67 same title? Mentions cursive recognition, and use of both command input of a gesture and location of gesture, combining gesture and direct manipulation. Text editing of handwriting like van Raamsdonk? Gesture are select character (downward stroke), change word (wavey horizontal), change line (reverse pigtail), transpose (includes selection), insert (caret), capitalize / uncapitalize (letter C), carriage-return, space, backspace, tab (same as PenWindows gestures?). Two stroke (tap + stroke in direction) for punctuation comma/period,/etc. Compound gestures (double flick, etc.) [FrishkopfLS61a] Frishkopf, L.S. and Harmon, L.D. "Machine Reading of Cursive Script", Information Theory (symposium proceedings, London 1960). C. Cherry, Editor, Butterworth, London, 1961, pp 300-316 Cited in Goodale83, Sayre73 [FrishkopfLS64a] Frishkopf, L.S. "Automatic Recognition of Handwriting", United States Patent 3,133,266, assigned to Bell Telephone Laboratories, New York, New York, 1964 Early script DCR recognition patent Script recognition on word basis, not letters, to get "context" X-ordinate is "general direction of writing" for script Cites disregard of position and size as a distinct advantage [FunkHL70a] Funk, Howard L. and Kamble, Stanley F. "Handwritten Character Recognition Apparatus", United States Patent 3,500,323, March 10, 1970, assigned to IBM Corporation, Armonk, New York. DCR with a mechanical pantograph tablet Six-zone/sector character recognition Cites real-time display for immediate feedback and correction Citable in the Xerox/Palm patent dispute [Gaffney65] Gaffney, J.E. and Kusnick, A.A. "Character Identification Device", United States Patent 3,199,078, assigned to IBM Corporation, New York, New York. Like BLRT chain codes without regard to position (!) for handwriting recognition Compare to SRI pen with BLRT chain codes Number of strokes in different end-to-end slant orientations: used modified forms of letters O, X, and Y "point touch" to indicate end-of-character for character segmentation (!) [GallensonL67a] Gallenson, L "A graphic tablet display console for use under time-sharing", Proc. AFIPS '67 (Fall FJCC), November 14-16, 1967, pp. 689-695 Graphic tablet display (GTD) console user-interface with digitizer integrated into display. Higher data sampling rates (200 points/second) when stylus in contact with display for inking. Filtering of coordinate data, resampling for minimum distance between points. [GarnerWR54a] Garner, W.R. "Context Effects and the Validity of Loudness Scales", Journal of Experimental Psychology, Vol 48 No 3, 1954, pp 218-224 Background on context showing median and mean (goodness and reaction time) are same for judgments [Genchi68] Genchi, Hiroshi, Mori, Ken-Ichi, Watanabe, Sadakazu, and Katsuragi, Sumio "Recognition of Handwritten Numerical Characters for Automatic Letter Sorting", Proceedings of the IEEE, Vol 56 No 8, August 1968, pp 1292-1301 Handwritten numeral recognition for postal codes: features are zones after height and width normalization (after segmentation): 95% correct per character, 0.1% wrong across three digits (similar to human) Gives frequency of writing variants / base forms / prototypes with and without model characters shown. Linethinning, normalization of size: rejects touching characters Handwriting OCR: features are segment types, similar to chain codes, for scanned images



[GilmoreJT58a] Gilmore, John T. and Peterson, Holmer P. "A functional description of the TX-0 computer", M.I.T. / Lincoln Laboratory Report 6M-4789-1, October 3, 1958. TK7855.M41.L7455 The TX-0 computer was used for early stylus input, including handwriting recognition, before the GRAIL project [GlatteH59] Glatte, H. "Shorthand Systems of the World", The Wisdom Library, Division of Philosophical Library New York, 1959 (partial copy) Gabelsberger shorthand: unistroke alphabet [GoldB59a] Gold, B. "Machine recognition of hand-sent Morse code", IRE Trans. on Information Theory, Vol 5 No 1, March 1959 (abstract only) Recognizer for human-sent Morse code, 95% accuracy (defined as decipherable by a human reader) without the use of a word dictionary or other context heuristics. Compare with ARRL project mid 1960-s using diode memory? [GoldbergE31a] Goldberg, Emanuel "Statistical Machine", United States Patent 1,838,389, December 29, 1931 Very early OCR character recognition patent: Goldberg patent later mentioned as predecessor to Memex by Vannevar Bush, as it included filing and indexing of records. [GoldbergHE1914] Goldberg, Hyman Eli "Controller", United States Patent 1,117,184, November 17, 1914 Cited in Tappert's patent list of 1986 on on-line/dynamic: handwriting recognition Very early patent on handwriting recognition: conductive ink written on paper dots triggers adding certain weighted values for the dots Very early patent on handwriting recognition: happens to show constrained writing of numerals (around dot patterns), segmentation into what looks like chain codes See other patents by Goldberg [GoldbergHE1915] Goldberg, Hyman Eli "Controller", United States Patent 1,116,663, December 28, 1915 Very early patent on handwriting recognition: writing on 7-segment sensor for Roman letters and Arabic numerals (alphanumerics) See other patents by Goldberg [GolshanN70a] Golshan, N. and Hsu, C.C. "A Recognition Algorithm for Handprinted Arabic Numerals", IEEE Trans. Systems Science and Cybernetics, July 1970, pp 246-250 Zone-based handwriting recognition: Features are like seven-segment LCD display for numerics: 1970 [GouldME36a] Gould, M.E. "Identifying Means", United States Patent 2,231,186, February 11, 1941 Zone/template-based optical object recognition: sensors/lamps in 5x10 grid are wired connected in particular patterns. When pattern of cells has input, circuit has enough power flowing to activate a relay. [Graham70x] Graham, D.N. "An Adaptive Character Recognition System", Sylvania Applied Research Laboratory, Research Note Number 718 Simplified writing style for Morse code training used as constrained writing: Reference to Morse Code writing style from American Radio Relay League? Cited in Brown 70 reference. [Gray1888a] Gray, Elisha "Art of Telegraphy", United States Patent 386,814, July 31, 1888 Telautograph: very early precursor to electronic stylus and digitizing tablet. Elisha Gray also in patent dispute for priority with Alexander Graham Bell for telephone. [Gray1888b] Gray, Elisha "Telautograph", United States Patent 386,815, July 31, 1888 Telautograph: very early precursor to electronic stylus and digitizing tablet. [Gray1891a] Gray, Elisha "Telautograph", United States Patent 461,470, October 20, 1891 Telautograph: very early precursor to electronic stylus and digitizing tablet. [Gray1891b] Gray, Elisha "Art Of and Apparatus for Telautographic Communication", United States Patent 461,472, October 20, 1891 Telautograph: very early precursor to electronic stylus and digitizing tablet. [Gray1893a] Gray, Elisha "Telautograph", United States Patent 491,347, February 7, 1893 Telautograph: very early precursor to electronic stylus and digitizing tablet [Gray1893b] Gray, Elisha "Telautograph", United States Patent 494,562, April 4, 1893 Telautograph: very early precursor to electronic stylus and digitizing tablet [GreaniasE57a] Greanias, E.; Hoppel, C.J.; Kloomok, M.; Osborne, J.S. "Design of Logic for Recognition of Printed Characters by Simulation", IBM Journal of Research and Development, Vol 1 No 1, 1957, pp 8 ff. OCR handwriting recognition of hand-printed digits and characters; simulated input using punch-card input of bitmaps of images; "proportional parts" using vertical (not horizontal) scanned input. [GreaniasE63a] Greanias, E., Meagher, P., Norman, R. and Essigner, P. "The recognition of handwritten numerals by contour analysis", IBM Journal of Research and Development, Vol 7 No 1, January 1963, pp 14-21 Cited in O'Callaghan70, Groner66a [Griffin67] Griffin, Eugene H. and Gunn, James W. "End of Character Detector", United States Patent 3,305,832, assigned to Sperry Rand Corporation, New York, New York, February 21, 1967 Cited in Tappert88b: on character segmentation on cursive script segmentation (Does OCR character segmentation) OCR scanning character segmentation (?) [GronerGF66a] Groner, G.F. "Real-Time Recognition of Handprinted Text", Memorandum RM-5016-ARPA, The Rand Corporation, October 1966



Handwriting recognition on RAND tablet: curvature feature, corner detection, endpoints, height, width, center, aspect ratio, position relative to writing baseline (on screen) Segmentation in handwriting by X overlap, centers, spatial separation Performance: goes into details of 20 minutes user practice, information training: 82% to 92%, 81% to 93%, 90% to 96%, depending on user training/familiarity Shows actual confusion pairs: 7 and >, G and C, 5 and S, + and T, 0 and ) (???), 2 and Z, 1 and I, (-1CL, [-1CL, etc. User-interface for handwriting recognition/editing of FORTRAN coding forms Quick writing more distorted than usual training samples 53 symbols. Recognition tablet at 4msecs/point or 250 Hz. Multiple stroke characters were not order dependent, segmentation by relative location [GronerGF66b] Groner, G.F. "Real-Time Recognition of Hand-printed Text", AFIPS Fall Joint Computer Conference #29, 1966, Spartan Books, Washington, D.C., pp 591-601 Mentions that subjective results of DCR are more important than statistics % correct [GronerGF67] Groner, G.F., Heafner, J.F., and Robinson, T.W. "On-line computer classification of handprinted Chinese characters as a translation aid", RAND Corporation publication P-3568, April 1967. IEEE Trans. Electronic Computers, Vol 16, December 1966/1967(?), pp 856-860 Tappert's bibliography [GronerGF68a] Groner, G.F. "Real-Time Recognition of Handprinted Symbols", in Pattern Recognition, proceedings of IEEE Workshop on Pattern Recognition, L.N. Kanal, editor, Thompson Press, Washington, D.C., 1968, pp 103108 Description of GRAIL system (which see), user interface for handwriting recognition text input, erase gesture, special symbols [GronerGF68b] Groner, Gabriel F. "Real-Time Recognition of Handprinted Symbols: Program Documentation", RAND Corporation Research Memorandum RM-5550-ARPA, 1968 IBM/360 assembly language program for online graphical-input character recognition using the RAND tablet Recognizes handprinted letters, numbers, punctuation marks, and geometric figures: separates/segments characters written in rapid succession and close proximity [GroverGF66] Grover, G.F. "Real-Time Recognition of Hand-Printed Text", Proceedings of the Fall Joint Computer Conference, 1966, p 591 Cited in Bernstein70 on user-interface [Handel31] Handel, P.W. "Statistical Machine", United States Patent 1,915,993, June 27, 1933 OCR Optical Character Recognition system using rotating disk with stenciled patterns for characters [Hansell39] Hansell, C.W. "Multiplex Facsimile Printer System", United States Patent 2,143,875, assigned to Radio Corporation of America, Delaware, January 17, 1939 Facsimile system which read characters printed in one line along a roll of paper tape or other material. Second(?) earliest U.S. Patent on dynamic/on-line handwriting character recognition: refers to low error rate, both for wrong character and unrecognized / mis-recognized character. Cited in Irland64: on a handwriting terminal. [HarmonLD62a] Harmon, L.D. "Handwriting reader recognizes whole words", Electronics, Vol 35, August 1962, pp 29-31 Tappert's bibliography on script handwriting recognition: script/dictionary handwriting recongntion [HarmonLD63a] Harmon, L.D. "Method and Apparatus for Reading Cursive Script", United States Patent 3,111,646, November 19, 1963, assigned to Bell Telephone Laboratories, New York, New York. Refers to same subject/inter-subject variation in writing/segmentation Patent on script recognition segmentation Groups characters first by certain characteristics, such as vertical extent, then recognizes by other features within the group [HarmonLD64a] Harmon, L.D. "Automatic Reading of Cursive Script", United States Patent 3,127,588, March 31, 1964, assigned to Bell Telephone Laboratories, New York, New York. Word-at-a-time script recognition Script handwriting recognition in conjunction with a one-dimensional conductive-wire digitizing tablet Patent on script recognition segmentation Zone-based recognition of script using one-dimension horizontal digitizer Gives purely electro-mechanical implementation of handwriting/script input in the patent [HarmonLD64b] Harmon, L.D. "Automatic reading of printing and script", presented to the American Association for the Advancement of Science, 29 December, 1964 Cited in Sayre73 [HarmonLD64c] Harmon, L.D. "Automatic Character Recognizer", United States Patent 3,050,711, August 21, 1962, assigned to Bell Telephone Laboratories, New York, New York. Recognized characters by spiral scan, and noting the encounters with elements of the character: gives purely electro-mechanical implementation in the patent [HarmonLD65a] Harmon, L.D. and Sitar, E.J. "Method and apparatus for correcting errors in mutilated text", United States Patent 3,188,609, June, 1965 Cited in Suen79 Context correction in character recognition, using combined diagram probability with the



two characters before and after character in question, including when a charactier is dropped/omitted (no recognition result) but place of a character is known. [HarperTP66a] Harper, T.P. "Electronic Handwriting Detection and Display Apparatus", United States Patent 3,271,515, September 6, 1996 Early light-gun/light-pen device. [Haverty68] Haverty, J.P. "GRAIL/GPSS: Graphic On-Line Modeling", RAND Corporation publication P-3838, 1968. Presented at an IBM Seminar on Operations Research in the Aerospace Industry: Models in Planning and Control, Newport Beach, April 1968 Describes GUI for logical flow diagram / flowchart input, to avoid delays of having input keypunched [HayesHV32] Hayes, H.V. "Radiant Energy Signalling", United States Patent 1,954,204, April 10, 1934 Very early patent on use of visible light wave transmission, which could be more directional (beam) than radio waves. Cited in Hieronymus patent on unknown "Psi-wave" radiation. [HeartFE70a] Heart, F.E.; Kahn, R.E.; Ornstien, S.M.; Crowther, W.R. and Walden, D.C. "The interface message processor for the ARPA computer network", AFIPS '70 (Spring) Proceedings of the May 5-7, 1970, spring joint computer conference SJCC, pp. 551..567 Arpanet IMP packet-switching router. Update routing tables every half second. Emphasis on support software: test harness for simulating host network traffic (on multiple IMPs), debugger built in on all operational units (not separate). [HelsonH65a] Helson, H. "Adaptation Level Theory, an Experimental and Systematic Approach to Behavior", Harper and Row, New York, 1964 User's perception changes to make data look "reasonable": use of context in human perception of images, such as handwriting recognition. [HieronymusTG49] Hieronymus, Thomas G. "Detection of Emanations from Materials and Measurement of the Volumes Thereof", United States Patent 2,482,773, September 27, 1949 Circuit for detecting unknown types of radiation (eloptic energy) using unknown physics to determine compositions of material from valence-electron characteristics at room temperature. Compare with Zimmerman 1995, Matias 2012, capacitive sensing of material. Cited in Analog Science article in 1960's by John W. Campbell. [HilditchCJ69a] Hilditch, C.J. "Linear skeletons from square cupboards", Machine Intelligence, Vol 4, 1969, pp 403-420 Fast algorithm for thinning of lines, applications include trails in bubble chamber photographs, chromosome images, etc. [HladyAM69a] Hlady, A.m. "A touch sensitive X-Y position encoder for computer input", Proc. Fall Joint Computer Convergence, 1969, pp. 545-551 Surface acoustic wave transparent touchscreen, describes construction of transducers for radiating acoustic waves [HochbergJ53a] Hochberg, J. and McAlister, E. "A quantitative approach to figural 'goodness'", Journal of Experimental Psychology, Vol 46, 1953, pp 361-364 Cited in IchikawaS84 [Hornbuckle67] Hornbuckle, G.D. "The Computer Graphics User / Machine Interface", IEEE Trans. Human Factors in Electronics, Vol HFE-8 No 1, March, 1967, pp 17-22 Cited in FoleyJD84 [HoshinoT69a] Hoshino, T. and Kiji, K. "Computer-Aided Design for a Reader of Hand-Printed Characters", Proceedings IJCAI 1969, AFIPS/ACM International Conference on Artificial Intelligence, pp 153-160, May 1969 Dynamic/on-line handwriting recognition using a light pen for input, using bitmaps (?) of images and not directional stroke information. Uses confusion matrix to measure recognition performance, but states that confusion matrix gives no aid in finding the deficiencies in sequential logic (decision tree?) -- contrast with feature matrix and classification boundaries in Pencept/Shillman? Sequential logic designed by "trial and error", sequential logic evaluates features extracted separately: features appear to be chains of stroke segments. [Hough62] Hough, P.V.C. "Method and Means for Recognizing Complex Patterns", United States Patent 3,069,654, December 18, 1962 straight-line detection in images: division of television image into framelets such that straight-line segments can be extracted and recognized. Mentions recognition of could-chamber particle tracks. Secondary mention of handwriting analysis recognition and map (sketch) recognition. Cited in Li89 [Hu62] Hu, M.K. "Visual Pattern Recognition by Moment Invariants", IRE Trans. Information Theory, Vol IT-8, February 1962, pp 179-187 Cited in Horowitz89 [Hughes68] Hughes, G.F. "On the mean accuracy of statistical pattern recognizers", IEEE Trans. Information Theory, Vol IT-14, 1968, pp 55-63



Martin,GL90 on the accuracy of neural nets [HuntWA37] Hunt, W.A. and Volkmann, J. "The Anchoring of an Affective Scale", American Journal of Psychology, Vol 49, 1937, pp 88-92 Recognition variability in humans affected by grouping of stimuli: As stimuli are varied as a group up or down a scale, human recognition boundaries shift to match [HuntWA38] Hunt, W.A. and Flannery, J. "Variability in the Affective Judgement", American Journal of Psychology, Vol 51, 1938, pp 507-513 Variability in the affective judgment (vs generative variability). Recognition variability in humans increases with more stimuli (noise?) due to decrease in separation/differences among samples Functional attributes experiments: response time in humans decreases parallel to "goodness" judgement (decrease in affective judgement variability) [HuntWA41] Hunt, W.A. "Anchoring Effects in Judgement", American Journal of Psychology, Vol 54, 1941, pp 395-403 Recognition variability in humans affected by grouping of stimuli. Psychophysics of human recognition variability [Irland64] Irland, E.A. and Morrison, C.G. "Real Time Reader for Hand-written Alpha Numeric Characters", United States Patent 3,142,039, assigned to Bell Telephone Laboratories, New York, New York, July 21, 1964 Extension of DimondTL63a patent for five-zone/sector recognition? to letters: 7-segment crossings / six zones. Says "D" normally written clockwise, "O" counter-clockwise Constrained handwriting to write around two dots one above the other: no punctuation or lower-case recognition, but full alphanumerics. [Ivaschentzo69] Ivaschentzo/Ivaschenko et al "Inductive Transducers for Graphical Input Devices", Soviet Journal of Instrumentation and Control, August 1969, pp 22-25 Early tablet digitizer technology survey reference (in Russian) [JacksonJ1894a] Jackson, John "The Theory and Practice of Handwriting", William Beverley Harison press, 1894 Early work on handwriting styles (cursive). Mentions effects of writing desks -- compare with WardBJ thesis? [JohnsonCI68] Johnson, C.I. "Interactive Graphics in Data Processing: Principles of interactive systems", IBM Systems Journal, Vol 7 No 3/4, 1968, pp 147-173 Mentions Sketchpad and light pens, primarily about graphics output software [JohnsonEA65a] Johnson, E.A. "Touch Display - a Novel Input/Output Device for Computers", Electronics Letters, Oct. 1965, Vol. 1 No. 0, pp 219-220 Cross-wire electrostatic capacitive tablet, used in touchscreen. [JohnsonEA67a] Johnson, E.A. "Touch Displays: A Programmed Man-Machine Interface", Proc. Conf. on The Human Operator in Complex Systems. W. T. Singleton et al, Eds., London: Taylor and Francis, Ltd.: pp 171-177 (partial copy) Expansion of 1965 Electronic Letters: de-bouncing of input, undo operation, pictures of touch architecture. [JohnsonEA69a] Johnson, Eric Arthur "Touch Displays", United States Patent 3,482,241, December 2, 1969 Cross-wire electrostatic capacitive tablet, used in touchscreen. Also mentions resistance. Works by mutual capacitance. Capacitance changes tuning of L/C circuit. [JohnsonRB56] Johnson, Reynold B. "Indicia-controlled Record Perforating Machine", United States Patent 2,741,312, April 10, 1956 Zone-based handwriting recognition terminal, using two guide dots, 10 digits. Illustrations show 80-column card input. Processing is electromechanical. Cited in Irland64: on a handwriting terminal [JohnsonTE63a] Johnson, Timothy E. "Sketchpad III: A Computer Program for Drawing in Three Dimensions", Proc. SJCC, 1963, pp. 347..343 3D drawing using 2D touchscreen/light pen. User can rotate object to make any section "flat", draw drop line to another point to show where the Z position should be. Then user can draw any 3D line by drawing a 2D line with the light pen. (But what about drawing curved lines in 3D-space?) [JohnsonTE63b] Johnson, Timothy Edward "Sketchpad III: Three dimensional Graphical Communication with a Digital Computer", Master's Thesis, Mech. Eng. Dept., MIT, May 1963 3D drawing using 2D touchscreen/light pen. Drawings are wire frame only: connected lines in 3-space. User can rotate object to make any section "flat", draw drop line to another point to show where the Z position should be. Then user can draw any 3D line by drawing a 2D line with the light pen. States there is a simple solution to drawing arbitrary surfaces, but not in this report. Low precision of light pen relative to drawing dealt with by cylindrical or spherical "snap-to" operation point on nearest line. [Kamentsky64] Kamentsky, L.A. "Character Recognition System", United States Patent 3,123,804, March 3, 1964 Zone-based constrained optical scanning OCR character recognition of handwritten digits on paper: user must write numerals around two vertically-aligned guide dots [KaplowR66a] Kaplow, R.; Brackett, J. and Strong, S. "Man-Machine Communication in on-line mathematical analysis", Proc. FJCC, 1966, pp. 465-477



Early graphical mathematical analysis program: compare with Macsysma/maxima. Graphical display, refers to briefly to graphical curve-tracing input, UI user interface mostly teletypewriter keyboard. [KayAC69a] Kay, Alan C. "The Reactive Engine", Ph.D. Thesis, University of Utah, 1969 (published excerpt only) Alan Kay's Dynabook thesis: published excerpt does not talk about touchscreen, but picture of "experimental display" shows IBM 1130 with tablet stylus or lightpen (hard to make out). [KimYT68a] Kim, Yung Taek and Evans, David C. "Syntax Directed On-line Recognition of Cursive Writing", U. Utah Comp. Sci. Technical Report 4-8, July 1968, ARPA Report AD770241 Cursive handwriting recognition using Sylvania data tablet. Data filtered by minimum required distance between coordinate pairs (to reduce processing overhead), interpolation bit curve fitting for wild points. Stroke segmentation at local Y maxima (up/down stroke segments), recognition by syntactic tree. [KingJH66] King, J.H. jr. and Tunis, C.J. "Some Experiments in Spoken Word Recognition", IBM Journal of Research and Development, January 1966, pp. 65..79 Speech recognition for limited vocabulary of 30 words -- digits and mathematical operations -- using filterbank spectrum analysis. Claimed over 99% correct recognition for both speaker-independent (actually multi-speaker) and single-speaker system, but test data was the same as the training data. [Kinsler62] Kinsler, L.E. and Frey, A.R. "Fundamentals of Acoustics", 2nd Edition, page 503, John Wiley and Sons, Incorporated, New York, 1962 Speed of sound in air changes 0.08% for 5 degrees C (sonic/acoustic digitizer tablets) [KnollAL69a] Knoll, A.L. "Experiments with characteristic loci for recognition of hand-printed characters", IEEE Trans. Computers, Vol 18, pp 366-372, 1969 Choice of samples affects recognition performance results [KoenigW61a] Koenig, Walter Jr. "Telautograph System", United States Patent 3,005,050, October 17, 1961 Early resistive-sheet/film digitizer, with stylus: two sheets (one flexible, one conductive, one uniform resistance), analog signals only, not digital. Speaks of detecting coordinates of the stylus. Two resistive layers, one for each ordinate X/Y. [Krantz61] Krantz, D.L. and Campbell, D.T. "Separating Perceptual and Linguistic Effects in Context Shifts upon Absolute Judgement", Experimental Psychology, Vol 62 No 1, 1961, pp 35-42 Human recognition variability: there are linguistic/semantic effects that change perception as well Human recognition variability: strongly affected by the most recent temporal stimuli [KuceraH67a] Kucera, H. and Francis, W.N. "Computational analysis of present day American English", Brown University Press, Providence, Rhode Island, 1967 Cited in Sinha88: as source of probability and statistics for context post-processing of English text [KuennapasT66a] Kuennapas, Teodor "Visual Perception of Capital Letters: Multidimensional ratio scaling and multidimensional similarity", Scand. Journal of Psychology, Vol 7, 1966 pp 189-196 nine capital letters: I, J, L, F, E, U, O, S, A distinguished in human perceptual recognition by the characteristics of Rectangularity, O/Roundness, and I Vertical Linearity [KuepfmuellerK44a] Kuepfmueller, Kurt and Reche, Karl "Telautograph", United States Patent 2,338,949, January 11, 1944 Telautograph: electronic (CRT) display and resistive tablet, using a stylus, instead of electomechanical input and output or linear resistor arps. [KuhlF63] Kuhl, F. "Classification and Recognition of Hand-Printed Characters", IEEE International Convention Record, Part 4, 1963, pp 75-93 Cited in Groner66a [LaningJH54a] Laning, J.H. Jr. and Zierler, N. "A Program For Translation of Mathematical Equations For Whirlwind I", Engineering Memorandum E-364, Instrumentation Laboratory, M.I.T., January 1954 Whirlwind computer: mathematical expression input and calculation using flexowriter (teletype-like) terminal. 2D mathematical input? Compare with later character-mode calculators with variables. [LeCaineH56a] LeCaine, Hugh "Touch Sensitive Organ", www.hughlecaine.com/en/tsorgan.html, Fetched February 2011 Early touch-input device, organ keyboard detecting velocity of key being struck to affect tone played by electronic music organ, control of attack and volume. See also Young 1984 bio. [LeiferJC67a] Leifer, J.C., Mittelman, Louis Jr.; Sobol, Erwin J. "Free Stylus Position Locating System", United States Patent 3,342,935,September 19, 1967 Capacitive grid digitizer tablet with stylus. Fine and coarse position determination. States no tilt error because only tip of stylus is sensed. [Leitner61] Leitner, R.G. "Telescribing Apparatus", United States Patent 2,975,235, March 14, 1961 Resistive sheet/film digitizer using two sheets: a conductive flexible sheet and a second resistive sheet (or grid of conductors). Output device is mechanical: Telautograph [LesterCHT38] Lester, C.H.T. "Single-stroke writing", Bibliography by T.R. Davis 1938 reference to unistroke writing styles?



[LevenshteinA66] Levenshtein, A. "Binary codes capable of correcting deletions, insertions and reversals", Soviet Physics - Dolkady, Vol 10 No 8, February 1966, pp 707-710 Cited in Bozinovic82 spelling correction context (Russian): error correction for chains of binary codes (bits) that handle insertion errors, deletion errors, and substitution errors, not just substitution errors. [LewisHR68a] Lewis, Harry R. "SHAPESHIFTER: An Interactive Program for Experimenting with Complex-Plane Transformations", Proc. 1968 ACM National Conference, pp. 717-724 Interactive mathematical modelling system. Grafacon digitizer stylus used for menu picking, uses Ledeen recognizer with scrub/rubout/erase gesture, Anderson parse for hand-printed two-dimensional mathematics formulas. [Licklider62] Licklider, J.C.R., and Clark, W.E. "On-Line Man-Computer Communications", Proceedings of the SJCC, 1963, p 329 Cited in Ellis69: DavisMR64 for early handwriting user interface? [LincolnA49] Lincoln, Abraham "Buoying Vessels over Shoals", United States Patent 6,469, May 22, 1849 Attaching additional floatation air chambers to a boat or ship to reduce the draft to get the vessel through shallow water. Only patent issued to a person who was also President of the United States. [LindgrenN65a] Lindgren, N. "Machine recognition of human language: Part III--cursive script recognition", IEEE Spectrum, May 1965, pp 104-116 Cited in Sayre73 [LiuJ69] Liu, J. "Real-Time Chinese Handwriting Recognition Machine", E.E. Thesis, Massachusetts Institute of Technology, Cambridge Massachusetts, 1969 Cited in Goodale83 [LivermoreB63a] Livermore, Benjamin "Device for Hand Printing / Hand printing device", United States Patent 39,296, July 21, 1863 Early (19th century) chorded keyboard, fits into a pocket and can be operated by one hand (PDA?). Includes example of alphabetic character chording similar to 16-segment or 9-segment displays, or to unistroke/graffiti. [LongER60a] Long, E.R. and Henneman, R.H. "An Experimental Analysis of Set: Variables Influencing the Identification of Ambiguous, Visual Stimulus-Objects", American Journal of Psychology, Vol 73, 1960, pp 553-562 Experiment on degraded/low-quality letter-patterns (dot-matrix characters): human recognition varies depending on what samples subjects were trained on Functional attributes experiments: tests of human character recognition using ambiguous stimuli. Compare with Shillman/Kuklinski. [LoomisHH60] Loomis, Herschel H. Jr. "Graphical Manipulation Techniques Using the Lincoln TX-2 Computer", Lincoln Laboratory, MIT, Cambridge Massachusetts Report 516-0017, November 11, 1960 DavisMR64 mentions on handwriting user interface (?) User-interface (CAD) on TX-2 computer, using lightpen for sketching / drawing circuit diagrams: does not include character recognition, but suggests using character recognizer by H.P. Peterson (Page 8) : find a citable a reference to that? [MIT52a] Digital Computer Laboratory, Massachusetts Institute of Technology "Project Whirlwind Training Program Material", M.I.T. 1952 Collection of technical reports on Project Whirlwind: Memorandum M-1624-1 Short Guide to Coding; Report R-90-1 The Binary System of Numbers; Engineering Note E-479 Basic Conversion Program; Memorandum M-1734 WWI Control Switches and Pushbuttons; Memoranum M-1623-1 Programming for In-Out Units (Contains brief description of light-gun / early light-pen) -- supports multiple simultaneous light-guns (multi-touch?); Engineering Note E-516 Comprehensive System of Service Routines; Engineering Notes E-484 Policy on Outside Users of Whirlwind I. [MITRE63a] MITRE Corporation "Semi-Automatic Ground Environment (SAGE)", mitre.org, fetched 2010 SAGE system fully deployed in 1963: Early graphical input/output system using light-gun (invented by Bob Everett), precursor to light-pen and to computer mouse. [MacDonaldJS66] MacDonald, John S. "Experimental Studies of Handwriting Signals", M.I.T. Research Lab of Electronics, March 31, 1996, Technical Report RLE TR-443 Digitizer tablet consisted of electrolytic fluid and current sensors at each side of a rectangular tank. Contains model of handwriting motion implemented as a simulation. Asserts that modelling of neuromotor or physical generation of handwriting motion may assist in understanding invariant features for handwriting recognition: cites Teager and Mermelstein. [MarrillWA63a] Marrill, T., Hartley, A.K., Evans, T.G., Bloom, B.H., Park, D.M., Hart, T.P., and Dorley, D.L. "CYCLOPS-1: A Second Generation Recognition System", AFIPS Conference Proceedings of the 1963 Fall Joint Computer Conference, Vol 24, pp 27-33, 1963 Light pen with handwriting recognition (single characters): Rhyne86 gives this as a reference to an early digitizer tablet, like RAND tablet [MartinWA65a] Martin, William A. "Sintax and Display of Mathematical Expressions", M.I.T. Artificial Intelligence Projet, Memo 85, Memorandum MAC-M-257, July 29, 1965



Automatic 2D layout of mathematical expressions (in Lisp). Lightpen (touchscreen) used to edit expressions, data structure has both positions/layout and parsing of expression, so that use can pick entire sub-expressions. [MartinWA67a] Martin, William Arthur "Symbolic Mathematical Laboratory", Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, Report No MAC-TR-36, January 1967 Contains chapter on parsing two-dimensional mathematical expressions, perhaps with handwriting recognition input Points out that with handwriting user-interface, knowing when an input is complete is unclear (p 235) [MartinWA67b] Martin, W.A. "A Fast Parsing Scheme for Hand-Printed Mathematical Expressions", Massachusetts Institute of Technology, Artificial Intelligence Project Memo No 145, Memorandum MAC-M-360, October 19, 1967 User-interface for parsing mathematics: untested [Masterson62] Masterson, J.L. and Hirsch, R.S. "Machine recognition of handwritten constrained Arabic numerals", IRE Trans. Human Factors and Electronics, Vol 3, pp 62-65, September 1962 Cited in Blatt88: 99.79% character recognition rate [McElwain62] McElwain, C.K. and Evans, M.E. "The Degarbler -- A Program for Correcting Machine-Read Morse Code", Information and Control, Vol 5 No 4, December 1962, pp 368-384 Cited in Peterson80: spelling dictionary Early spelling corrector for context, tailored for typical missrecognition errors in machine-read Morse Code [MermelsteinP64a] Mermelstein, P. and Eden, M. "A System for Automatic Recognition of Handwritten Words", AFIPS Conference Proceedings of the 1964 Fall Joint Computer Conference, Spartan Books, Vol 26, pp 333-342, 1964 Cursive handwriting recogntion, character segments at points of low velocity, classify segment as type of curve segment, recognize if can be generated from model for character writing styles. Shows photograph of electromechanical telautograph as type of tablet. Cited in Rhyne86 as early reference to a RAND-type tablet [MermelsteinP64b] Mermelstein, Paul and Eden, Murray "Experiments on computer recognition of connected handwritten words", Information and Control, Vol 7, 1964, pp 255-270 Simulated (not actual) system for online handwriting recognition based on simple model of writing motion, segementation, approximate matching, and some syntax/context constraints. No actual handwriting done? Cited in Sayre73 [MerrymanCT70a] Merryman, Coleman T. and Restle, Frank "Perceptual Displacement of a Test Mark Toward the Larger of Two Visual Objects", Journal of Experimental Psychology, 1970, Vol 84 No 2, pp 311-318 Study of visual context affecting human perception/recognition of marks. [MikesG70] Mikes, George "The Land of the Rising Yen: Japan", Andre Deutsch, Publisher, Great Britain, 1970, Library of Congress Number 70-118214 Survey article of the quirks of Japanese Syllabic characters and language [MillerGA51a] Miller, G.A., Heise, G.A. and Lichten, W. "The intelligibility of speech as a function of the context of the test materials", Journal of Experimental Psychology, Vol 41, 1951, pp 329-335 Cited in Long60: Human recognition/intelligibility of speech depends on context: greater intelligibility if limited set of test words, used in sentences, words repeated. [MillerGA56a] Miller, G.A. "The Magical Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information", The Psychological Review, Vol 63 No 2, March 1956, pp 81-97 Wonderful early article on human perception: distinctive features, context, limits of absolute judgements, combined features, multiple dimensions of features [MillerGM69] Miller, G.M. "On-line recognition of hand-generated symbols", in Proceedings of Fall Joint Computer Conference, AFIPS, Vol 32, November 1969, pp 591-601 Freeman code segments for handwriting recognition, 1969 Parsing strokes into segments, multiple-stroke characters: single-stroke characters a one-stroke character can also be a component of a multiple-stroke character Stroke segmentation using enclosing rectangle, overlap User interface for two-dimensional form [MinnemanMJ66] Minneman, Milton J. "Handwritten Character Recognition Employing Topology, Cross Correlation, and Decision Theory", IEE Trans. on Systems Science and Cybernetics, Vol SSC-2 No 2, December 1966, pp 8696 Handprinting recognition off-line, OCR, claims 98.5% recognition accuracy, test data was writing samples of single-stroke digits [MoodeyHC42a] Moodey, Hannah C. "Telautograph System", United States Patent 2,269,599, January 13, 1942 resistive-sheet/film digitizer for a Telautograph system, using AC voltages and sensor stylus [MoriKI70] Mori, Ken-Ichi; Genchi, Hiroshi; Watanabe, Sadakazu; and Katsuragi, Sumio "Microprogram Controlled Pattern Processing in a Handwritten Mail Reader-Sorter", Pattern Recognition, Pergamon Press, 1970, Vol 2 pp 175-185 off-line handwriting recognition, digits only for postal/zip codes in Japan



[MortonGA47] Morton, George A. and Flory, Leslie E. "Reading Aid for the Blind", United States Patent 2,420,716, May 20, 1947 Stylus pick-up unit with optical sensor, which translated degree of dark or light on the paper into an audible tone: sensor could have multiple elements [MunsonJH68a] Munson, J.H. "Experiments in the recognition of hand-printed text: Part I---Character recognition", Proceedings of Fall Joint Computer Conference, Vol 33, Thompson Books, Washington, D.C., December 1968, pp 1125-1138 Samples of Fortran characters on coding sheets, with extra constraints. Forms data-entry [MunsonJH68b] Munson, John H. "The Recognition of Hand-printed Text", in Pattern Recognition, proceedings of IEEE Workshop on Pattern Recognition, L.N. Kanal, editor, Thompson Press, Washington, D.C., 1968, pp 115-140 Cites handwriting recognition back to 1955. Handwriting recognition in context, FORTRAN coding sheets Features are concavities, contours, connected regions, spurs Says Clemens65 doesn't work [MyersTE60a] Myers, T.E. "Variable Resistance material", United States Patent 2,951,817, September 6, 1960 Conductive rubber: polyvinyl chloride flexible/pliable material with embedded metal conductive particles, becomes conductive under pressure [Nagy68a] Nagy, G. "Classification Algorithms in Pattern Recognition", IEEE Trans. Audio and Electroacoustics, Vol AU-16, No. 2, June 1968, pp 203-212 (partial copy) opy) Comparison of different statistical algorithms, tested on a training set of OCR characters [Nagy68b] Nagy, G. "State of the Art in Pattern Recognition", Proceedings of IEEE, Vol 56 No 5, May 1968, pp 836-862 (hand-printed) data under actual working conditions likely to be much worse than test data [NagyG70a] Nagy, G. and Tuong, N. "Normalization Techniques for Handprinted Numerals", CACM, Vol 13 No 8, August 1970, pp 475-481 Correction for optical distortion in OCR character scanning Rotational invariance loses distinction between 6 and 9 (critique of fully generalized pattern recognition?) [NassimbeneEG61a] Nassimbene, E.G. "Utensil for Writing and Simultaneously Recognizing the Written Symbols", United States Patent 3,182,291, August 25, 1961 DCR recognition with a photo/optical/light-sensing stylus on a tablet(!), position determined by amount of light falling on stylus sensor from illumination around the tablet: optical tablet [NeillJ69] Neill, J. "Numeric script mail sorter", Proceedings of automatic pattern recognition, pp 49-65, May 1969 Cited in Blatt88: 85%-99.7% character recognition rate [NeisserU60a] Neisser, Ulric and Weene, Paul "A Note on Human Recognition of Hand-Printed Characters", Information and Control, Vol 3, pp 191-196, 1960 Human recognition success rate is only 97% Study of human recognition testing using sign-in sheets (unconstrained characters) Gives human results for confusion-pairs in handwriting samples, "S-J" Has example of human difference between writer's intent and reader's perception for definition of correctness Strong recommended for any review of machine performance on handwritten characters [NewmanEB52a] Newman, E.B. and Gerstman, L.J. "A new method for analyzing printed English", Journal of Experimental Psychology, Vol 44, 1952, pp 114-125 Statistical consistency of character in context in several language: some languages have redundant letters (can be removed without affecting amount of information content.) Cited in Suen79 on context statistics [NewmanEB60a] Newman, E.B. and Waugh, N.C. "The redundancy of texts in three languages", Information Control, Vol 3, 1960, pp 141-153 Context statistics (letter pairs) have similar distribution in three languages with discrecte but disimilar alphabets: English, Russian, Samoan. Later alphabet generally correlates with greater redundancy. Cited in Suen79 on context statistics. Early work on statistical methods in language analysis. [NewmanWM68a] Newman, W.M. "A system for interactive graphical programming", 1968 Spring Joint Computer Conference, AFIP Conference Proceedings, Vol 32, Montvale, N.J. pp 47-54 touch/drag (and select/tap?) interactions with a lightpen stylus Cited in Wallace76 for "virtual devices" [NewmanWM68b] Newman, W.M. "A Graphical Technique for Numerical Input", Computer Journal, Vol 11 No 1, May 1968, pp 63-71 Cited in FoleyJD82 Stylus/pen user-interface: "Light Handle" user-interface recognizes clockwise/counterclockwise spirals as "shaft turning" spiral/circular gestures to control numeric value. Compare with later pie menus/marking menus? [NishidaN70a] Nishida, Nobou and Sakaguchi, Mitsuhito "Hologram Graphic Data Tablet", United States Patent 3,658,402, April 25, 1972 Optical tablet with small hologram cells on a plate, each hologram encodes its position, position determined by light coded value in light. Requires mechanical alignment of light source to transmissive plate. [NugentWR66a] Nugent, W.R. and Buckland, L.F. "Improved Text Editing using Hand-Drawn Commands and Data: A Technique for RAND tablet and CRT Display", Inforonics, Inc., First and Second Quarterly Progress



Reports, Cambridge, Massachusetts, August-November 1966 Cited in Bernstein70 on user interface for text editing with handwriting recognition Friedman 68 same title? Mentions cursive recognition, and use of both command input of a gesture and location of gesture, combining gesture and direct manipulation? Text editing of handwriting like van Raamsdonk? [NugentWR67a] Nugent, W.R. and Buckland, L.F. "MOSAIC - The Improved Editing of Scientific Text by Handdrawn Commands and Data: A Technique for RAND Tablet and CRT Display", Third Quarterly Report, Inforonics, Cambridge, Massachusetts, 1967 Same reference as FriedmanSR68? Cited in Coleman69 on gesture / handwriting user interface, text input and editing using gestures (?) in 1967. [OCallaghanJ68] O'Callaghan, J. "Pattern recognition using some principles of the organism - environment interaction", Ph.D. Thesis, A.N.U., Canberra, Australia, September 1968 O'Callaghan70: early handwriting recognizer [OCallaghanJ70] O'Callaghan, J.F. "Problems in on-line character recognition", in "Picture Language Machines", S. Kaneff, editor, Academic Press, New York, 1970 Broad review of thinking on handwriting recognition, 1970, discussion from a conference Contains long Q/A on O'Callaghan's handwriting recognition: segmentation, features, performance time, rotation/tilt: feature space (for binary comparison) is a broad hump, not a fine line. Mentions problem of mal-adaptation to bad data in trainable adaptive recognizer [Orr68] Orr, N.W. and Hopkin, V.D. "The role of the touch display in air traffic control", The Controller, Vol 7, 1968, pp 7-9 Cited in Beringer89 on electronic ink, integrated tablet/display [PattonR69] Patton, Robert "Graphic Data Tablets", The Electronic Engineer, November 1969, pp 50-55 Cites Dr. Michael Pilla, Bell Labs Human Factors, on writing on separate tablet and display as not a problem. [PeckerEA63a] Pecker, Edwin A. Zone-based recognizer for handwritten digits using a single "+" arrangement of four conductive lines. [Peters63] Peters, R.D. "Signature Identification Device", United States Patent 3,113,461, December 10, 1963 Signature verification device using electro-mechanical meters to measure writing time, X and Y position (velocity?) via meters, which might be some other kind of display. Writing pad appears to be a resistive sheet digitizer with a tethered stylus. See also Herbst76 on signature verification using force vs. time [PostmanL45] Postman, Leo and Miller, G.A "Anchoring of Temporal Judgments", American Journal of Psychology, Vol 58, 1945, pp 43-53 Absolute scale and absolute judgment doesn't exist: depends on context of human recognition / perception Cited on Kuklinski's Ph.D. thesis, how an anchor affects perception of graphical context [PoultonEC65a] Poulton, E.C. "Letter differentiation and rate of comprehension in reading", Journal of Applied Psychology, Vol 49, 1965, pp 358-362 Cited in Suen86 as study of inherent (to humans) legibility of different writing styles. Compare with Pencept pair-wise comparison [PrattF42a] Pratt, F. "Secret and Urgent", Blue Ribbon Books, Garden City, New York, 1942 Cited in Fisher75: frequency of letters in English text [Priver67] Priver, Arthur S. and Boehm, Barry W. "Curve Fitting and Editing Via Interactive Graphics", RAND Corporation Research Memorandum P-3742, 1967. Present at the ACM Symposium on Interactive Systems for Experimental Applied Mathematics, Washington, August 1967 RAND Tablet stylus to enter a curve, and try various ways of fitting, editing and displaying the curve, with Groner's character recognition scheme for alphanumeric inputs [RAND61] RAND Corporation "50 Years of Looking Forward", RAND Review, Fall 1998. Available at http://www.rand.org/publications/randreview/issues/rr.fall.98/50.html In 1961, researchers create the RAND Tablet, the first two-dimensional writing surface that allows humans to communicate instantly with computer through characters printed on a tablet. (Note: Stylator may actually have been earlier). JOSS interactive computer system [RankinBK65a] Rankin, B. Kirk III, Sillars, Walter A., and Hsu, Robert W. "On the Pictorial Structure of Chinese Characters", National Bureau of Standards Technical Note 254, January 4, 1965 Syntactic rules for Chinese characters: composed radicals and nodes. Study of what frequent combinations of strokes should be recognized as linguistic units. Grammar implementation similar to Chomsky' phrasestructure grammars (generative model for Chinese characters). [RankinBK67a] Rankin, B. Kirk III, Siegel, Stephanie, McClelland, Ann, and Tan, James L. "A Grammar for Component Combination in Chinese Characters", National Bureau of Standards Technical Note 286, 1967 Syntactic rules for Chinese characters: composed radicals and nodes. Study of what frequent combinations of strokes should be recognized as linguistic units. Grammar implementation similar to Chomsky' phrasestructure grammars (generative model for Chinese characters).



[RankinK70a] Rankin, Kirk and Tan, James L. "Component Combination and Frame-Embedding in Chinese Character Grammars", National Bureau of Standards Technical Note 492, February 1970 Syntactic rules for Chinese characters: composed radicals and nodes. [RenggerRE68a] Rengger, R.E. and Parks, J.R. "A Survey of Handprinting", National Physical Laboratory, Ministry of Technology, Autonomics Division, March 1968 Measure of skew angle from vertical of handprinted/handwritten characters Context: written "I" in middle of sentence usually done without serifs, but if alone, seriffed. Handwriting/handprinting: variability caused by incomplete closure, extra curves and crossovers, misaligned limbs Contains handwriting samples/data collection: 55 writers (hands) not enough to get all likely variability [RhoadsKG70a] Rhoads, Kevin G. "SYSHAK: or, what a crock OS/360 is", SIPB (Student Information Processing Board), Massachusetts Institute of Technology, 1970 Class notes on exploits of security weaknesses in IBM OS/360 Operating system, mostly to put program into supervisor mode. [RippyDE65a] Rippy, D.E. and Humphries, D.E. "MAGIC: A Machine for Automatic Graphics Interface to a Computer", Proc. FJCC 1965, pp. 819-830 Early graphical display subsystem using special-purpose computer and voice-grade communications lines: light pen used to generate locator (cursor) on display, when locator (cursor) at correct position, separate manual switch to select. Refers to locator "servoing" to the light pen using up/down counters. Interruptbased subroutines for interactions. [RobbinsMF70a] Robbins, M.F. and Beyer, J.D. "An Interactive Computer System Using Graphical Flowchart Input", CACM, Vol 13 No 2, February 1970, pp 115-118 User interface for flowchart definition using handwriting: no symbol recognition. Focus is on simulation of circuit of the flowchart (?) diagram and showing simulated output on an oscilloscope. [Roberts62] Roberts, Lawrence G. "Recent Development in Optical Character Recognition at M.I.T.", in "Optical Character Recognition", Spartan Books, 1962 Mentions early Teager electromagnetic grid tablet ("electronic drafting board"), with simple singlecharacter recognition Eden and Halle, only 18 strokes / stroke-types used to construct/write all English characters (??), simulation of persons handwriting by parameter variation: compare with later work on graphonomics Early dictionary-based recognition of handwriting using dictionary of strokes [RobertsLG66a] Roberts, L.G. "The Lincoln Wand", Fall Joint Computer Conference 1966, Spartan Books, Washington D.C., pp 223-228 Ultrasonic/acoustic digitizer, digitizes in three 3-D dimensions, used as pointing device [RomeroA62a] Romero, Adolph "Electric Writing Board", United States Patent 3,019,425, January 30, 1962 Essentially a DC multi-touch/electronic ink switch tablet. Electric writing board supporting erasable electroconductive writing. X/Y grid of conductive points, each point matched to a lightbulb in an electric sign (both have the same grid). Writing stylus/pencil/pen leaves electroconductive material on the ends of the electrodes where they are exposed on the grid, so the writing shows up on the electric sign as the corresponding bulbs turn on. Erase by removing the electroconductive material. Can also write with conductive ink/material on paper, and put the paper face-down on the exposed electrodes. [RomeroA64a] Romero, Adolph "Stylus Controlled Sign with Contact Grid of Parallel Plates", United States Patent 3,128,458, April 7, 1964 Better contact grid for electric writing board of RomeroA62a [RosenfeldA69a] Rosenfeld, Azriel "Picture Processing by Computer", Computing Surveys, Vol 1 No 3, September 1969, pp 147-174 Survey of early techniques for computer processing of pictures by translating to a picture description language / picture language. Image compression includes describing fixed uniform regions, or short-code encoding of most frequent grayscale pixel values, not just approximations. Refers to handwriting recognition, pointing out that it is not strictly OCR, uses dynamic information. [RosenthalAH47a] Rosenthal, Adolph H. "Two-way Television Communication Unit", United States Patent 2,420,198, May 6, 1947 Very early videoconferencing system, using half-silvered mirror and television/video camera to let use look directly at screen for eye contact. Compare to ClearBoard. [RovnerPD69a] Rover, P.D. and Henderson, D.A. Jr. "On the implementation of AMBIT/G: a Graphical Programming Language", Proc. IJCAI 1969: 9-20 Graphical editor for 2D two-dimensional visual programming language AMBIT/G. Recognized handwritten Symbols used to modify program drawing or as system commands (gestures), also light targets (menu/icon). Long lists of light targets are scrolled/TURNPAGE to next portion. Cited by Buxton, 1969 gesture recognition, for using the TX-2 "recognizer". [RuddiesGH66] Ruddies, Guenther H. "Deine Handschrift / Dein Ruin: Mach mehr aus Deiner Schrift", Verlag und Druckhaus Dr. Jenner, Munich (hardcopy book)



"f\ür Deine graphologische Beurteilung bei Bewerbung, beruflichem Aufstieg und Partnerwahl" Describes use of graphological evaluation for employment: describes individual handwriting characteristics: compare to some aspects of graphonomics by Teulings? [SayreKM65a] Sayre, Kenneth M. "Recognition: A Study in the Philosophy of Artificial Intelligence", Notre Dame, Indiana, University of Notre Dame Press, 1965 Cited in Goodale83 [ScriptureEW1900a] Scripture, Edward W. "Studies from the Yale Psychological Library", Yale University, New Haven, Conn. 1900 Contains "Researches on movements used in writing" by Cloyd N. McAllister: Has an electronic (electric) writing tablet, regularly generated sparks from a writing stylus left equally-timed dots on the paper while subject was writing. Cites French study that connected writing is *slower* than hand printing -- because of stylish writing? Notes on baseline of writing. [SearsFW60a] Sears, Francis Weston and Zemansky, Mark W. "College Physics: Mechanics, Head and Sound. Third Edition", Addison-Wesley, 1960 (hardcopy book) Standard physics reference and textbook [Shannon51] Shannon, C. "Prediction and entropy of printed English", Bell System Technical Journal, Vol 30, 1951, pp 50-64 Cited in Suen79 on dictionary statistics Early study on dictionary of frequent pairs in English, applied to compressive encoding of text based on encoding next character based on statistical likeliness. [ShawAC70a] Shaw, Alan C. "Parsing of Graph-Representable Pictures", Journal of the A.C.M., Vol 17 No 3, July 1970, pp 453-481 Parsing of pictures (actually black/white line diagrams: block letters, electrical circuit, etc.) into primitive components (small drawing elements) and grammar for specifying how the elements are connected. One example of a photograph, but image reduced to black/white and illegible. Cited in FoleyJD82 [ShawJC58a] Shaw, J.C., Newell, A., Simon, H.A., and Ellis, T.O. "A Command Structure for Complex Information Processing", RAND Corporation, Santa Monica, California, Report No P-1277; PB-164 088 Also cited as [JCC 13] Proceedings of the Western Joint Computer Conference, May, 1958 Mathematical expression parsing user-interface? [ShepardDH53] Shepard, D. H. "Apparatus for Reading", United States Patent 2,663,758, December 22, 1953 OCR Optical Character Recognition system using mechanical flying-spot scanner with rotary drum [SheridanTB63a] Sheridan, T.B. and Ferrell, W.R. "Remote Manipulative Control with Transmission Delay", IEEE Trans. on Human Factors in Electronics, Sep. 1963, Vol 4 No 1, pp. 25-29 (partial copy) User-interface responsiveness: if less than 45 milliseconds between event and system response, perceived as no delay. If greater than 300 milliseconds, perceived by user as sluggish, resulting in move-and-wait behavior by user. [Simek67] Simek, J.G. and Tunis, C.J. "Hand-printing input device for computer systems", IEEE Spectrum, Vol 4, 1967, pp 72-81 Claims handwriting is faster than typing (??) Reports over 98% accuracy with a specially-trained subject (the author) [Sitar61] Sitar, E.J. "Machine recognition of cursive script: the use of context for error detection and correction", Bell Telephone Laboratories, Murray Hill, New Jersey, Memorandum for file, 12 September 1961 Cited in Sayre73 [SmithLB70] Smith, Lyle B. "A Survey of Interactive Graphical Systems for Mathematics", Computer Surveys, Vol 2 No 4, December 1970, pp 261-297 Very early graph-fitting algorithms, etc. Mentions light pen as input device to some mathematical system. References to early tablets: RAND tablet, "graphic tablet display" (tablet and display integrated) from System Development Corporation 1967, "beam pen" input/output device. One reference to handwriting recognition of characters (Mermelstein 1964) Cited in FoleyJD82 [SmithlineET64a] Smithline "Limited Vocabulary Script Reader", IBM Technical Disclosure Bulletin, Vol 7 No 6, November 1964, pp 473-475 Cited by Crane75 (partial copy) [SpinradRJ65a] Spinrad, R.J. "Machine recognition of hand-printing", Information and Control, Vol 8 No 2, 1965, pp 124-142 Hand-printing recognition rate 93.5% -- not sure how measured. Error charts appear similar to Shillman confusion matrix. Refers to reconstruction of mutilated text in English: spell checker / spelling correction? [StevensME61a] Stevens, M.E. "Automatic Character Recognition - A State-of-the-Art Report", U.S. Department of Commerce, Washington, D.C., NBS Technical Note 112, May 1961 hand-written numerals recognition using vector crossing technique : a standardized single-stroke numeric character style tables of frequency of occurrence of upper-case and lower-case characters Very extensive bibliography of handwriting recognition as of 1961. Cited in Groner66a



[StevensME70a] Stevens, Mary Elizabeth "Information Acquisition, Sensing, and Input: A Selective Literature Review", U.S. Department of Commerce, Washington, D.C., NBS Monograph 113-1, March 1970 Extensive review of handwriting, voice, and optical character recognition as of 1970. Cites numerous references: Ledeen and Wolfburg recognizer, Simek, Data Trend "MIMO" (Man-in/Machine-out) handwriting recognition platen (digitizer) as of 1965, Burger 1964 sketch/gesture recognition for chemical symbols, Goldberg 1931 patent, etc. Several references to automatic handwriting recognition and interactive computer user interfaces on tablets in early 1960s. [StevensSS58a] Stevens, S.S. "Adaptation-level vs. the Relativity of Judgment", American Journal of Psychology, Vol LXXI No 4, December 1958 Classic study on context and adaptation affecting human perception/recognition [StoneL54] Stone, Leo "On the Principal Obscene Word of the English Language -- An Inquiry, with Hypothesis, Regarding its Origin and Persistence", International Journal of Psycho-analysis, 1954, Vol 35, pp 30-56 (abstract only) Scottish in origin. See also Fairman 2007 [StoneRC1889a] Stone, Ross C. "Telegraphic Apparatus", United States Patent 400,141, March 26, 1889 Precursor of electric tablet: conductive wired stylus can be dragged across holes (dashes and dots) in perforated plate at a uniform rate of speed to send Morse code, Morse characters arranged in appropriate keyboard or other configuration for easy use. [StotzR63a] Stotz, R. "Man-Machine Console Facilities for Computer-Aided Design", Proceedings of AFIPS 1963 Spring Joint Computer Conference. Cited in DavisMR64 on early handwriting user-interface? Not clear about recognition or gestures. Early hardware for CAD design: descrption of light pen, display storage scopes graphic display. Light buttons -on screen menus and actions, can be embedded in picture being displayed: compare icon. [SutherlandIE63a] Sutherland, Ivan E. "SKETCHPAD: A Man-Machine Graphical Communication System", Ph. D. thesis, Massachusetts Institute of Technology, January 1963 Shows early light-pen digitizer design; refers to "pseudo-pen location" for editing graphics, locate nearest object. [SutherlandIE63b] Sutherland, I.E. "SKETCHPAD: A Man-Machine Graphical Communication System", Spring Joint Computer Conference 1963, Spartan Books, Baltimore, Maryland, pp 326-ff. Graphical GUI drawing program, hardware uses early lightpen. Sketching system with snap-to-object, graphical constraints, recognition of graphical patterns (e.g. zig-zad, makes pretty). Cited in FoleyJD82 on stylus user-interface [SutherlandIE64b] Sutherland, Ivan E. "Sketchpad Demo (video) 1 of 2", Excerpt from early 1960's TV report, shown at SIGGRAPH '83: Fetched 2010 http://youtube.com/watch?v=USyoT_Ha_bA Generally credited as (almost) earliest direct-manipulation interface, often cites on gesture input [SutherlandIE64c] Sutherland, Ivan E. "Sketchpad Demo (video) 2 of 2", Excerpt from early 1960's TV report, shown at SIGGRAPH '83: Fetched 2010 http://youtube.com/watch?v=BKM3CmRqK2o Generally credited as (almost) earliest direct-manipulation interface, often cites on gesture input [SutherlandIE65a] Sutherland, Ivan E. "The Ultimate Display", Proc. IFIP congress, 1965, pp. 506-608 Essay on displays: not just visual, but haptic and physical output (e.g. force-feedback to a joystick), can simulate real physics, but also unreal physics: e.g. negative mass. Input can be not just keyboard or pointing (lightpen, RAND tablet), but sensing any body part, eye motion, etc. Cites to graphical constraints of Sketchpad, computer animation of Ken Knowlton. See also Alternate Reality Kit (literalism vs. magic) [SutherlandIE66a] Sutherland, I.E. "Computer Inputs and Outputs", Scientific American, September 1966 Overview article of Sutherland's "Sketchpad" system user-interface: describes the "interrupt" for I/O Describes Rand system, caret edit mark gesture to edit handwritten text, graphical editing Refers to sketchinput for prettying up drawings (user interface) Refers Ellis66/Ellis69 user-interface for flowcharts and handwriting recognition (see "further reading" in issue?) [SutherlandIE68a] Sutherland, I.E. "A head-mounted three-dimensional display", Proc. FJCC Fall Joint Computer Conference 1968 pp. 767-764 Binocular head-mounted display for simulating 3D virtual environments. Head position and orientation determined by four ultrasonic transducers, or electromechanical sensor. [SutherlandWR66b] Sutherland, William Robert "On-line Graphical Specification of Computer Procedures", Ph.D. Thesis, M.I.T, January 1966. Also MIT Lincoln Laboratories Technical Report 405, May 1966 Graphical programming languages / visual programming languages [TauschekG29] Tauschek, G. "Reading Machine", United States Patent 2,026,329, December 31, 1935 OCR Optical Character Recognition device using photo electric cell, and rotating drum with stencilled patterns for characters [Taylor54] Taylor "Indicating Apparatus", United States Patent 2,672,605, March 16, 1954 On a handwriting input terminal. Basically an X/Y electromagnetic digitizer with a grid of wires, but the



output runs to ordinary electronics, not a computer. Grid wire encoding is not simple binary, but rather one-bit change in encoding from one wire to the next, to avoid bad signals from intermediate states (Gray coding). Cited in Irland64 [TeitelmanW63a] Teitelman, Warren "New Methods for Real-Time Recognition of Hand-Drawn Characters", Report 1015, Bolt Beranek and Newman, Cambridge, Massachusetts, June 1963. Also MIT Master's Thesis, Dept. of Math., 1962 Cited in Purcell77 [TeitelmanW63b] Teitelman, Warren "ARGUS - a program that recognizes hand-drawn characters in real time", Artificial Intelligence Project, MIT Computation Center Memo AIM-53, 1963 User manual for trainable handwriting recognition software described in "New Methods for Real-Time Recognition of Hand-Drawn Characters". Light-pen digitizer. Refers to construction of decision tree classifier. [TeitelmanW64a] Teitelman, W. "Real-time recognition of hand-drawn characters", AFIPS Proceedings of Fall Joint Computer Conference, Vol 26, Spartan Books, Baltimore, Maryland, pp 559-575, October 1964 Refers to putting constraints on recognition. Machine learning handwriting recognition using hypothesis testing. Criticizes invariants of Stylator system. Mentions recognizing mathematic expressions. [Teixeira68] Teixeira, J.K. and Sallen, R.P. "The Sylvania data tablet: a new approach to graphic data input", Proceedings of Spring Joint Computer Conference, pp 315-323, 1968 Cited in Rhyne86 Reference to early digitizer tablet, like RAND tablet, only transparent [ThayerFW1878] Thayer, Frederick W. "Improvement in Masks", United States Patent 200,358, February 12, 1878 Original patent on baseball catcher's mask, showing heavy wire cage, goat-hair padding. [ThereminLS28a] Theremin, Leo Ssergejewitsch "Method of and Apparatus for the Generation of Sounds", United States Patent 1,661,058, February 28, 1928 Theremin: two-handed capacitive proximity sensors used to control pitch and volume of a musical instrument. Cited as early example of capacitive position sensing for touchscreens and proximity tablets. [ThomasRB67] Thomas, R.B. and Kassler, M. "Character recognition in context", Information Control, Vol 10, pp 43-64, 1967 Cited in DosterW77 [TinkerMA28a] Tinker, M.A. "The relative legibility of the letters, the digits, and of certain mathematical signs", Journal of Generative Psychology, Vol 1, 1928, pp 472-494 (partial copy) Cited in Engel73 for early work on confusion/substitution errors in human recognition [TinkerMA63a] Tinker, M.A. "Legibility of print", Iowa, Iowa State University Press, 1963 Cited in IchikawaS84 : human recognition of characters [Tomita67] Tomita, Shingo, Noguchi, Shoichi, Oizumi, Juro, and Iwamoto, Kangi "Recognition of Handwritten Katakana Characters", Electronics Communications of Japan, Vol 50, 1967, pp 174-183 OCR of handwritten recognition, features are line segments, quantized direction (chain) codes, types of joins/intersections, connection matrices [TownsendLG38a] Townsend, L.G. "Method of and Apparatus for the Indexing and Photo-Transcription of Records", United States Patent 2,121,061, June 21, 1938 Visually similar to Vannevar Bush "As We May Think" : microfilming of paper records to save space, combined with punch cards with serial number identifier for record, documents may be found by punched card, automatically displayed. [TurnerJA70a] Turner, J.A. and Ritchie, G.J. "Linear current division in resistive areas: Its application to computer graphics", Proc. SJCC, 1970, pp. 613-620 Perhaps the grand-daddy of resistive-sheet digitizer papers: Stylus injects a current into a point on a resistive sheet, sheet is grounded at two sides, current to the each side divides proportionally. Use isolation diodes for X and Y pairs of edges, some edge distortion due to point contacts of diodes. States 1% (of full scale) global accuracy, data rate up to 10,000 co-ordinates/second. [UhrL61a] Uhr, L. and Vossler, C. "A pattern-recognition system that generates, evaluates, and adjusts its own operators", Proceedings of World Joint Computer Conference, Vol 19, 1961, pp 555-570 cited in O'Callaghan70, adaptive recognition (neural nets?) [VanRaalteJA68a] Van Raalte, John A. "Reflective liquid crystal television display", Proc. of IEEE, Vol 58 No 12, December 1969, pp. 2145-2149 (abstract only) Early reflective (non-backlit) LCD television display prototype, addressable in real time [VanderGon62] Van der Gon, J.J.D., et al "A handwriting simulator", Physics of Medicine and Biology, Vol 6 No 3, January 1962, pp 407-414 Cited in Ehrich78 on modelling handwriting motion First author name also in print as van der Gon Denier [VanderGonDenier65] Van der Gon Denier, J.J. and Thuring, J. Ph. "The guiding of human writing movements", Biological Cybernetics, Vol 2 No 4, Feb 1965 pp 145-158 First author name also in print as van der Gon Fast handwriting occurs without feedback (and is thus



sloppy?), Shape is determined by muscle timing, not force [Vernon62] Vernon, H.C., and Walsh, Robert R. "Character Recognition Method and Apparatus", United States Patent 3,058,093, October 9, 1962 Zone-based character recognition, scanning intersections with zones [Viterbi67] Viterbi, A.J. "Error bounds for convolutional codes and an asymptotically optimal decoding algorithm", IEEE Trans. Information Theory, Vol IT-13, 1967, pp 260-269 Cited everywhere (up until about 1980) for context correction / context in character recognition [VolkmannFC61a] Volkmann, Frances C. and Engen, Trygg "Three Types of Anchoring Effects in the Absolute Judgment of Hue", Jnl of Exp. Psychology, 1961, vol 61, No 1, pp. 7-17 (partial copy) Anchoring effect: factors that shift the classification / pattern recognition in human subjects. [VosslerCM64a] Vossler, C.M. and Branston, N.M. "The use of context for correcting garbled English text", Proceedings of 19th ACM National Conference, Philadelphia, 1964, pp D2.4-1 to D2.4-13 (abstract only) Just Abstract: see also DosterW77 context/dictionary to aid recognition: dictionary of English words containing probability of occurrence, and letter digram frequencies based on English text: See also Viterbi algorithm [WGBH64a] WGBH (Public Television Station) "Computer Sketchpad (Video)", www.wgbh.org, fetched 2009 Video of Sketchpad from 1964: user interface with gestures and lightpen input. See video archives. [WaalJDD32a] Wall, J.D.D. "Printscript", Bibliography by T.R. Davis 1932 reference to unistroke writing styles? [WaltonHF1917a] Walton, Herman F. "Variable Electric Sign", United States Patent 1,231,821, July 3, 1917 Writing board using array of conductive bumps and a wired conductive stylus, as user writes on tablet, closes relay circuits to turn on lamps in matching array of lamps: can show electronic ink on sign. [WeizenbaumJ67a] Weizenbaum, Joseph "Contextual Understanding by Computers", CACM vol 10 no 8, August 1967, pp. 474-480 Eliza, early natural language understanding (typing text) simulating conversation. [West60] West, G.P. "Method and Apparatus for Sensing Handwritten or Printed Characters", United States Patent 2,964,734, December 13, 1960 Zone-based character recognition input, using electromechanical relays / logical sensing circuits [WeverEG28a] Wever, Ernest Glen and Zener, Karl Edward "The Method of Absolute Judgment in Psychophysics", Psych Rev, Vol 35, 1928, pp 466-493 Differential sensitivity in judgements, rather than absolute: compare to pairwise-comparison method for character recognition of Pencept for handwriting, functional attribute method of Shillman et al [Wikipedia64] www.wikipedia.org "PLATO (computer system)", www.wikipedia.org, fetched 3/22/2010 PLATO (Programmed Logic for Automated Teaching Operations), early educational computer system with custom terminal in 1964: bitmapped plasma graphics display and 16x16 low-resolution infrared touchscreen/touchpad input 1964. Project later did not involve custom hardware. [WilliamsTG69] Williams, Thomas G. "On-Line Parsing of Hand-Printed Mathematical Expressions: Final Report for Phase II", NASA Contractor Report CR-1455, Washington, D.C., December 1969, prepared by System Development Corporation Bernstein70, Martin71 cited this on user interface for mathematical 2-D parsing. User writes in two dimensions, compiler translates to linear form (shown at top of display) an evaluates. Characters are translated into recognized characters in two-dimensional form for editing in real time. Uses bounding rectangles in parsing. Gesture recognition described, handwriting recognition is in a separate publication. "Scrub mode" is a scratch-out gesture. Arrow gesture indicates move/edit. Horizontal line gesture for "open space/move" editing operation. Double horizontal line for close up. Dot (tap) plus second line to move a character. Two perpendicular stroke gesture for select/move group. See also Phase I and Phase IV reports. [Wilner66] Wilner "Dynamic alphanumeric Hand Printing Recognition System", IBM Technical Disclosure Bulletin, Vol 8, No 9, Feb 1966, pp 1205-1207 Cited in Donahey76 zone handprinting/handwriting recognition system [WisemanNE69a] Wiseman, N.E. et al "PIXIE: A New Approach to Graphical Man-Machine Communication", Proc. 1969 CAD Conf, IEE, Southampton 1969, pp 463-ff Early reference to pie menus: Cited in Pier 92 for earliest pop-up pie menus, might be a gesture? [WooPW62a] Woo, Paul W. "Ultrasonic Data converter", United States Patent 3,134,099, May 19, 1964 Acoustic digitizer, using an ultrasonic transducer in a stylus and linear arrays of microphones on the edges of the surface, measures time delay like sonar [WrightG52a] Wright, G.G.N. "The Writing of Arabic Numerals", Scottish Council for Research in Education Series No 33, pp 16-ff, University of London Press, London, 1952 (hardcopy book) Segmentation and variation on the digit 2, similar to chain codes Shows variability in writing styles for 2 and Z, confusion matrix: very interesting in demonstration handwriting variability on historical scale [YarbroughLD69a] Yarbrough, Lynn D. "Letter to the Editor: on using a graphic tablet", CACM, Vol 11 No 5, May



1968, pp. 294 and 377 Four-corner quadrilateral alignment/registration correction for a paper on a digitizing tablet: same calculation later applied for registration of tablet to display on a touchscreen. Example is a single correction for the entire paper, paper may be stretched (or photocopied) or tilted. [ZadehLA65a] Zadeh, L.A. "Fuzzy sets", Information and Control, Vol 8, 1965, pp 338-353 Original paper on fuzzy sets, used for visual recognition/classification.Biswas81 cites on fuzzy sets for recognition. Electronic file includes 2012 PowerPoint presentation on fuzzy set theory. [ZernikeF34] Zernike, Fritz "Beugungstheorie des Schneideverfahrens und seiner Verbesserten Form, der Phasenkontrastmethode", Physica, Vol 1, 1934, pp. 689-704 Rotationally-invariant character recognition using polynomial moments. Zernike polynomials widely cited for optical recognition of certain patterns when viewed through circular apertures e.g. human eye, lens, radar image. Cited in Khotanzad90 Abstract: Auf Grundlage der Abb'eschen Beugungstheorie der optischen Abbildung wird das Aussehen eines Hohlspiegels mit willkürlich verlaufenden kleinen Abweichungen beim Foucaul tschen Schneideverfahren und beim neuen Phasenkontrastverfahren berechnet. Es werden die orthogonalen "Kreisfächenpolynome" gefunden und auf die Beugungserscheinungen beim kreisförmigen Spiegel angewendet. [Zobrak67] Zobrak, M.J. and Sze, T.W. "A method of recognition of hand drawn line patterns", Proceedings of 1st Princeton Conference on Information Sciences and Systems, 1967, pp 240-244 handwriting recognition using eight-direction chain codes / direction sequences: other work mentioned for Chinese handwriting recognition. penhist (Prev page)

Main Page


"You can win all the battles, and still lose the war." -- Samurai saying "Die Zeit ist nur das, was man an der Uhr abliest." -- Albert Einstein, bekannter Physiker und Geigenspieler




Most of these are in my personal collection, either as physical items or electronic files. I can be contacted at [email protected]. Copyright © Tue Jul 14 21:08:43 EDT 2015 . Mirrors of this page can be found at: http://www.ruetersward.com/pens/biblio75.html http://users.rcn.com/rwservices/pens/biblio75.html http://rwservices.no-ip.info:81/pens/biblio75.html (Generally most up-to-date) biblio70 (Prev page)

Main Page


[ANSI72a] ANSI "Draft standard for ANSI74: Character Set for Hand-Printing", Draft Standard, American National Standards Institute, Inc, 1972 Draft for ANSI74 standard handwriting styles, with subset character sets [ANSI74a] ANSI "Character Set for Hand-Printing", American National Standards Institute, Incorporated, Standard X3.45, 1974 ANSI74 (ANSI X3.45) standard handwriting styles, with subset character sets [ARRL73] ARRL "Learning the International Morse Code", published by the American Radio Relay League, Newington, Connecticut, 1973 Shows preferred, rapid writing style for block-printed capital letters and digits, with stroke direction: taught by the US Military also, constrained handwriting for human recognition [AbbsJH71a] Abbs, J.H. and Sussman, H.M. "Neurophysiological Feature Detectors and Speech Perception: A Discussion of Theoretical Implications", Journal of Speech and Hearing Research, Vol 14, 1971, pp 23-36 There are specific feature detection cells in human vision, and that there must be something similar for hearing: in any case, a good recognition system will be modelled on these feature extraction mechanisms. E.g. Vision has specific cells for horizontal and vertical lines. [AhlgrenRC71a] Ahlgren, R.C., Ryan, H.F. and Swonger, C.W. "A character recognition application of an iterative procedure for feature selection", IEEE Trans. Computers, Vol C-20, September 1971, pp 1067-1086 (abstract only) Measure a number of features in a (typed character) dataset, then pick the features for recognition based on pairs of correlations [AhoAV75a] Aho, A.V. and Corasick, M.J. "Efficient String Matching: An Aid to Bibliographic Search", Communications of A.C.M., Vol 18, pp 333-340, 1975 String pattern matching [AlbanoA74] Albano, A. "Representation of digitized contours in terms of conic arcs and straight-line segments", Computer Graphics Image Processing, Vol 3, 1974, pp 23-33 Constrained handwriting styles to make them readable by OCR



[AndersonRA71a] Anderson, R. A. and Sibley, W. L. "Web Programming and Adaptive Heuristics", RAND Report R876-ARPA, September 1971 Web language (directed graph grammar): graphical programming (visual programming) language on a tablet could be transformed into resistor, capacitor, or other components, transformed by heuristics into higher-level logical objects. [AndersonRH72a] Anderson, Robert H. "Programming on a tablet: A proposal for a new notation", Proceedings of the symposium on Two-dimensional man-machine communication, pp 113-123. Los Alamos, New Mexico. Sponsored by SIGPLAN : ACM Special Interest Group on Programming Languages Mentions use of two-dimensional tablet GUI for a programming interface for a graphical editor: actual article discusses a block-oriented programming language, somewhat similar to that of Plamondon from about 1990. Cites GRAIL (flowcharting, text); Anderson, Blackwell and Bernstein (mathematical notation); Datastructures (AMBIT) [Applicon73] Applicon Incorporated "Computerized Graphic Processing System: System User's Manual", Applicon Incorporated, September 1973 Sketch recognition / world's first gesture recognition user-interface in a commercial product: describes Ledeen gesture recognizer? [AshfordRF72a] Ashford, Richard F. Jr "RADIK: An Interactive Graphics and Text Editor", Master's thesis, US Naval Postgraduate School, Moneterey CA, June 1972 XDS 9300 computer with ADAGE AGT/10 graphics computer, light-pen input, basic graphics operations (rotation/translation/scaling of drawings, etc.) [BBN75a] BBN Bolt, Beranek and Newman "Interface Message Processors for the ARPA Computer Network", BBN Report 2988, Quarterly Technical Report 8, 1 October 1974 to 31 December 1974 Deployment of IMP processors for ArpaNet: debugging of single IMPs in simulation works well, debugging in cluster not well. Code checksummed periodically for reliability. Multiple rundudant processors easy to bug, system continues operation even with some wires pulled out, etc. Case-analysis of network surviving partial failures. Compare with high-availability systems? [BaerJA72a] Baer, James A. and Hill, John W. "Optical-to-Tactile Image Conversion for the Blind", Stanford Research Institute Project Report, Projects 1417 and 8647, 30-June-1972 Optacon touch sensor for the blind: small set of vibrating haptic transducers with hand-held scanner. Output to one or two fingers. Intended as accessbility aid for reading of printed material. [BarkanH73a] Barkan, Harold; Barkan, Edward D.; and Swartz, Jerome "Transparent Touch Controlled Interface With Interactively Related Display", United States Patent 3,757,322, September 4, 1973 Transparent digitizer touchscreen using transparent switches on a display. Mentions need to mount closely to display to minimize visual parallax so that combination appears to user as a single device. Note: multitouch -- "child touches the clock face" and a second image concurrently. Small pads for use with stylus for high resolution (contact density). Cites to Johnson 1969. [BatchelorBG74] Batchelor, B.G. "Practical Approach to Pattern Classification", Plenum Press, London/New York, 1974 Chapter 2: Contains examples of boundary theory decision surface diagrams, but doesn't discuss it that way. Refers to human "teacher" for trainable recognition: e.g. perceptual/cognitive ambiguity about what is an "A" vs an "H". Decision surfaces in multi-dimensional feature analysis (pair-wise comparison?). Statistics: similarity, distance, and compactness of decision surface/classification. Implies boundary theory vs category theory. Accidentally shows European variants of hand-written "H" character [BeckF73a] Beck, F. and Stumpe, B. "Two Devices for Operator Interaction in the Central Control of the New CERN Accelerator", CERN report 73-6, Geneva, 24 May 1973 Transparent capacitive touchscreen over CRT display. Copper on glass is "invisible". Approach of finger changes capacitance to ground. See also Stumpe. [BeckerPW72] Becker, Peter W. and Nielsen, K.A. "Pattern recognition using dynamic pictorial information", IEEE Trans. Systems, Man, and Cybernetics, July 1972, pp 434-437 Book review of 1974 reference: use of context, frequency of occurrence of binary words (two characters / character pairs?) in character recognition. [BeckerPW74] Becker, Peter W. "Recognition of Patterns Using the Frequencies of Occurrence of Binary Words", Springer Verlag, New York and Vienna, 2nd edition, 1974 (abstract only) Book edition of doctoral thesis on combining subpatterns that are classified/recognized less reliably to get a better overall recognition performance [BennetJR75a] Bennet, J.R. and McDonald, J.S. "On the measurement of curvature in a quantized environment", IEEE Trans. Computers, Vol C-24, 1975, pp 803-820 [BernsteinMI71a] Bernstein, Morton I. "Computer Input Output of Two-Dimensional Notations", Proceedings of Second Symposium on Symbolic and Algebraic Manipulation, Los Angeles, California, March 23-25, 1971 Review/opinion article on mathematical expression input and output using tablet. Cites to GRAIL, etc.



[BernsteinMI75a] Bernstein, M.I. "Interactive Systems Research: Interim Report for the Period 16 September 1974 to 15 March, 1975", System Development Corporation, Santa Monica, California, Report No SDC-TM-5243/003/00, 15 May, 1975 NTIS citation index Speech understanding system, describes acoustic-phonetic and lexical-mapping processes [BernsteinMI75b] Bernstein, M.I. "Interactive Systems Research: Final Report to the Director, Advanced Research Projects Agency, for the Period 16 September 1974 to 15 September, 1975", System Development Corporation, Santa Monica, California, Report No SDC-TM-5243/004/00, 15 November, 1975 NTIS citation index Speech understanding system, describes acoustic-phonetic and lexical-mapping processes [BerthodM74] Berthod, M. and Maroy, J.P. "Morphological Features and Sequential Information in Real-time", Proceedings of the 2nd International Joint Conference on Pattern Recognition, August 1974, pp 358-363 Refers to Blesser, et al and functional attributes (distinguishing features). Mentions use of pressure (actually, force) and timing (!) in handwriting recognition -- very gesture-like. [BlesserB72a] Blesser, B., and Ozonoff, D. "A Model for the Radiologic Process", Radiology, Vol 103 No 3, pp 515521, June, 1972 Barry's paper on perception in reading medical X-rays. Set error leading to recognition/perception bias: you found what you are looking for, so you do not recognize other, fairly obvious things. Plausible interpretation == grammar of seeing, what is a legitimate perceptual "sentence". [BlesserB73a] Blesser, B., Shillman, R., Cox, C., Kuklinski, T., Ventura, J., and Eden, M. "Character Recognition Based on Phenomenological Attributes", Visible Language, Vol 7 No 3, 1983, pp 209-223 Examples of attributes/features: functional closure, perceptual closure, physical closure [BlesserB73b] Blesser, B., Shillman, R., Kuklinski, T., Cox, C., Eden, M. and Ventra, J. "A Theoretical Approach to Character Recognition Based on Phenomenological Attributes", Proceedings of the 1st International Joint Conference on Pattern Recognition, 1973, pp 33-40 Published version of Shillman's Thesis work: pair-wise distinguishing characteristics (features) of handprinted characters. [BlesserB74a] Blesser, B., Shillman, R., Kuklinski, T., Cox, C., Eden, M., and Ventura, J. "A Theoretical Approach for Character Recognition Based on Phenomenological Attributes", International Journal of Man-Machine Studies, Vol 6, pp 701-714 Character recognition based on ambiguously shaped characters or "difficult cases" of low quality Graphical context rules extracted via psychological experimentation Arguments for boundary-theory pair-wise comparison for recognition Contains chart of Shillman's ambiguously shaped characters [BoumaH71a] Bouma, H. "Visual recognition of isolated lower case letters", Vision Research, Vol 11, 1971, pp 459-474 Cited in Milloy78 for confusion/substitution errors in human recognition for lower-case characters Cited in Bozinovic89 for human recognition by outline of word (as shown in WrightG52) [BremermannHJ71] Bremermann, H.J. "What Mathematics Can and Cannot Do for Pattern Recognition", in "Zeichenerkennung durch biologische und technische Systeme", Proceedings of the 4th Congress of the Deutsche Gesellschaft f\ür Kybernetik, Berlin, pp 31-44. Published by Springer-Verlag, New York, April 6-9, 1971 Fundamental knowledge of the brain would greatly help on pattern recognition by machine Difficulties in pattern recognition are generally combinatorial Most theoretical papers on pattern recognition are quite worthless Problems like handwritten character recognition are not even well-defined Handwritten "A"s are not mathematically defined: just what people agree on Qualitative and formal-language "theorems" do not help solve pattern recognition problems Pattern recognition theorems (perceptron/neural-net) working without quantitative combinatorial analysis Adaptive systems may not converge on optimum performance: stick on local maxima Most papers give no reasons for choice of recognition features Most recognition systems fail on choice of features Features chosen must separate patterns in recognition Humans do not use mathematical features, like string probabilities Gives fundamental reasons why polynomial mathematical features will not work Cybernetic recognition (measure deformation from paradigm) vs fuzzy sets vs perceptrons, etc. Steepest descent, other gradient methods in statistical pattern recognition fail due to local minima Picture grammars (linguistic scene descriptions) useless if elements cannot be identified Creative imagination of researchers better than statistical recognition Examples of nature best design inspiration in recognition Many papers just an exercise in fashionable recognition techniques "mount Bayes is overpopulated" -- critique of statistical pattern recognition Feature extraction plays a role in visual cortex [BridleJS73] Bridle, J.S. "An efficient elastic-template method for detecting given words in running speech", British Acoustical Society Spring Meeting, April 1973, Chelsea College, London, Paper 73SHC3 Cited in Kruskal83 [BurckhardtCW71] Burckhardt, Christof W., Levrat, Bernard, and Perrin, Marie-Christine "Character Recognition with Incremental Rate Encoding", Kybernetik, Vol 8 No 5, 1971, pp 204-206, Germany.



Handwriting recognition using acceleration chain codes as the feature (?) [BurtonRP73a] Burton, Robert Preece "Real-time measurement of multiple three-dimensional positions", PH.D. Thesis, Department of Computer Science, U. Utah, 1973; also Tech. Report UTEC-CSc-73-122, June 1973 3D three-dimensional digitizer for multiple points. Points are light-emitting diodes LEDs. The light-emitting diodes are easily distinguished from one another by flashing them on and off in sequence, under computer control. The position of a light source is measured by scanning the environment. Fast one-dimensional, geometric scans quickly determine planes in which a light source lies. The point of interest is at the intersection of these planes. [BurtonRP74a] Burton, Robert P. and Sutherland, Ivan E. "Twinkle-box: a three-dimensional computer input device", AFIPS Joint Computer Conference, May 6-10 1974, Chicago, pp 513-520 Cites to Lincoln Wand, Lawrence G. Robert, 1963, three-dimensional acoustic/sonic digitizer: this design is an optical digitizer in three dimensions [Caelli75] Caelli, Terrence M. and Finlay, David C. "Pattern discrimination of Gaussian distributions of orientation and line length parameters", Perception and Psychophysics, 1975, Vol 16 No 3, pp 489-493 From Kuklinski file [Caskey72] Caskey, David L. and Coates, C.L. "Machine Recognition of Handprinted Characters", Electronics Research Center, University of Texas at Austin, Technical Report No 126, May 1, 1972 (hardcopy book) Broad (1972) review of problems handwriting recognition must deal with: consistent performance (gross features only are used), etc. Has many pages of samples of the handwriting collection they used [Caskey73] Caskey, D.L. and Coates, C.L. Jr "Machine recognition of handprinted characters", Proceedings of the 1st International Joint Conference on Pattern Recognition, November 1973, pp 41-49 handwriting recognition on for writing the ANSI standard characters: constraints [CataldoJW74] Cataldo, J.W. "Lettering", Davis Publications, Worcester Massachusetts, 1974 (hardcopy book) Many examples of calligraphy, hand-lettering, embellishments on basic hand-written letter forms, including stroke directions, hooks, etc. Handwriting variability. [CencelJA73a] Cencel, J. Arthur "Capacitive Keyboard", United States Patent 3,750,113, July 31, 1973 Capacitive keyboard, keys wired into matrix of rows and columns. Individual keys are read by addressing rows, then columns, and the state read into a shift register where the index in the shift register matches the row/column address of the key. Thus, all keys can be addressed and read independently for N-key / whole keyboard rollover. Compare with Kaplow multi-touch [CerfVG74a] Cerf, Vinton G. and Kahn Robert E. "A Protocol For Packet Network Intercommunication", IEEE Trans on Communications, Vol Com-22 No 5, May 1974, pp. 637..648 Early packet-switching-network paper describing TCP/IP (pre-UDP), including varying packet sizes, end-toend error checking, tolerance of transmission failures. [ChandrasekaranB71] Chandrasekaran, B. "Independence of measurements and the mean recognition accuracy", IEEE Trans. Information Theory, Vol IT-17, 1971, pp 452-456. Corrections in Vol IT-19, 1972 Martin,GL90 on analysis of how well neural nets work [ChangCY73] Chang, C.Y. "Dynamic programming as applied to feature subset selection in pattern recognition systems", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-3, March 1973, pp 166-171 Cited in Fisher75 [ChowC75a] Chow, C. "Effect of Line Width on recognition of V's and Y's", S.B. Thesis, Department of Electrical Engineering and Computer Science, August 1977, Massachusetts Institute of Technology, Cambridge, Massachusetts. Massachusetts Institute of Technology undergrad Thesis on line width and functional attributes [ChristensenC71a] Christensen, Carlos; Wolfberg, Michael S.; and Fischer, Michael J. "A Report on AMBIT/G, Vols 1-4", Applied Data Research Inc. Report CA-7102-2611, February 26, 1971 Final report on project "Machine-Independent Software Programming" AMBIT. Visual programming language, touchscreen graphical editing with gestures, running on TX-2 computer. References to Ledeen recognizer, constraint languages. agd symbolic debugger. See additional materials in files. [ChristensenC71b] Christensen, Carlos "An Introduction to AMBIT/L, A Diagrammatic Language for List Processing, Third Edition", Applied Data Research Inc. Report CA-7102-2211 , February 22, 1971 Implementation of visual programming language AMBIT/G, represented as two-dimensional directed-graph diagrams for both data and rules. [ChristensenC71c] Christensen, Carlos and Wolfberg, Michael S. "AMBIT/G as an Implementation Language", Applied Data Research Inc. Report CA-7103-0211, March 2, 1971 Visual programming language AMBIT/G, requires definition of all data structures/representations before starting programming algorithms. [ClarkRK71] Clark, R.K. "Identification System", United States Patent 3,621,720, November 23, 1971 Cited in Herbst76 signature verification using tip force/pressure vs. time [Context73] Context Corporation "OCR in Word Processing - a guide for the word processing supervisor", 9 Ray



Avenue, Burlington, Massachusetts 01803, 1973 [CooperLN73a] Cooper, Leon N. "A Possible Organization of Animal Memory and Learning", Collective Properties of Physical Systems, Nobel 24 (1973), pp 252-264 Brief talk on Leon Cooper on getting interested in neural networks and memory, after getting Nobel prize re superconductivity. Founder of Nestor handwriting recognition company. [Corballis75] Corballis, M.C. and Roldan, C.E. "Detection of symmetry as a function of angular orientation", Journal of Experimental Psychology: Human Perception and Performance, Vol 1, 1975, pp 221-230 Cited in IchikawaS84 Angle (vertical vs horizontal) perception in humans? [CormackEO74] Cormack, Elizabeth O. and Robert H. "Stimulus configuration and line orientation in the horizontal-vertical illusion", Perception and Psychophysics, Vol 16 No 2, 1984, pp 208-212 Context of adjacent lines in human perception of vertical and horizontal line length: human feature perception (many other references on this topic) [CottonI72] Cotton, I. "Network graphic attention handling", Online 72 International Conference, Brunel University, Uxbridge, England September 1972, pp 465-490 using keyboard, shaft encoder, other devices as virtual replacements for lightpen/tablet/mouse. finger as stylus, Lincoln Wand as 3D digitizer. Cited in Wallace76 for "virtual devices" [CoxC74] Cox, C., Blesser, B., Eden, M. "The Application of Type Font Analysis to Automatic Character Recognition", Proceedings of the 2nd International Joint Conference on Pattern Recognition, Lyngby, Denmark, Aug 13-15, 1974 OCR: Variability is two sources: type font, and scanner/ink effects. Most variability among characters describable in terms of stylistic consistency. Stated aim of pre-processing is to reduce variability recognizer must deal with. Adaptive recognition should analyze for stylistic consistency as a separate variable, and separate variability from recognition. Variability model is NOT same as picture grammar approach. [CraneHD75a] Crane, Hewitt David, and Savoie, Robert Ellis "Handwriting System", United States Patent 3,930,229, December 30, 1975, assigned to Stanford Research Institute, Menlo Park, California. SRI/Crane's accelerometer-pen character recognition: Uses ULDR chain codes, similar to BLRT chain codes [CraneHD75b] Crane, Hewitt David and Savoie, Robert Ellis "Special Pen and System for Handwriting Recognition", United States Patent 3,906,444, September 16, 1975, assigned to Stanford Research Institute, Menlo Park, California. SRI/Crane's accelerometer-pen for signature verification/recognition e [CummingsTF73a] Cummings, T.F. "Variable Keyboard for Terminal Displays", IBM Technical Disclosure Bulletin, July, 1973, pp. 575-576 On-screen/virtual keyboard with labels displayed on monitor, transparent plastic keys over the display. Keyboard functions change with application. (On-screen menus) [DarringerJA75a] Darringer, J.A. et al "Speed Pen", IBM Technical Disclosure Bulletin, Vol 18 No 7, December 1975, pp 2374-2375 Acoustic digitizer: horizontal raised lines across surface, piezo transducers or microphones pick up sound of ballpoint pen's vertical movement over lines while user writing signature. Biometric verification of signature based on Y velocity profile. [DayAM72] Day, A.M., Parks, J.R. and Pobgee, P.J. "On-line written input to computers", in "Machine Perception of Pictures and Patterns", The Institute of Physics, London, England, 1972, pp 233-240. Also cited as Proceedings of the Conference on Machine Perception of Patterns: Tedding England. Claims 2% error rate (but 8.5% reject): feedback to users greatly improves handwriting recognition Handwriting recognition features are coding zones, sequence similar to chain codes, number and length of horizontal/vertical strokes Describes resistive-sheet/film digitizer tablet, one character big: user-interface for handwriting, one-character input [DeQuervainA75a] de Quervain, Alfred and Wild, Peter "Information-bearing Devices and Projection Display Systems Therefor", United States Patent 3,895,866, July 22, 1975 Early LCD projector patent. Static information can be printed on the glass as well. [DertouzosML72a] Dertousoz, M.L. "Graphic Data Tablet", United States Patent 3,705,956, assigned to Computek, Incorporated, Cambridge, Massachusetts, December 12, 1972 Early electromagnetic digitizer tablet patent, using Gray Code grid pattern encoding, using phase of induced voltage [DudaRO72a] Duda, R.O and Hart, P.E. "Use of the Hough transform to detect lines and curves in pictures", Graphics and Image Processing, Vol 15, pp 11-15, 1972 Finding/detecting lines in B/W pictures: example is edges of a machine cabinet, extracts parameters first, lines detected indirectly. Cited in Kahan87 regarding a statistical Bayesian classifier using binary features. [Dyche72] Dyche, James W. "Handwriting Authentication Technique", United States Patent 3,699,517, assigned to Sylvania Electric Products Inc., Cotber 17, 1972 Herbst76 cites this on signature verification using force, velocity, pen lifts, correlationsamong them



Signature verification using velocity and acceleration, using a graphic tablet [Ehrich73] Ehrich, Roger W. "A Contextual Post-Processor for Cursive Script Recognition - Summary", Proc. 1st Int. Joint Conf. Pattern Recognition, Oct. 1973, pp. 169-171 Cited by Marlin Eller, Microsoft Pen Computing group [Ehrich75] Ehrich, Roger W. and Koehler, K.J. "Experiments in the Contextual Recognition of Cursive Script", IEEE Trans. Computers, Vol C-24 No 2, February 1975, pp 182-194 Contextual pre-processor for script using letter length, word context, stroke parsing: best character alternatives given to word-context recognizer (likelihood ratio) Due to scope of problem, one becomes mired in programming 1.3% error when training and test data were identical Word-wise recognition of hand-written script Refers to certain degenerated, trivial, toy recognition problems and their pointlessness Need for low-quality data in training a system Goshtasby88 cites for binary letter pairs (digrams) for spelling context, following by spelling dictionary look-up for what's left [Eichen73] Eichen, Howard R. and Paulsen, Robert H. "Processing of Coordinate Values of Points Entered in a Graph Board or the like", United States Patent 3,720,948, ,assigned to Co-Data Corporation, Fullerton California, March 13, 1973 Describes dividing a tablet surface into a simulated keyboard area, a sketch area, virtual tablets, etc. Relevant to Schumer patent [ElectronicsReview71] Electronics Review "Program Recognizes Handwritten Math", Electronics Review, Vol 44, No 18 (1971) Cited in Sklarew99 patent [Engel73] Engel, G.R., Dougherty, W.G., and Jones, G.B. "Correlation and Letter Recognition", Canadian Journal of Psychology, 1973, Vol 27 No 3, pp 317-326 Mentions response bias in measuring human recognition errors Problems of matching recognition results of tests and predictions based on only one variable: human recognition is a correlation function of multiple variables Points out problems of weighting multiple variables/features [Feng75] Feng, H.Y. and Pavlidis, T. "Decomposition of polygon into simpler components: Feature generation for syntactic pattern recognition", IEEE Trans. Computers, Vol C-24, Jun 1975, pp 636-656 Cited in Agui79 [FernaldOH73a] Fernald, Olaf H. "Optical graphic data tablet", United States Patent 3,761,877, September 25, 1973 Cited in Tappert's patent list of 1986 on on-line/dynamic handwriting recognition Cited in Fox88: on annotation drawing/handwriting Optical digitizing tablet: stylus shines a light on a transparent glass plate, with a scanner underneath [Fisher75] Fisher, Edward G., Riseman, Edward M., and Hanson, Allen R. "Feature Selection Using Non-redundant Thresholded Measures", preliminary manuscript, University of Massachusetts, Amherst, Massachusetts, 1975 Pair-wise discrimination: guidelines for when to work harder on hard-to-separate classification pairs [Fitzgerald75] Fitzgerald et al "Procedure for rectifying a roughly sketched drawing", IBM Technical Disclosure Bulletin, Vol 18, No 7, December 1975, pages 2339-2342 Cited in Fox92 [FitzhughRJ71a] Fitzhugh, Robert J. and Katsuki, David "The touch-sensitive screen as a flexible response device in CAI and behavioral research", Behavioral Research Methods and Instrumentation, Vol 3 No 3, 1971, pp. 159164 Translucent (transparent) touchscreen with rear-projection, using X/Y grid of conductors with physical contact of an X and a Y wire. Mentions multiple touches as something to be detected and rejected. Did not find a transparent and flexible conductive material: polacoat is conductive and transparent, but not flexible. Metallic-coated mylar for resistive/conductive film touchscreen. [FodorJA74] Fodor, J.A., Bever, T.G., and Garrett, M.F. "The Psychology of Language", McGraw-Hill, New York, 1974 Adjustment of perception of stimulus to be what is "reasonable": We accept hypotheses about the identity of individual letters and words only if they yield a sensible interpretation of longer sequences like phrases and sentences. (Context information in recognition) [FoleyD71] Foley, D. "The Probability of Error on the Design Set as a Function of the Sample Size and Feature Size", PhD dissertation, Syracuse University, June 1971 Cited in Meisel: 85 [FoleyJD74] Foley, J.D. and Wallace, V.L. "The Art of Natural Graphic Man-Machine Conversation", Proc. IEEE, Vol 62 No 4, April 1974, pp 462-471 (partial copy) Cited in FoleyJD84 Cited in Wallace76 for "virtual devices" (virtual tablet) [ForneyGE73a] Forney, G.E. Jr. "The Viterbi algorithm", Proceedings of the IEEE, Vol 61, 1973, pp 268-278 Cited in Sinha88, Hull83a Description of Viterbi algorithm, applications to text/character/OCR recognition using context.



[FreedmanMD74a] Freedman, M.D. "Optical character recognition", IEEE Spectrum, Vol 11 No 3, March 1974, pp 44-52 Cited in Nouh84 [FreemanH74a] Freeman, Herbert "Computer processing of line-drawing images", Computing Surveys, Vol 6, 1974, pp 57-97 Long treatment of art on extracting line drawings / line structures (line traces) from images of line drawings, and also on semantic understanding of line structures. Cites to terrain recognition from topological map data. Cited in Kerrick88 [FuKS74] Fu, K.S. "Syntactic Methods in Pattern Recognition", New York: Academic Press, 1974 Cited in Baird84 [FujisakiH71] Fujisaki, H., Nagai, S. and Hidaka, N. "On-line recognition of hand-written numerals", Annual Report of the Engineering Research Institute, Faculty of Engineering, University of Tokyo, Japan, Vol 30, August 1971, pp 103-110 Tappert's bibliography [GoldsteinIP74a] Goldstein, Ira P. "Summary of MYCROFT: A System for Understanding Simple Picture Programs", Artificial Intelligence, Vol 6 No 3, 1975, pp 249-288, also M.I.T. AI report 305, Logo report 10, May 1974 Program to repair distorted line drawings (not images) automatically. Pitch: Compare with later graphical constraints, constraint-based programming, graphical syntactic rules, Waltz 1970's on picture understanding. [GoldsteinIP74b] Goldstein, Ira P. "Understanding Simple Picture Programs", Technical Report 294, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1974 Earlier, more detailed presentation of other Goldstein 74. Drawing program uses Turtle graphics, planner creates program. [Goodwin75] Goodwin, N.C. "Cursor Positioning on an Electronic Display Using Light-pen, Light-gun, or Keyboard for Three Basic Tasks", Human Factors, pp 289-295, Vol 17 No 3, 1975 Study of positioning time with various cursor, pen, keyboard devices [Granlund72] Granlund, G.H. "Fourier preprocessing for hand print character recognition", IEEE Trans. Computers, Vol C-21, pp 195-201, 1972 [Granzotti73] Granzotti, A.N. "Free Hand Drawing Display System Utilizing Light Pen to Write and Erase Information", United States Patent 3,758,717, September 11, 1973, assigned to International Business Machines Corporation, Armonk, New York. User-interface: Hardware patent on digitizer erasing on display with a light pen, makes broad claims on erasure/editing of freehand writing or sketching of electronic ink [Grether73] Grether, C.B. and Stroh, R.W. "Subjective Evaluation of Differential Pulse-Code Modulation Using the Speech 'Goodness' Rating Scale", IEEE Trans. Audio and Electroacoustics, Vol AU-21 No 3, June 1973, pp 179184 "goodness" scale for subjective evaluation of speech quality: relates to cognitive psychology experiments to extract functional attributes Method for subjective evaluation of cognitive/psychophysical perception/recognition, where theory is not adequately developed for quantitative prediction [Gries71] Gries, D. "Compiler Construction for Digital Computers", Wiley & Sons, New York, 1971 A standard reference on compiler /programming language design [GuentherJA73a] Guenther, Joel Arthur "Machine recognition of hand-sent Morse code using the PDP-12 computer", Air Force Institute of Technology Report AD-786 492, December 1973 Master's Thesis at USAF Inst. of Tech. Recognizer for human-sent Morse code, 95% accuracy, biggest problems is detecting the tone in the presents of noise (sampling) [HallRE72] Hall, R.E. and Hulbert, L.N. "Machine Recognition of Symbols", United States Patent 3,676,848, July 11, 1972, assigned to Ferroxcube Corporation, Saugerties, New York. Early patent on zone/region on-line character recognition [Hanson73] Hanson, A.R. and Riseman, E.M. "System Design of an Integrated Pattern Recognition System", Proceedings of the 1st International Conference on Pattern Recognition, Washington, D.C, October 30 - November 1, 1973 (abstract only). COINS Technical Report 73C-5, University of Massachusetts, Amherst, Massachusetts, June 1973 Cited in Fisher75 [HansonAR74a] Hanson, A.R., Riseman, E.M., and Fisher, E. "Context in Word Recognition", COINS Technical Report 74C-6, August 1974 Cited in DosterW77 [HarmonLD72] Harmon, L.D. "Automatic Recognition of Print and Script", Proceedings of the IEEE, Vol 60, 1972, pp 1165-1176 Cited in Litvin82, Nouh84 [Heinz72] Heinz, B., Paprottka, H., and Schulte-Derne, M. "Meine liebe Fibel: Schreiblehrgang", Verlag Ferdinand



Kamp Bochum, 1972 German/European writing style textbook [Hemstreet73] Hemstreet, Harold S. "Video Amplifier", United States Patent 3,713,100, January 23, 1973 Cited in Bokser88; Cited in Tappert88b: on segmentation combined with character recognition Optical/typed character recognition by comparing with a set of individual prototypes for each possible shape, and taking best match score. Comparison may be pixel-wise or slice-wise comparison. [HenckelsLP71] Henckels, L.P. "A Parametric Method for Developing and Evaluating Handprinted Character Recognition Algorithms", ScD Thesis, Department of Electrical Engineering, Massachusetts Institute of Technology, June 1971 Massachusetts Institute of Technology Thesis on generating simulated/artificial variability in handwriting Find samples at the boundary of where recognition problems occur Says problems in recognition algorithms should be found systematically Says need 10,000 training samples to get feature space error down to 1% Says Neisser's figure for human recognition too poor for practical device(!) Says testing takes up most of the effort, but is treated too lightly Quotes "dominant factor factor in performance is care in printing" [HenrichonEG75] Henrichon, E.G. and Bloom, H.J. "Method and System for Optical Character Recognition", United States Patent 3,930,231, December 30, 1975, assigned to Xicom Data Entry Corporation, Newton Upper Falls, Massachusetts. Three methods of character recognition are matrix matching, Feature extraction, and curve tracing [HerotCF74a] Herot, Christopher F. "Using Context in Sketch Recognition", Master's Dissertation, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1974 Context in sketch recognition (sketch editing?) on tablet. Example is syntactic rules for construction of a house: e.g. house/living-room/wall/window [HerrickEM74] Herrick, E.M. "A Taxonomy of Alphabets and Scripts", Visible Language, Vol VIII No 1, Winter 1974, pp 5-32 [HitachiI75] Hitachi, I. "Direct Input System for Handprinted Programs and Data", Central Research Laboratory of Hitachi, Ltd., Tokyo, Japan, 1975 Cited in Purcell77 [Hlady75] Hlady, A.M. "Touch Sensitive Position Encoder using a Layered Sheet", United States Patent 3,916,099, October 28, 1975, assigned to Canadian Patent and Development Limited, Ottowa, Canada. Digitizer using surface acoustic wave: transparent touchscreen [HoltAW74] Holt, Arthur W. "Algorithm for a Low Cost Hand Print Reader", Computer Design, February 1974, pp 85-89 Simple OCR for handwriting recognition, using a single vertical line as a constraint, measuring closed/open loops and areas as features [Honeywell75a] Honeywell Information Systems "MULTICS Programmer's Manual Reference Guide, Series 60 (Level 68)", Honeywell Information Systems AG91-01 Dec. 1975 Multics operating system architecture and overview: CLI command language, input/output facilities, file system. Segmentation/paging, virtual memory; access control lists for selective sharing (ACL), access isolation mechanism (AIM), ring structure and levels of privilege. Introduction to Multics software development environment, dynamic (run-time) linking to subroutines. Unstructured, sequential, indexed files. Section VI on access control, modes, syntax of ACLs. AIM allows delegation of access proviges grantor has via ACL. Processes (and all objects) identified by name Project.User.Tag. [HongJP74] Hong, J.P. "Pattern Recognition: Invariant Stochastic Feature Extraction and Statistical Classification", in JPL Space Programs Summary 37-58, Vol III, 1974 Character recognition features used by IBM OCR page reader. Measurements on data (feature extraction) and classification algorithm closely tied to problem at hand [HooSK75] Hoo, S.K. "Coordinate Locator", IBM Technical Disclosure Bulletin, Vol. 18 No. 5, October 1975 pages 1498-1499 Cited in Bennett 91 optical stylus digitizer tablet somewhat similar to Anoto: visually transparent surface using a coded grid placed over a display: appears to give proximity detection also. [Howells75] Howells, J.A. "Apparatus for Recognizing Hand Printed Characters", United States Patent 3,909,785, September 30, 1975, assigned to Amperex Electronics Corporation, Hicksville, New York. Nine-zone/sector/region recognition [HurstGS72a] Hurst, George S. "Electrical Sensor of Plane Coordinates", United States Patent 3,662,105, May 9, 1972 Resistive sheet digitizer, two resistive sheets with AC voltage gradients in orthogonal directions. Single probe (not stylus) punctures both sheets and picks up both signals. [HurstGS74a] Hurst, George S. "Electrographic Sensor for Determining Planar Coordinates", United States Patent 3,798,370, March 19, 1974



Resistive sheet digitizer, single sheet, voltage gradient applied alternately in vertical and horizontal direction, using second conductive sheet and intermediate layer of deformable insulation such as a gel, that permits the two sheets to be pressed into effective contact. [HurstGS75a] Hurst, George S. and Colwell, William C. Jr. "Discriminating Contact Sensor", United States Patent 3,911,215, October 7, 1975 Resistive sheet digitizer, single sheet, voltage gradient applied alternately in vertical and horizontal direction, using second conductive sheet and small spacer bumps to hold two sheets apart: palm discrimination versus narrow stylus tip. [HussainABS72a] Hussain, A.B.S., Toussaint, G.T., and Donaldson, R.W. "Results Obtained Using a Simple Character Recognition Procedure on Munson's Handprinted Data", IEEE Trans. Computers, Vol 21, February 1972, pp 201-205 (abstract only) Refers to Munson's data collection / test sample data base for handwriting [IJCPR74] IJCPR Program "Second International Joint Conference on Pattern Recognition", Lyngby, Denmark, August 1974 [IannouJT75] Iannou, J.T. "Digital Position Measurement System with Stylus Tilt Error Compensation", United States Patent 3,873,770, March 25, 1975, assigned to Bendix Corporation, Southfield, Michigan. Tilt correction patent for tablet digitizer Tappert's patent list shows this as 3,875,770. [IchikawaT74] Ichikawa, T. and Yoshida, Y. "Online Recognition of Handprinted Characters with Associative ReadOut of Patterns in Memory", Proceedings of 2nd International Joint Conference on Pattern Recognition, August, 1974, pp 206-207 Cited in Litvin82 [IijimaT72] Iijima, Taizo and Mori, Kenichi "Pattern Identification Systems Operating by the Multiple Similarity Method", United States Patent 3,688,267, and Tokyo Shibaura Electric Co., Limited, August 29, 1972 From a old list of Polhemus patents Inner-product comparison (pixel map) OCR character recognition against standard reference template patterns, RMS average of match scores compared to threshold [IijimaT75] Iijima, Taizo and Mori, Kenichi "Pattern Identification System", United States Patent 3,906,446, assigned to Taizo Iijima and Tokyo Shibaura Electric Co., Limited, Tokyo, Japan. Cited in MaedaK87 OCR of characters (handwritten?) using sums of squares of values for similarity: improvement over patent 3,688,267. [IngallsD75a] Ingalls, Dan "Bit BLT", Xerox PARC Inter-Office Memorandum to Alto Users, November 19, 1975 First description of BIT BLT routines for Xerox Alto computer: Bit Block Transfer. Main operation is FillBitMap(...). includes Source code with date of April 20, 1976 [IrbyCH74a] Irby, Charles H. "Display techniques for interactive text manipulation", Proc. NCC '74, Chicago May 610, pp. 247-255 NLS text editing with mouse for text selection and keyboard: could show two (text) files at once, multiple users edit same text from separate terminals. [Ishizaki74] Ishizaki, Hiroyuki; Toba, Teruo; Umeda, Shozo "System for Reading Out the Coordinates of Information Displayed on a Matrix Type Display Device", United States Patent 3,832,693, August 27, 1974, Assigned to Fujitsu Light pen digitizer, subdividing display into blocks in the X and then in the Y direction. [Jans75] Jans, C.L. "An Investigation of U-V Discrimination", S.B. Thesis, Department of Electrical Engineering and Computer Science, May 1975, Massachusetts Institute of Technology, Cambridge, Massachusetts. U-V recognition: functional attributes Blesser's and Shillman's group at Massachusetts Institute of Technology [JohnsonRG72a] Johnson, Ralph G. and Fryberger, David "Touch Actuable Data Input Panel Assembly", United States Patent 3,673,327, June 27, 1972 Optical touchscreen designs. Light beams in X/Y grid over CRT display, finger interrupts and X and a Y beam. Internal reflections in deformable layer, finger pressing on layer interrupts internal beams. Sonic Rayleigh waves in surface of glass detected acoustically. [JohnsonS75] Johnson, S.C. "YACC: Yet Another Compiler Compiler", Computing Science Technical Report No 32, 1975, Bell Laboratories, Murray Hill NJ 07974 UNIX documentation on YACC; regular-expression pattern matching [JohnstonVS74] Johnston, V.S., and Chesney, G.L. "Electrophysiological Correlates of Meaning", Science, Vol 186, December 1974, pp 944-946 Neuronal activity, evoked potentials in frontal lobe (but not visual cortex), matches human perception/recognition of recognizing ambiguous character shapes [Julesz75] Julesz, B. "Experiments in the Visual Perception of Texture", Scientific American, April 1975, pp 34-43 "It is well known from neuro-physiological studies that the visual systems of cats and monkeys incorporate cluster detectors in several stages of hierarchically increasing complexity, beginning at the retina of the eye and extending to the highest levels of the cerebral cortex." Why adaptive recognition is not human



perception/cognition: multiple levels Perception (low-level, immediate) vs cognitive (high-level, takes time) Mentions cognitive/perceptual problem of reading alphabets [KaplowR73a] Kaplow, Roy; Schneider, David; Smith, Franklin C. Jr.; and Stensrud, William R. "Computer Assistance for Writing Interactive Programs: TICS", ACM SIGCUE Computer Uses in Education Bulletin Vol 7 No 2, April 1973, pp. 7-14 TICS system for authoring tutorials for interactive education. Cites to Coursewriter. Examples include foreign-language instruction. Courses structured as web of nodes, connected by multiple possible branches. Compare with TutorText early 1960s. [KayA72] Kay, Alan C. "A Personal Computer for Children of All Ages", Xerox Palo Alto Research Center: Proceedings of the ACM National Conference, 1972 Early paper by Alan Kay on Dynabook, does not show handwriting input: touchscreen (strain gauges at corners) allows on-screen virtual keyboard display, some references to drawing. Early personal computer with ARPANet/Internet. [KnowltonKC75a] Knowlton, K.C. "Virtual Pushbuttons as a Means of Person-Machine Interaction", Proceedings of the IEEE Conference on Computer Graphics, Pattern Recognition, and Data Structure, May 1975, pp 350-351 Cited in FoleyJD82 [KnowltonKC75b] Knowlton, K.C. "Interactive Input-Output Computer Terminal with Automatic Relabeling of Keyboard", United States Patent 3,879,722, assigned to Bell Laboratories, April 22, 1975 Re-lable keyboard by projecting images on top. Compare with Kaplow. See Knowlton77 [Kolers75] Kolers, Paul A. and Perkins, David N. "Spatial and Ordinal Components of Form Perception and Literacy", Cognitive Psychology, Vol 7, 1975, pp 228-267 Graphical context on rotated text perception with human readers: relation to dyslexia. [KorolevE71] Korolev, E. "On automatic recognition of context", Proceedings of the International Conference on Computer Linguistics, 1971 Cited in Suen79 [Krause73] Krause, P. and Bleichrodt, H. "Experiments on direct input and recognition of handwritten digits and handprinted letters with computers", Post Office Research Department, London, England, 9-16, March 1973 Cited in Blatt88: 97.1% character recognition rate [KuipersJ75] Kuipers, Jack "Object Tracking and Orientation Determination Means, System and Process", United States Patent 3,868,565, February 25, 1975 Polhemus patent on generating rotating fields to digitize position and angular position at same time (3-D tablet plus rotation, theta, phi, rho), six-dimensional electromagnetic digitizer (3,868,564 is a motor patent from the USSR) [KuklinskiT73] Kuklinski, T. "Goodness-Time Experiments", internal report, Cognitive Information Processing Group, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, May 18, 1973 Cognitive experiments showing correlation between human rating of "goodness" of paired character shapes, and recognition time [KuklinskiT74] Kuklinski, T., draft manuscript "Plasticity Effects in the Perception of Handprinted Characters", Massachusetts Institute of Technology, Cambridge, Massachusetts, February 20, 1974 Graphical context affects perception of characters, not just semantic context [KuklinskiT75] Kuklinski, T., S.M. and E.E. Thesis "Plasticity Effects in the Perception of Hand-printed Characters", Massachusetts Institute of Technology, Cambridge, Massachusetts, 1975 Graphical context affects perception of characters, not just semantic context [Labov72] Labov, W. "The Boundaries of Words and Their Meanings", Conference on New Ways to Analyzing Variation in English, Washington, D.C., October 28, 1972 Variability/uncertainty a fundamental property of formal description Linguistics/perception/cognition as study of boundaries, not categories Justification of fundamental philosophy for our boundary-cognitive Analysis of functional attributes/features Feature analysis: not single property, but co-occurrence of large number of items [LeeT75] Lee, T.K-Y. "The Phenomenon of Line Addition in Character Recognition", S.B. Thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, May, 1975 Y-V discrimination: functional attributes Barry's and Shillman's group at Massachusetts Institute of Technology [Lemelson75] Lemelson, Jerome H. "Scanning System and Method", United States Patent 3,918,029, November 4, 1975 Optical hand-held scanner using a single sensing element, straight-edge or other guides for moving the stylus across a page: compare to optical digitizers [Lesk75] Lesk, M.E. "Lex - A Lexical Analyzer Generator", Computing Science Technical Report No 39, 1975, Bell



Laboratories, Murray Hill NJ 07974 UNIX documentation on LEX: regular-expression pattern matching [Lesser75] Lesser, Victor, Fennel, Richard, Erman, Lee and Reddy, Raj "Organization of the Hearsay II Speech Understanding System", IEEE Trans. Acoustics, Speech and Signal Processing, Vol ASSP-23 No 1, pp 11-24, 1975 Cited in Purcell77 [LinWC74] Lin, W.C. and Scully, T.L. "Computer Identification of Constrained Hand-printed Characters with a High Recognition Rate", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-4, November 1974 Statis/OCR handwriting/handprinted character recognition, if first level classifier has no clear winner (20 weighted features), uses factors of two highest-scored candidates. 20-segment constrained handwriting style, Gaussian point-to-line distance to segments. Limited punctiation: period/dot does not fit onto segments. [Locke71] Locke, John L. and Locke, Virginia L. "Deaf Children's Phonetic, Visual, and Dactylic Coding in a Grapheme Recall Task", Journal of Experimental Psychology, 1971, Vol 89 No 1, pp 142-146 Study of human character recognition for handwritten letter pairs, comparing the effects of auditorally similar letters to visually similar letters in deaf children. Refers to phonetic coding in memory. [MachineDesign72] Machine Design "The Instant Draftsman", Machine Design magazine, April 6, 1972, pages 6872 Cited in Fitzgerald76 patent, example of sketch recognition / sketch processing [Malevard72] Malevard, Lucien C.; Marty, Pierre "Graphic Data Transcription system", United States Patent 3,632,874, January 4, 1972 Resistive sheet/film digitizer, using a single sheet and a conductive stylus and wire. Mentions integration with a display by applying the measured voltage directly to the deflection plates of a CRT. [Manis71] Manis, Melvin "Context Effects in Communication -- Determinants of Verbal Output and Referential Decoding", in "Adaptation-Level Theory", New York: Academic Press, M.H. Appley, editor, 1971, pp 237-255 Theory for perceptual interpretation, subjective impression for context Kuklinski has margin notes on psycho-physical (perception), judgmental, and linguistic context Refers to other work on semantic (language) context Page 239 - relates to inadvertent bias in testing recognition results, since judgment affected by the rest of the characters the subject looks at (or writes) Page 240 - Kuklinski's margin notes on all the forms of context bias [MartinWA71a] Martin, William A. "Computer Input/Output of Mathematical Expressions", 2nd Symposium on Symbolic Algebraic Manipulation, A.C.M., March 23-25, 1971, Los Angeles, California, pp 78-ff. Cited in FoleyJD82: for user-interface for mathematical input, two-dimensional. Illustration shows use of character extents, spacing, size, position, for parsing two-dimensional hand-written mathematical formulae. [Meisel72] Meisel, W.S. "Computer-Oriented Approaches to Pattern Recognition", Academic Press, New York/San Francisco/London, 1972 (partial copy) Compares "abstract" issues of pattern recognition with heuristic/linguistic approach specific to problem area Feature selection by statistics "gives no insight into dependencies among features" Inter-set and intra-set feature distances an ill-defined concept "curse of dimensionality": how feature interact and overlap a hard problem Project features onto one dimension and you cannot tell how features inter-relate [MillerLK72] Miller, Leon K. "Letter Recognition: Effects of interitem similarity and report requirements", Perception and Psychophysics, Vol 11 No 3, 1972, pp 252-256 Human recognition/perception: human readers (non-Arabic, non-Hebrew, non-Chinese) perceive characters to the left of the fixation better than to the right [MillerSW71] Miller, S.W. "Display Requirements for Future Man-Machine Systems", IEEE Trans. Electronic Display, Vol ED-18 No 9, September 1971, pp 616-ff. CRT displays are too large, and resolution too low for many application (ca. 1971) [MitchellRJ74a] Mitchell, R.J. "Pressure Responsive Resistive Material", United States Patent 3,806,471, April 23, 1974 Pressure-sensitive conductive rubber: conductive particles in an elastomeric binder [Moayer75] Moayer, B. and Fu, K.S. "A syntactic approach to fingerprint pattern recognition", Pattern Recognition, Vol 7, 1975, pp 1-23 [MoriwakiM75a] Moriwaki, Masahiro, Sakaguchi, Mitsuhito, and Mita, Yoshinari "Hologram Graphic Data Tablet Apparatus Using a Vernier", United States Patent 3,906,465, September 16, 1975 Nominally optical digitizing tablet using hologram patterns printed in a glass surface. [MossD75a] Moss, David "Character Recording System", United States Patent 3,903,502, September 2, 1975 Character recognition for simplified character set using endpoints/termini only, which had to touch one of six reference points/zones in a hexagonal pattern [MottEM72a] Mott, E.M. "A Transparent Touch -Screen Device for Interactive Computer-Graphics Displays", Report RHEL/R 248, Rutherford High Energy Laboratory, 1972



Infrared-optical touchscreen [MuellerRE74a] Mueller, Robert E. "Direct Television Drawing and Image Manipulating System", United States Patent 3,846,826, November 5, 1974 Optical drawing input (analog digitizer) using optical prisms that can respond to any touching object: rubber stamps, fingers, brushes: light affects flying-spot scanner for a "paint-pulse" input. Also shows piano-type keyboard for selecting visual effects. Later cited by Greene for multi-finger/object (multi-touch) drawing system using same principle. [Nakata72] Nakata, K., Nakano, Y., and Uchikura, Y. "Recognition of Chinese Characters", Proc. of Conference on Machine Perception of Patterns and Pictures, pp.45-52 (1972) OCR by projecting in X and in Y direction for histogram [NarasimhanR71a] Narasimhan, R. and Reddy, V.S.N. "A Syntax-Aided Recognition Scheme for Handprinted English Letters", Pattern Recognition 3, pp 345-361, 1971 [NaylorWE71a] Naylor, W.C. "Some studies in the interactive design of character recognition systems", IEEE Trans. Computing, Vol 20, September 1971, pp 1075-1086 Cited in Blatt88: 80.4% character recognition rate [NegroponteN73a] Negroponte, N. "Recent Advances in Sketch Recognition", NCC 1973, AFIPS Press, pp 663-675 Cited in FoleyJD82 Sketch recognition as a form of data compression: primary example is recognizing straight lines, simple curves. Constraints (two lines parallel, same length, etc.) [NegroponteN75a] Negroponte, Nicholas "Sketching: A Computational Paradigm for Personalized Searching", Working Paper, Architecture Machine Group, Department of Architecture, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1975 Cited in Purcell77: Indexing by handwritten ink user-interface (?) [NemirovskySM71a] Nemirovsky, S.M., Sternberg, J., and Lieberman, G. "Signature Identification by Means of Pressure Patterns", United States Patent 3,618,019, November 2, 1971, assigned to Conversational Systems Corporation, New York, New York. Signature verification, mis-called signature identification, using pressure pattern without position. Pressure transducer is strain gauge or piezoelectric sensor in stylus/pen, etc, alternative the tablet/platen is a force/pressure transducer. [Neuhoff75] Neuhoff, D.L. "The Viterbi algorithm as an aid in text recognition", IEEE Trans. Information Theory, Vol IT-21, 1975, pp 222-226 Cited in Sinha88 [NewmanWM73a] Newman, W.M. and Sproull, R.F. "The Ledeen Character Recognizer", in Principles of Interactive Computer Graphics, McGraw-Hill, New York, 1973, pp 575-582, Appendix VIII The handwriting/gesture symbol recognition system used by Applicon CAD/CAM system: everyone used to copy it. Trainable. Refers to training to recognize a full alphabetic character set, but with some neography required. [NewmanWM73b] Newman, W.M. and Sproull, R.F. "Principles of Interactive Computer Graphics, second edition", ISBN 0-07-046338-7, McGraw-Hill, New York, 1973, 1979 (hardcopy book) Chapter eleven: cannot use mouse for tracing, compared with digitizing tablet. Mentions RAND tablet (electromagnetic), voltage-gradient (resistive tablet), Sylvania tablet (electrostatic/capacitive), acoustic tablet (SAC: see Science Accessories Corporation) [NewmanWM73c] Newman, W.M. and Sproull, R.F. "Tablets: Principles of Interactive Computer Graphics, second edition", ISBN 0-07-046338-7, McGraw-Hill, New York, 1973, 1979 Scan of book excerpt: RAND tablet, etc. [Nishida72] Nishida, Nobuo and Sakaguchi, Mitsuhito "Hologram Graphic Data Tablet", United States Patent 3,658,402, April 25, 1972 Digitizing tablet, nominally optical: holograms embedded at different positions on the tablet [NollAM75a] Noll, A. Michael "Tactile Man-Machine Communication System", United States Patent 3,919,691, November 11, 1975 Electromechanical 3-D rod (3DOF) with force feedback to simulate surfaces haptically. [OkaM74x] Oka, Masatomo and Yasuhara, Makoto "Signature Verification Experiment Based on Nonlinear Time Alignment: A Feasibility Study", Research Manuscript, 1974 (date approximate), Nippon Electric Company, Tokyo Japan From Kuklinski's old files Dynamic time-warping / non-linear time alignment via dynamic programming for signature verification: published later in 77/78 [PapenmeierFH75a] Papenmeier, F.H. "BRAILLEX - Das erste Gerät mit einer elektronischen Braillezeiler", Papenmeier.de, fetsched 2015 Historical notes on Papenmeier BRAILLEX, early/first tactile Braille display. [Pavlidis74] Pavlidis, Theo and Horowitz, S.L. "Segmentation of Plane Curves", IEEE Trans. Computers, Vol C-23 No 8, 1974, pp 860-870



Best piece-wise fit of line segments to a polygon: split and merge segments to drive error below a threshold Piece-wise line segment fit helps on feature extraction, data compaction, and noise filtering [Pavlidis75] Pavlidis, Theo, and Ali, F. "Computer Recognition of Hand-written Numerals by Polygonal Approximations", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-6, pp 610-614, 1975 Chain codes - cite this one so Pavlidis gets his licks in [PerlmanR74a] Perlman, Radia "TORTIS (Toddler's Own Recursive turtle Interpreter System)", Report LOGO-9, M.I.T. Artificial Intelligence Lab, March 1974 Logo graphical turtle, with control interface with physical buttons for toddlers: buttons labeled with command and numbers, with a memory, so see whether toddlers (ages 4-6) would learn to program. Cited in Minsky84 [PeronneauG72a] Peronneau, Georges and Poizat, Henri "Position-Indicating System", United States Patent 3,657,475, April 18, 1972 Touch-panel digitizer using force gauges/sensors at three or four corners of a plate: compute location of touch from corner forces. Plate can be transparent, and sensors are only at the corners. [PobgeePJ71] Pobgee, P.J. and Parks, J.R. "Applications of a Low Cost Graphical Input Tablet", IFIP 1971, NorthHolland, Amsterdam, pp ??, (TA-4-169) Cited in FoleyJD72 [PobgeePJ75] Pobgee "Graphical Input Apparatus for Electrical Apparatus", United States Patent 3,885,097, May 20, 1975 3D electromagnetic digitizer (do not work on metallic objects) 0.032" resolution [PowersVM72a] Powers, V.Michael "Handwritten Character Recognition from Pen Direction", Naval Postgraduate School, Monetery CA April 21, 1972 Describes character recognition based solely on chain codes / stroke direction / arc segments [PowersVM73a] Powers, V.Michael "Pen Direction Sequences in Character Recognition", Pattern Recognition, Vol 5. pp 291-302, March 1973 Describes character recognition based solely on chain codes / stroke direction [Quarmby71] Quarmby, David J. and Rastall, John "Experiments on Handwritten Numeral Classification", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-1 No 4, October 1971, pp 331-338 Edge-defining algorithms for OCR of handprinting/handwriting, maximum likelihood statistical classifier [Ramer72] Ramer, U. "An iterative procedure for the polygonal approximation of plane curves", Computer Graphics Image Processing, Vol 1, November, 1972, pp 244-256 Cited in Burr81 [RebaneGJ74a] Rebane, George J. and Grossman, Norman R. "Graphic Input Device", United States Patent 3,875,022, December 24, 1974 Rear-projection display on translucent acoustic time-delay touchscreen digitizer, receivers for sounds of tap/contact of stylus with the digitizer screen. Sound generator in stylus tip. [RichardsonS72a] Richardson, S.; Chan, H.L.; Lee, A.; and Teo, S.T. "The Mueller-Lyer Illusion: A Cross-Cultural Study in Singapore", Ergonomics, Vol 15 no 3, 1972, pp. 293-298 Study showing that culture affects optical perception: Singapore Chinese more subject to Mueller-Lyer linelength illusion than Singapore British [RisemanEM71a] Riseman, E.M. and Ehrich, Roger W. "Experiments in the contextual recognition of cursive script", IEEE Trans. Computers, Vol C-24, 1975, pp 182-194 Cited in Goshtasby88 for binary letter pairs (digrams) for spelling context, following by spelling dictionary look-up for what's left, compare with Viterbi algorithm or with trigrams [RisemanEM71b] Riseman, E.M. and Ehrich, Roger W. "Contextual word recognition using binary digrams", IEEE Trans. Computers, Vol C-20, April 1971, pp 397-403 Cited in DosterW77 on a spelling dictionary: important point of diagram is only zero/non-zero value [RisemanEM71c] Riseman, Edward M. "Logical Networks for Feature Extraction", IEEE Trans. on Sytems, Man, and Cybernetics Vol CMC-1 no 1, January 1971, pp. 43-ff Automatic determination of useful features for OCR character recognition (on thinned images) by mathematical determination of clusters matching desired outcomes. Notes that the test data set was unrealistically small training set. [RisemanEM73a] Riseman, E.M. and Ehrich, Roger W. "Contextual word recognition using binary digrams", IEEE Trans. Computers, Vol C-20, April 1971, pp 397-403 Preliminary abstract of Fisher75 [RisemanEM73b] Riseman, E.M., Hanson, A.R. and Fisher, E.G. "Feature Selection Using Thresholded Measures", Proceedings of 1973 International Conference on Cybernetics and Society, Boston, Massachusetts, November 5-7, 1973 [RisemanEM74] Riseman, E.M., and Handson, A.R. "A contextual postprocessing system for error correction using binary N-grams", IEEE Trans. Computers, Vol C-23, 1974, pp 480-493 (abstract only)



Cited in DosterW77 on a spelling dictionary. File also contains list of numerous reference *to* Riseman for context information in handwriting and speech recognition. [RodgersJL73a] Rodgers, James L. "Apparatus for Converting the Position of a Manually Operated Instrument into an Electrical Signal", United States Patent 3,767,858, October 23, 1973 Electrostatic grid digitizer, same signal at same frequency fed on one side and other side but at different phase, stylus detects phase angle across tablet: resistor ladder to apply phase shift on each conductor. Vertical and horizontal elements of grid/matrix operate at different frequencies simultaneously, so X and Y determined simultaneously. [RodgersJL75a] Rodgers, James L. and Tyrrell, Josph L. "Electrical Writing Pen and Sensor", United States Patent 3,886,311, May 27, 1975 Stylus design of capacitive/electrostatic grid digitizer, tablet is the transmitter. Cord on stylus. [RosenfeldA71a] Rosenfeld, A. and Thurston, M. "Edge and curve detection for digital scene analysis", IEEE Trans. Computers, Vol C-20, May 1971, pp 562-569 Cited in Teh89 [RosenfeldA72a] Rosenfeld, A., Thurston, M., and Lee, Y.H. "Edge and curve detection: further experiments", IEEE Trans. Computers, Vol C-21, July 1972, pp 677-715 Cited in Teh89 [RosenfeldA73a] Rosenfeld, A. and Johnston, E. "Angle detection on digital curves", IEEE Trans. Computers, Vol C-22, September 1973, pp 875-878 Cited in Teh89 [RosenfeldA75a] Rosenfeld, A. and Weszka, J.S. "An improved method of angle detection on digital curves", IEEE Trans. Computers, Vol C-24, September 1975, pp 940-941 Cited in Teh89 [RosenfieldJM73] Rosenfield, John M., Cranston, Fumiko E., and Cranston, Edwin A. "Japanese Language Calligraphy", in The Courtly Tradition in Japanese Art and Literature, Fogg Art Museum, Harvard University, 1973, pp 14-24 Background history of Chinese/Japanese Kanji, Katakana, hiragana: educated Japanese cannot recognize Kana scripts [SaltzerJH74a] Saltzer, Jerome H. "Protection and the Control of Information Sharing in Multics", CACM Vol 17 No 7 July 1974, pp. 388..402 Overview of protection mechanisms in Multics and file system: ACL access control lists, list of user/project/compartment (extendible) and access rights, associated with object. (Capability system would associate with user.) Description of "*" wildcard/don't-care syntax. [SaltzerJH74b] Saltzer, Jerome H. "The Protection of Information in Computer Systems", Proc. Fourth ACM Symposium on Operating System Principles, October 1973 (alternate version of CACM 1974 paper) Overview of protection mechanisms in secure computer systems: ACLs access control lists embedded in indirect objects -- separate from object for which permission needed, separate access controller granting access, looks for match. Mentions encipherment/encryption (notes that security classifications from military restrict publication of ideas): authentication by handwriting signature verifification or rhythm of keyboard typing (biometrics), physical tokens (unduplicatable identity card/token). Multi-node networked systems. [SaltzerJH75a] Saltzer, Jerome H. and Schroeder, Michael D. "The Protection of Information in Computer Systems", Proc. IEEE, Vol 63 no 9, September 1975, pp. 1278-1308(alternate version of CACM 1974 paper) IEEE journal version of 1974 ACM paper. Overview of protection mechanisms in secure computer systems: ACLs access control lists embedded in indirect objects -- separate from object for which permission needed, separate access controller granting access, looks for match. Mentions encipherment/encryption (notes that security classifications from military restrict publication of ideas): authentication by fingerprint reader, handwriting signature verifification or rhythm of keyboard typing (a.k.a. keystroke dynamics) (biometrics), physical tokens (unduplicatable identity card/token). Multi-node networked systems. Quote: "Work in progress is not well represented by published literature." [Sayre73] Sayre, Kenneth M. "Machine Recognition of Handwritten Words: A Project Report", Pattern Recognition 5, pp 213-228, 1973 79% handwriting recognition on 84 cursive script words (OCR) Does cursive script recognition as part of handwriting recognition (OCR) Statistical recognition (digrams) can make recognition worse: "fo" more frequent than "fa" in English, so ambiguous "far" would always be recognized wrong as "for" 90% per character gives 81% per pair, 72% per triplet, etc.: fix with context Used handwriting data samples from Bell Telephone Labs (OCR) Cites very early work by Harmon, internal Bell Telephone Labs reports [Schlang74] Schlang, Arthur "Electro-Optical Handwritten Character Reader", United States Patent 3,784,982, assigned to Isotec, Incorporated, Woodbury, New York, January 8, 1974 Cited in Tappert88b: on segmentation combined with character recognition OCR scanner for handwritten characters: hardware to detect inflection points, endpoints, sub-stroke (horizontal and vertical segments)



of strokes Describes two-level recognizer with (generic) first-level and detail-level feature analysis [SearsFW74a] Sears, Francis W.; Zemansky, Mark W.; and Young, Hugh D. "College Physics, Fourth Edition", Addison-Wesley, 1974 (hardcopy book) Standard physics reference and textbook [Shepard72] Shepard, Roger N. "Psychological Representation of Speech Sounds", in "Human Communication: A Unified View", Edward E. David, Jr. and Peter B. Denes, editors, McGraw-Hill Book Company, 1972, pp 67-ff. Confusion data matrix on phoneme recognition/discrimination in speech recognition: degree of similarity, confusability, association, psychological "proximity": how to find out a distance metric for pattern recognition. proximity analysis, signal-to-noise ratio in pattern space, multi-dimensional scaling [ShillmanR74a] Shillman, R., Kuklinski, T.T., and Blesser, B.A. "Experimental Methodologies for Character Recognition Based on Phenomenological Attributes", Proceedings of the 2nd International Joint Conference on Pattern Recognition, Copenhagen, Denmark, August 13-15, 1974, pp 195-201 functional attributes reliable (e.g. tail) on block character recognition: graphical context biases the attribute equally for all characters. Did not include context where only certain characters with the attribute were shifted? [ShillmanR74b] Shillman, R., Cox, C., Kuklinski, T., Ventura, J., Eden, M., and Blesser, B. "A Bibliography in Character Recognition: Techniques for Describing Characters", Visible Language, Vol VIII No 2, Spring 1974, pp 151-166 General references on character recognition, especially cognitive psychology aspects Includes references on insight through study of character formation / generative variability Includes references on psychological descriptions of characters [ShillmanR74c] Shillman, R.J. "Seeing is Deceiving: The Perceived Shortening of Thick Lines", manuscript in preparation, Cognitive Information Processing Group, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, 1974 Brief user study of apparent line shortening for wider/narrow characters: graphical context on character recognition. [ShillmanRJ74] Shillman, R. "Character Recognition Based on Phenomenological Attributes: Theory and Methods", PhD Thesis, Massachusetts Institute of Technology, Department of Electrical Engineering, 1974 Paper copy only. Lists features that are not perceptually significant Shillman's Thesis: also published as BlesserB73 [ShillmanRJ75a] Shillman, Robert J. "Hand Printed Digit Reading under Conditions of Limited Control", Draft manuscript, Massachusetts Institute of Technology, 1975 OCR for discrete boxed handwriting character recognition of digits with cost and accuracy of keypunch typing transcription manually [ShinarD73a] Shinar, David and Owen, Dean H. "Effects of form rotation on the speed of classification: The development of shape constancy", Perception and Psychophysics, Vol 14 No 1, 1973, pp. 149-154 Rotationally invariant (at 90 degree increments) recognition of simple geometric shapes, apparently by learning shapes in each orientation: not specific to character recognition. [Simon72] Simon, J.C., Checroun, A. and Roch, C. "A Method of Comparing Two Patterns Independent of Possible Transformations and Small Distortions", Pattern Recognition, Vol 4, 1972, pp 73-81 Cited in Baird84: for template matching in feature space using minimum distance? [SiyP74] Siy, Pep, and Chen, C.S "Fuzzy Logic for Handwritten Numeral Character Recognition", IEEE Trans. on Systems, Man, and Cybernetics, November 1974, pp 570..575 Trainable system using subsets of curved line segments as the 'branch feature set' [SmithEE74] Smith, E.E. and Spoehr, K.T. "The Perception of Printed English: a Theoretical Perspective", in "Human Information Processing", B.H. Kantowitz, editor, Lawrence Hillbaum Associates, Hillsdale NJ 1974, pp 231-275 Template matching vs feature analysis is a continuum, not disjoint Cites several papers on low-level feature extraction in human perception Argument against dynamic matching: "theoretical and empirical reasons for feature extraction" Cites different recognition mechanism for handwritten and printed characters Cites no real consensus on actual features humans use in recognizing letters (which is why functional attribute tests tell you what features) [SpanjersbergAA74a] Spanjersberg, A.A. "Combinations of different systems for the recognition of Handwritten digits", Proceedings of the Second International Joint Conference on Pattern Recognition, August, 1974, pp 208209 Cited in Blatt88: 97.8% character recognition rate [SpillaneRJ75a] Spillane, R.J. "Keyboard Apparatus for Personal Identification", IBM Technical Disclosure Bulletin 04-75, p. 3346, April 1, 1975 Biometric authentication of users based on typing pattern while entering standard enrty code: time pattern and key pressure (force) chracteristics. See also Saltzer 1975. Compares to telegraph key signal ("Fist").



[SteinmannH74a] Steinmann, Helmut "Capacitive Detector Device", United States Patent 3,826,979 Capacitive sensor, used to detect presence of water (or similar fluid) on the other side of a car windshield, and turn on the windshield wipers. Distinguishes between different materials, e.g. does not respond to debris on the windshield. [StruvenWC73a] Struven, W.C. "Experience with Touch Panel Control at SLAC", SLAC technical report SLAC-PUB1191 (A), March 1973 512x512 cross-wire (capacitive?) touch panel digitizer on CRT display, menu buttons arranged 10 x 13. Refers to "panel" as term for heirarchy of menus [Suen73a] Suen, Ching Y. "Factors Affecting the Recognition of Handprinted Characters", Proc. Intl. Conf. on Cybernetics and Society, November 1973, pp. 174-175 OCR recognition of handwriting affected by scanner, lighting, pencil type. Handwritten data collection done in the field, where computer not available for data entry. [Suen74a] Suen, Ching Y. "Human Factors in Character Recognition", Proc. Conf. and Symp. on Systems, Man and Cybernetics, Octb 1974, pp. 253-258 Study of legibitility of 30 different handwriting styles taught in North America. Block printing more legible than cursive (in general). [Suen75] Suen, C.Y. "Handwriting Education - A Bibliography of Contemporary Publications", Visible Language, Vol IX No 2, Spring 1975, pp 145-158 Refers to unpublished treatise on handwriting style variations in North America [SutherlandIE74a] Sutherland, I.E. "Three-dimensional Data Input by Tablet", Proceedings of the IEEE, Vol 62 No 4, April 1974, pp 453-461 (abstract only) multiple-pen user input on large-area tablet digitizer , three dimensional 3D input by providing input for multiple perspective views simultaneously. Compare with Buxton multi-hand/two-hand input, multi-touch. [SutherlandNS73] Sutherland, N.S. "Object Recognition", Chapter 8 of "Handbook of Perception" Carterette and Firedman, ed., Academic Press, New York and London, 1983 General essay on human pattern recognition. Visual recognition of objects: segmentation of images, grouping of features, work for varying rotation and orientation. Cites multiple studies that recognition is by syllable rather than by phoneme. [Szanser72] Szanser, A.J. "Elastic matching in automatic pattern recognition", in Machine Perception of Patterns and Pictures, Teddington, April 12-14, 1972, pp 328-333 Cited in DosterW77 on spelling dictionary [SzanserAJ73a] Szanser, A.J. "Bracketing techniques in elastic matching", Computer Journal, Vol 16 No 2, pp. 132-134, 1973 Elastic matching applied to general strings of characters/codes: spread up by assuming initial part of pattern to be matched is not garbled, bracking occurences of that sequence in input string. [TaggartJR73a] Taggart, James Richard "Reading a Sketch by HUNCH", Master Dissertation, EE., M.I.T. May 1973 Data-reduction in sketch recognition. Corners are important feature, detected better when taking into account drawing speed -- jitter when drawing at low speed badly affected line-straightening. Proposal to use pressure: slowly drawn or heavy pressure means user input is more deliberate. Sylvania data tablet, transparent tablet/touchscreen, two distance levels for proximity sensing. Storage-tube display (hard to erase, compared to drawing with physical ink). Replay sketching data at same timing as input -- compare with interactive system such as Wang Freestyle? Proposal for graphical compiler to process (recognize) sketches: compare to 2D mathematical input? [TeraiH72a] Terai, Hidekazu and Nakata, Kazuo "On-line, Real-time Recognition of Handwriting Chinese (Kanji) Characters and Japanese (Katakana} Syllabary", Central Research Laboratory of Hitachi Ltd, 1972, Monograph, originally published in Japanese for Inst. of Electronics and Comm. Eng. of Japan, June 15, 1972 Recognition of Japanese/Chinese character using chain codes (direction codes) for first level classification of fundamental strokes [TeraiH75a] Terai, Hidekazu and Nakata, Kazuo "Text Editing System using On-Line Real-Time Handwriting Character Recognition", Central Research Laboratory of Hitachi Ltd, Sept, 1975, Monograph, originally published as full paper for Journal of Information Processing Society of Japan, Vol 15 No 6, June 1974 proof-reading symbol recognition, writing directly over the strokes of a character on a tablet character recognition input in a special area of the tablet Cited in Sklarew99 prior art [Thompson75] Thompson, K. and Ritchie, D.M. "UNIX Programmer's Manual -- Sixth Edition", Bell Laboratories, Murray Hill, New Jersey, 1975 General UNIX reference [Tou72] Tou, Y.T. and Gonzales, G.C. "Recognition of Handwritten Characters by Topological Feature Extraction and Multilevel Categorization", IEEE Trans. Computers, Vol 21, pp 776-785, July 1972 Cited in Goodale83 [TownsendJT71a] Townsend, J.T. "Theoretical analysis of an alphabetic confusion matrix", Perception and



Psychophysics, Vol 9, 1971, pp 40-50 Mathematical treatment of using confusion matrix with three different recognition algorithms: choice model, and finite sensory state models. Suggest that better character recognition will need a model of specific distinguishing features: compare to Pencept? [Turner72] Turner, John A. "Electrical Probe-position Responsive Apparatus and Method", United States Patent 3,699,439, October 17, 1972, assigned to Automatic Radio Manufacturing, Inc. Melrose MA Resistive-film digitizer, ratiometric measurement, mounted on front of CRT display [UllmanJR74] Ullman, J.R. "A use of continuity in character recognition", IEEE Trans. Systems, Man, and Cybernetics, 1974, pp 294-300 Cited in Goshtasby88 using edge directions for recognizing hand-printed character if the continuity information between characters is utilized (stroke hooks and continuation marks?) [UncapherKW71] Uncapher, K.W. "The RAND Video Graphic System - An Approach to a General User-Computer Graphic Communication System", RAND Corporation Research Report R-0753-ARPA, April 1971. Presented to AGARD Avionics Panel Technical Symposium on Data Handling Devices, Istanbul, Turkey Jun 1-4, 1970 Copy on file has best image of RAND tablet and display hardware: shows photographic image and electronic text/ink displayed on same surface Multiple users on one graphics system: 32 consoles, each with up to 8 different input devices. Mentions development of all-digital video. Compare with later Schumer patent. [VanDam71] Van Dam, Andries and Rice, David E. "On-line Text Editing; A Survey", Computing Surveys, Vol 3 No 3, September 1971, pp 93-ff. Describes a tablet-based text editor from Carnegie-Mellon University (Coleman69), using symbols/gestures on a RAND tablet: decision tree to recognize symbols. Ease of correction makes requirement for perfect recognition be less. Cost of computers made handwriting recognition not cost-effective in 1971. [VolpeJW75a] Volpe, John W. "Capacitance Matrix Keyboard", United States Patent 3,921,166, November 18, 1975 Capacitive matrix keyboard, similar to capacitive touchscreen. Physical keys move bridging conductor near capacitive elements. [WaltzDL72a] Waltz, David L. "Generating Semantic Descriptions from Drawings of Scenes with Shadows", Ph.D. Thesis, Massachusetts Institute of Technology, Department of Electrical Engineering, 1972 Analysis of line-drawing images using combinatorial algorithms: compare permutation-free search with chain codes in handwriting recognition and structural grammars in sketch recognition [WardJR74a] Ward, Jean Renard "Computer-driven Tutorials on German word-Order Using The TICS Computer Sub-System", SB Thesis, Course VI, M.I.T, May, 1974 Tutorial system for German grammar doing automatic grammatical analysis according to the two-prong word-order grammar of Lohnes and Strothmanm. [WeaverAC74a] Weaver, Alfred C. "On-Line Character Recognition", Department of Computer Science, University of Illinois at Urbana-Champaign, Report No UIUCDCS-R-74-660, August, 1974 Thesis (?) on proposed low-cost handwriting recognition product? Voltage-gradient (resistive sheet) tablet. Lists stroke directions for on-line characters [WeinbergGM71] Weinberg, Gerald M. "The Psychology of Computer Programming", Littleton Educational Publishing, 1971 (hardcopy book) Essays on the role of personality and real-world-practice versus methodology-theory in software development. Anniversary edition 25 years later compared state of progress with that of 1971 publication. [WeisgerberRA75a] Weisgerber, Robert A. "Evalaution of an Ink Print Reading Aid for the Blind: The Sterotoner. Final Report", Veterans Administration Report AIR-3900-12/75-FR, December 1975 Aid for the blind: hand-held wand with optical sensor, device generates different pitch tones to indicate sensor reading. Slit camera (10 sensors verical) user scans across printed material, generates tones for each sensor. Compare to Optacon. [WhetstoneA71a] Whetsone, Albert; Fine, Samuel; Bank, William; and Trumbull, Stanely Phillips "Spark Pen", United States Patent 3,626,483, December 7, 1971 SAC Science Accessories Corporation acoustic stylus for sonic digitizer: generated shock energy sound wave by spark. [WhetstoneA75a] Whetsone, A. and Domyan, S. "Recognize hand-printed characters with a simple algorithm", Electronic Design, February 1, 1975 Summagraphics original design of tablet for character recognition, zone based, numerics of single-strokes and double-strokes [WhetstoneA75b] Whetsone, Albert L. and Brenner, Alfred E. "Position Determination Devices", United States Patent 3,904,821, September 9, 1975 Summagraphics magnetostrictive tablet, using sheet of magnetostrictive material [Widrow73] Widrow, B. "The rubber mask technique, Parts I and II", Pattern Recognition, Volume 5, 1973, pp



175-211 Cited in Burr81 pattern recognition using a template (e.g. shape/classifier of chromosomesin images) by allowing non-exactly-matching signatures: elastic matching on handwriting (?) recognition. [WilliamsTG71] Williams, Thomas G. "An On-Line System for Hand-Printed Input: Final Report for Phase IV", NASA Contractor Report CR-126036, Washington, D.C., August 1971, prepared by System Development Corporation User writes in two dimensions, compiler translates to linear form (shown at top of display) an evaluates. Characters are translated into recognized characters in two-dimensional form for editing in real time. Uses bounding rectangles in parsing. Gesture recognition described, handwriting recognition is in a separate publication. "Scrub mode" is a scratch-out gesture. Arrow gesture indicates move/edit. Horizontal line gesture for "open space/move" editing operation. Double horizontal line for close up. Dot (tap) plus second line to move a character. Two perpendicular stroke gesture for select/move group. [WilliamsTG72a] Williams, Thomas G. "An On-Line two-dimensional computation system", Proc AFIPS '72, proc. of FJCC, pp. 499-506 The Assistant Mathematician (TAM), two-dimensional hand-printed mathematical input for computer programming: visual programming language VPL. Transparent/translucent digitizer tablet with rearprojected CRT display. Mathematic expression is displayed as recognized by system for confirmation by user in semantic context: semantic ambiguity of mathematical language (e.g. "i" as SQRT(-1), variable, or number). Scrubbing (rubout) gesture, editing of electronic ink. [WinstonPH72a] Winston, P.H. "The Massachusetts Institute of Technology Robot", in "Machine Intelligence 7", edited by B. Meltzer and D. Michie, pp 431-463, Edinburgh University Press, Edinburgh, Scotland, 1982 [WipkeW71a] Wipke, W. and Whetstone, A. "Graphic digitizing in 3-D", Computer Graphics (ACM Newsletter) Vol 5 No 4, 1971 Cited by ClarkJH76 as 3D digitizer [WolfbergMS72a] Wolfbeerg, Michael S. "AMBIT/L Programming System User's Guide", ARPA report 1228, CA7201-1711, January 17, 1972 User's guide for AMBIT/L visual programming language / digrammatic language for list processing. Refers to Drawing of AMBIT/L Programs and Their Encodement. See AMBIT/G for lightpen/tablet user interface. [Woods73] Woods, Donald R. and Lyon, James M. "The INTERCAL Programming Language Reference Manual", privately published technical report, 1973 INTERCAL: The world's most bizarre, yet clearly defined, programming language (Geek humor). Compare with spoof on Basic at Data General, 1974? [Xebec74] Xebec Systems, Incorporated "Pattern recognition allows use of handwritten characters in data-entry system", IEEE Spectrum, new product section, Alphabec-70 handwriting recognizer, 566 Xavier Avenue, Sunnyvale, California 94086, October 1974, p 98 Product announcement on SRI/Xebec handwriting character recognition using accelerometer pen stylus. See Hewitt Crane [YamamotoS74] Yamamoto, Shinji and Nakata, Kazuo "Pattern Recognizing System", United States Patent 3,829,831, August 13, 1974, assigned to Hitachi Limited. Cited in Wang,PSP85 32x32 and 8x8 pixel bit-map character recognizer for OCR of Chinese [YasuharaM75a] Yasuhara, M. "Experimental Studies of Handwriting Process", Report of the Research Laboratory of Communication Science, University of Electro-Communications, Japan, Vol 25-2, (Science and Technology section), pp 233-254, March 1975 Mathematical model for pen dynamics in script, but not print or stroke connection [YhapEF75] Yhap, E.F. "Keyboard Method for Composing Chinese Characters", IBM Journal of Research and Development, Vol 19 No 1, May 1975, pp 60-70 40-key special keyboard for input of Chinese Kanji characters, keys used to input shape of the character. [ZadehLA73a] Zadeh, L.A. "Outline of a new approach to analysis of complex systems and decision processes", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-3, pp 24-44, 1973 Biswas81 cites on fuzzy sets for recognition. Focus is on linguistic variables: e.g. height is a variable with possible values short, tall, very tall, extremely tall, medium, etc. A calculus of fuzzy sets instead of a precise calculus of numbers. [ZadehLA75a] Zadeh, L.A., Fu, K.S., Tanaka, K., and Shimura, M. "Fuzzy Sets and Their Applications to Cognitive and Decision Processes", London: Academic Press, 1975 Biswas81 cites on fuzzy sets for recognition. Compendium / graduate text of papers on fuzzy sets and fuzzy logic. Specific applications include handwriting recognition, fuzzy grammars and pattern recognition: compare with chain codes and parsing rules for sketch recognition. [ZadehLA75b] Zadeh, L.A. "The concept of a linguistic variable and its application to approximate reasoning -- I", Information Science, Vol 8 pp 199-249, 1975 Tanaka82 on fuzzy sets. Linguistic variable is variable who value is (overlapping/qualitative) set



membership: e.g. age: young, not young, old, very old vs. 21, 22, 23. Applications to pattern recognition. [ZahnCT74a] Zahn, Charles T. Jr. "An Algorithm for Noisy Template Matching", Proceedings of IFIP 74, 1974, pp 727-732 Post-processing of a distribution of features points in 2D Bayesian space, attempt to cover "noise" from translation,rotation, and noise displacement of the individual points. Cited in Baird84. Only tested with small number of artificial samples. [ZimmermannR72] Zimmermann, R. "Dynamische Beschreibung von Zeichenschreibweise und ihre Ausnutzung zur Erkennung von Handschriftzeichen", Nachrichtentechnische Zeitschrift, Vol 20 No 12, December 1972, pp 542-544 Dynamic description of character drawing and its use for on-line recognition of handwritten characters (in German) zone and chain-code recognition for handwriting/hand-printing of symbols. Based on Doctoral thesis, T.U. Aachen 1972. Describes use of direction information in recognition, light pen on display. biblio70 (Prev page)

Main Page







Main Page


[Agui79a] Agui, Takeshi and Nagahashi, Hiroshi "A Description Method of Handprinted Chinese Characters", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-1 No 1, January 1979 Chinese handwriting recognition: features are concatenate, crossing, near, relative location among partial patterns. [Agui79b] Agui, Takeshi and Nagahashi, Hiroshi "A Coding Method of Chinese Characters", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-1 No 4, October, 1979 Chain codes, production rules, block codes, to encode Chinese characters as chain-code strings for comparison [AhoAV77] Aho, A.V. and Ullman, J.D. "Principles of Compiler Design", Addison-Wesley, Reading, Massachusetts 1977 Compiler/programming language design [Aldefeld80] Aldefeld, B., Levinson, S.E., and Szymanski "A minimum-distance search technique and its application to automatic directory assistance", Bell Systems Technical Journal, Vol 59 No 8, pp 1343-1356, October 1980 Cited in Kahan87: : best method of organizing spelling dictionary for context in character recognition (actual paper uses spoken spelled speech recognition of letters) Context raises 80% on letters to 98.6% on names, 71% to 97.2% on 18000 name entries [AliF77a] Ali, F. and Pavlidis, T. "Syntactic recognition of handwritten numerals", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-7, 1977, pp 537-541 [AndersonRH76] Anderson, Robert H. "Intelligent Terminals: A Potential Solution to Complexity in User-Network Interfaces", Proceedings of ASIS Annual Meeting, San Francisco, California, October 4-9, 1976 NTIS citation index [ApseyR78a] Apsey, R. "Human Factors of Constrained Hand-print for OCR", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 4, pp 292-296, April 1978 Constraints on handwriting using various "pre-printing" guides: dots, boxes, shapes, lines [ArakawaK78a] Arakawa, K., Odata, K., and Masuda, T. "On-line recognition of hand-written characters --



Alphanumeric, hiragana, katakana, kanji", Proc. 4th IJCPR, Kyoto, Japan, November 7-10, 1978, pp 810-812. Published later as (Arakawa82) [AraziB78a] Arazi, B. "Handwriting Identification by Means of Run-Length Measurements", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-7 No 12, pp 878-ff, December 1977 off-line signature verification, study of vertical-line vs. horizontal-line run-length to do signature verification [ArmstrongWW79a] Armstrong, William W. and Gecsei, Jan "Adaptation Algorithms for Binary Tree Networks", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-9 No 5, May 1979, pp 276-ff adaptive tree networks, heuristic responsibility, with specialized sub-trees [Azure77] Azure, Leo L. "Electronic Notebook for Use In Data Gathering, Formatting, and Transmitting System", United States Patent 4,016,542, assigned to Azurdata, Inc., Richland, WA, April 5, 1977 Portable hand-held data-entry device, with radio link to a central computer. Uses a simple keyboard, does not refer to hand-written input [BabcockRT77] Babcock, R.T. "Simulation Method of Feature Selection for Unconstrained Handprinted Characters", M.S. Thesis, Department of Electrical Engineering and Computer Science, June 1977, Massachusetts Institute of Technology, Cambridge, Massachusetts 2-Z recognition: Blesser's and Shillman's group at Massachusetts Institute of Technology [Badler79] Badler, Norman I. and Smoliar, Stephen, W. "Digital Representations of Human Movement", Computing Surveys, Vol 11 No 1, March 1979 Translation of Labanotation dance notation to computer graphics of human movement: dance gestures [Baecker76] Baecker, Ronald "A conversational extensible system for the animation of shaded images", Proc. of 3rd Annual Conference on Computer Graphics and Interactive Techniques, Philadelphia, PA, pp 32-39 Pen/lightpen system for interactive animation. Images, and their motion, sketched by user free-hand. Citation to Alan Kay presentation at 1972 ACM conference [Baecker79a] Baecker, Ronald, Buxton, William, and Reeves, William "Towards Facilitating Graphical Interaction: Some Examples from Computer-Aided Musical Composition", Proc. of 6th Man-computer Communications Conference, Ottawa CA, May 29-30, 1979 Musical notation recognition on digitizer [Baecker79b] Baecker, Ronald "Digital Video Display System and Dynamic Graphics", Computer Graphics, Vol 13 No 2, August 1979 Advantage of raster graphics terminal (e.g. video/TV) over random scan terminals (vector graphics): digital representation of image stored in computer, not drawn in storage scope on display. [Baecker80a] Baecker, R. "Towards an Effective Characterization of Graphical Interaction", in Methodology of Interaction, Guedj et al, editors, North-Holland Publishing Company, 1980, pp 127-ff User-interface: best user interface is device dependent, not device independent. Broad list of abstract qualities (type of feedback, etc.) of good user interfaces. Gesture-recognition for musical scores input. [Baecker80b] Baecker, Ronald "Human-computer interactive systems: A State-of-the-art review", in Processing of Visible Language 2, Kolers, Paul A., Wrolstad, Merald E., and Bouma, Herman, editors, Plenum Press, New York and London, 1980, pp 423-443 User-interface review: trainable handwriting character recognizers: recognizer must be accurate and rapid Gesture-recognition for musical scores input [BakerJ80] Baker, Janet MacIver, Dialog Systems Inc. "Brief Status Summary for Automatic Speech Recognition at the start of the 80's", SAE Technical Paper Series 800195, Soc. of Automotive Engineers, 400 Commonwealth Drive, Warrendale, Pennsylvania 15096 Report on speech recognition technologies, more than vendors and commercial systems, by founder of Dialog System (later Dragon) founder Janet Baker. Reports on *independent* testing of continuous speech / limited vocabulary system for digits and control words. [BarnardP76a] Barnard, P. and Wright, P. "The Effects of Spaced Character Formats on the Production and Legibility of Handwritten Names", Ergonomics, Vol 19 No 1, Jan 1976, pp 81-92 (re box-less) wider character spacing makes stuff less readable Boxed input should match normal writing size (handwriting constraints) [BarrowHG77] Barrow, H.G., Tenenbaum, J.M., Bolles, R.C., and Wolf, H.C. "Parametric correspondence and chamfer matching: Two new techniques for image matching", Proc. 5th Int'l. Joint Conf. on Artificial Intelligence, August 1977, pp 659-663. Also SRI International Technical Note 153 Cited in Burr81 Approximate matching/recognition for visual images of a landscape against a perspective/point-of-view generated image from landscape (topographic?) data, without requiring camera calibration in advance. Compare with structure-through-motion? [Bartram78] Bartram, D.J. "Post-iconic visual storage: Chunking in the reproduction of briefly displayed visual patterns", Cognitive Psychology, Vol 10, 1978, pp 324-355 (abstract only) Cited in IchikawaS84 Subjects presented (flashed) sets of disks: remembered by groups/chunks. Is this



where Buxton got his "chunking" ideas? [BeckerP77] Becker, Peter W. "Recognition of Patterns Using the Frequencies of Occurrence of Binary Words", Springer Verlag, Wien and New York, second revised edition first edition 1968, previously doctoral thesis: no physical copy, citation in IEEE. Use of spelling dictionary (with word frequency) for character/handwriting recognition. [BellDE76a] Bell, D.E. and La Padula, L.J. "Secure Computer System Unified Exposition and Multics Interpretation", Mitre Corporation, technical report ESD-TR-75-306 / MTR-2997 Rev.1, March 1976 Bell-La Padula security model: based on security levels (MAC mandatory access controls) of Dept. of Defense. Part of Multic security analysis. read/write/execute/delete manners of use / access modes. [BernsteinMI76] Bernstein, M.I. "Interactive Systems Research: Interim Report to the Director, Advanced Research Projects Agency, for the Period 16 September 1975 to 15 March", 1976, System Development Corporation, Santa Monica, California, Report No SDC-TM-5243/005/00, 15 April 1976 NTIS citation index Speech understanding/recognition system [Berson77] Berson, Tom "Dynamic Handwriting Recognition by Computer", Ph.D. Thesis, University of London, 1977 [Berthod79] Berthod, M. and Maroy, J.P. "Learning in Syntactic Recognition of Symbols Drawn on a Graphic Tablet", Computer Graphics and Image Processing, Vol 9, 1979, pp 166-182 The main problem in handwriting recognition is wide range of variability Big practical problem of on-line character recognition (vs OCR) is large deformations in handwriting Statistical approaches: complex classifying scheme and meaningless features Problem of statistical recognition is features do not correspond to any visual entity Says on-line recognition should allow added new symbols (writable icons for commands) (?) Character recognition needs techniques adapted to characters, not perception of drawings in general Character recognition should use human-meaningful concepts "chain codes" of straight line, plus/minus curve, pen-lift, cusp Two-level chain code, decision tree like our BLRTs, nodes (but adaptive?) Gives a grammar of permissible sequences for a sort of chain code [Berthod80] Berthod, M. and Ahyan, S. "On line cursive script recognition: A structural approach with learning", Proceedings of the 5th International Conference on Pattern Recognition, Miami Beach, Florida, December, 1980, pp 723-725 Cited in Bozonivic89 Cited in Bozinovic89 as doing adaptive/training handwriting recognition, but with syntax of letter formation and writing dynamics for strokes [BestRM80a] Best, Robert M. "Preventing software piracy with crypto-microprocessors", Proc. IEEE Spring COMPCON '80, San Francisco Feb 25..28, 1980 pp. 466..469 Early paper on blocking software piracy by encrypting software as distributed, or other media on videodisks, decryptec by secure system (trusted system) processor. Decoding is one instruction at a time (which makes decoding easier to hack), secure hardware. Cited in Tyger/Dyad for secure co-processors. [Bezdek77] Bezdek, J.C. and Castelaz, P. "Prototype classification and feature selection with fuzzy sets", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-7 No 2, 1977, pp 87-92 Cited in Biswas81 [Bjorklund77] Bjorklund, C.M. "Syntactic analysis and description of stroke-based shapes", Proceedings of IEEE Pattern Recognition and Image Processing Conference, Troy, New York, June 6-8, 1977, pp 198-202 [BlesserB76a] Blesser, B., Kuklinski, T.T., and Shillman, R.J. "Empirical Tests for Feature Selection Based on a Psychological Theory of Character Recognition", Pattern Recognition, Vol 8, pp 77-85, 1976 How we pick functional attributes: compare with other phenomenological articles, confusion matrix. [BobickTW78a] Bobick, Thomas W. "Capacitive touch-activated transducer system including a plurality of oscillators", United States Patent 4,103,252, July 25, 1978 Capacitive touchpad/arrow keys using four electrodes, finger proximity changes timing of RC oscilator circuits in each electrode. [Boldridge77] Boldridge, A.G. and Freund, R.W. "Personal Identification Apparatus", United States Patent 4,035,768, assigned to Veripen Incorporated, New York, New York, July 12, 1977 Signature verification using pressure (actually, tip force) [Bolt80] Bolt, R.A. "'Put-That-There': Voice and Gesture at the Graphics Interface", SIGGRAPH '80 Proceedings, published as Computer Graphics, Vol 14 No 3, July 1980, pp 262-270 User interface combining pointing, hand gestures, and voice recognition. See also Schmandt 1980 thesis [BorningA79] Borning, Alan "ThingLab -- A Constraint-Oriented Simulation Laboratory", Ph.D. thesis, CS Dept. Stanford University, published as SSL-79-3 July 1979 editing of graphical drawings/electronic ink, no gesture recognition [Bouchard80] Bouchard, D.C. and Toussaint, G.T. "Heuristic search methods for efficient use of dictionary information in text recognition", School of Computer Science, McGill University, Technical Report SOCS 80.5, May, 1980 Cited in HullJJ83



[Brayton79] Brayton, Robert K. "Character Recognition System and Method Multi-Bit Curve Vector Processing", United States Patent 4,177,448, assigned to IBM Corp, Armonk, New York, December 4, 1979 Setting up character recognition based on large character training set Shapes are canonicalized to standardized/normalized pieces of curves/segments [Bresenham77] Bresenham, J.E. "A Linear Algorithm for Incremental Digital Display of Circular Arcs", CACM, Vol 20 No 2, February 1977, pp 100-106 Circle-drawing algorithm on bit-mapped displays: cites algorithms for other curves [Bridle79] Bridle, J.S. and Brown, M.D. "Connected word recognition using whole word templates", Proceedings of the Institute for Acoustics, 1979, pp 25-28 Cited in Kruskal83 [Briem79] Briem, G.S.E. "Wanted: Handwriting That Fits Modern Pens", Visible Language, Vol XIII No 1, 1979, pp 50-62 Background on handwriting variability effects from stylus design, for physical writing pen instrument, e.g. nib pen versus ballpoint, etc. [BrownMD76a] Brown, Maxine D. and Smoliar, Stephen W. "A graphics editor for labanotation", ACM SIGGRAPH Computer Graphics Newsletter, Vol 10 Issue 2, Summer 1976, pp. 60-65 Graphical editor user interface for labanotation -- written notation for dance movements. Prototype used keyboard and graphical terminal only, plans to add tablet input (with gestures?) [BrownMK80a] Brown, M.K and Rabiner, Lawrence R. "An Adaptive, ordered, graph search technique for dynamic time warping for isolated word recognition", IEEE Trans. Acoustics, Speech and Signal Processing, Vol 30, 1982, pp 535-544 Cited in Schomaker86 [BrownMK80b] Brown, M.K. and Ganapathy, S. "Cursive Script Recognition", 5th International Conference on Character Recognition, 1980, pp 47-51 Cursive script recognition: uses training data-base intentionally with lots of variability [BurkeB76] Burke, Barbara "Reading Writing", M.I.T. Reports on Research, Vol 4 No 3, November 1976 Report on work of Shillman, Blesser, and Kuklinski [Burr79] Burr, D.J. "A technique for comparing curves", Proceedings of the IEEE Conference on Pattern Recognition and Image Processing, Chicago, 1979, IEEE New York, pp 271-277 Cited in Kruskal83 [Buxton79] Buxton, William; Sniderman, Richard; Reeves, William; Patel, S.; and Baecker, Ronald "The evolution of the SSP score editing tools", Computer Music Journal, Vol 3, 1979, pp 14-25 Reprinted in "Foundations of Computer Music", MIT Press, Cambridge MA 1985, pp 376-402 music-editing notation, single/uni-stroke handwriting character recognition Shows other GUIs for music input and editing other than recognition Cited in Wolf87a [Calvert80] Calvert, T.W., Chapman, J., and Patla, A "The Integration of Subjective and Objective Data in the Animation of Human Movement", Conference Proceedings: ACM 0-89791-021-4/80/0700-0198 Labanotation: dance gestures [Canada78] CSA "Numeric Character Set for Hand-printing", Canadian Standards Association, Standard Z243.34.1M1978, February 1978 Canadian equivalent of ANSI74 hand-print standard to constrain handwriting for OCR [Carau78] Carau, F.P. "Easy-to-use, High-Resolution Digitizer Increases Operator Efficiency", Hewlett-Packard Journal, December 1978, pp 2-13 Electrostatic Digitizer measuring X and Y sequentially in time causes line to bow due to velocity-related errors. Gives first-order correction. Numerous descriptions of other digitizer position errors, such as nonorthogonal axes, and how to correct for them. [CardSK78a] Card, Stuard K.; English, William K.; and Burr, Betty J. "Evaluation of Mouse, Rate-Controlled Isometric Joystick, Step Keys, and Text Keys for Text Selection on a CRT", Ergonomics, Vol 21 No 8, 1978 pp. 601-613 Mouse faster for text pointing than joystick, arrow keys, or keys. Cites other work on Grafacon tablet, not included in this study. [Carvey76] Carvey, P.C. "Electrographic System", United States Patent 3,975,592, assigned to Applicon, Incorporated, Burlington, Massachusetts, August 17, 1976 Digitizer using coarse, then fine scanning [CatmullE78a] Catmull, Edwin "A Hidden-Surface Algorithm with Anti-Aliasing", Proc. SIGGRAPH '78 5th annual conf. on Computer graphics and interactive techniques, pp. 6..11 Early paper related to alpha blending, applied to anti-aliasing in computer graphics. [Cederberg78] Cederberg, R.L.T. "An iterative algorithm for angle detection on digital curves", Proceedings of 4th International Joint Conference on Pattern Recognition, Kyoto, Japan, November 7-10, 1978, pp 576-578 [ChangP76] Chang, P. "Efficient Keyboard Layouts for Inputting Chinese Characters", B.S. Thesis, Department of



Electrical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts May 1976 117 confusions/ambiguous character in 10000 defined Chinese/Kanji even if every basic stroke has a unique keyboard stroke (due to positions of strokes?) [ChangS77] Chang, S. and Nagy, G. "Deposit-Slip-First Check Reading", IEEE Trans. Systems, Man, and Cybernetics, January 1977, pp 64-68 Operations in OCR: bank can control deposit slip format, but not checks [ChaoYM77a] Chao, Yao-Ming "Recognition of handwriting Chinese Characters", 6.362 Student Paper, M.I.T., Fall 1977 direction codes / chain-codes, Chinese characters recognition [ChenCH77a] Chen, C.H. "Statistical Pattern Recognition - Review and Outlook", Systems, Man, and Cybernetics Review, Vol 6 No 4, August, 1977, pp 7-8 Feature extraction remains key problem in pattern recognition: features must be based on problem at hand Context aids statistical pattern recognition [Chinnuswamy80] Chinnuswamy, P. "Recognition of handprinted Tamil characters", Pattern Recognition, Vol 12, 1980, pp 141-152 Cited in Loy82 on Indian handwriting script (Tamil) [ClarkJH76a] Clark, James H. "Designing Surfaces in 3-D", CACM,Vaol 19 No 8, August, 1976, pp. 454-460 Input of 3-D surfaces using control points and spline fitting for surfaces. Three-dimensional digitizer is electromechanical: wand with three filament lines attached to end, sensors read how much line is paid out in for all lines and computes X/Y/Z position. Head-mounted stereoscopic display. Cites to Lincoln Wand. [CoffinS78a] Coffin, Stephen "Spatial frequency analysis of block letters does not predict experimental confusions", Perception and Psychophysics, Vol 23 No 1, 1978, pp 69-74 Human recognition of letters uses features, not mathematics: Fourier spatial frequency Refers to edge and line detection/recognition in human vision, then Position as something different from low-level features Criticizes studies of human recognition that had visual distraction [Computerwoche77] Computerwoche "Online-DE mit dem Bleistift: Datenerfassung als Nebensache", Computerwoche 21/1977, May 20, 1977 Quest Automation Datapad "Datapad puts your pencil Online to a computer". Dorset U.K. Handwriting recognition input on a paper form, 32-character LED display, uses Nova 16-bit minicomputer as controller. Statement is that data entry is done just on the side automatically while person is filling out regular paper form. [Computerwoche80a] Computerwoche "Quest Automation Ltd.: Handschriftleser für Direkterfassung", Computerwoche 16/1980, April 18, 1980 Quest Automation Datapad/Micropad, newer version of Datapad: 40-character display. Says less than one hour training required. Reference to booth at CeBIT exposition. [Computerwoche80b] Computerwoche "Statt über Keyboard mit der Hand: Micropad gibt Sauklaue ein", Computerwoche 10/1980, March 7, 1980 Quest Automation Datapad/Micropad handwriting tablet / keyboard replacement. [CooperLN76] Cooper, Leon N., and Elbaum, Charles "Information Processing System", United States Patent 3,950,733, assigned to Nestor Associates, New York, New York, April 13, 1976 Nestor patent: neural net, adaptive memory [CooperLN77] Cooper, Leon N., and Elbaum, Charles "Information Processing System", United States Patent 4,044,243, August 23, 1977, assigned to Nestor Associates, Stonington, Connecticut Nestor Graphics patent [CoxCH76] Cox, C.H. III and Coueignoux, P. "Concise Letter/Type Font Description: Theory and Computer Implementation", Internal report, Massachusetts Institute of Technology Industrial Liaison Program, Department of Electrical Engineering and Computer Science, November 20, 1976 OCR variability based on thinned "skeletons", generatively similar to chain-codes. Rules for line thinning, seraphs for typed characters. [CoxCH78] Cox, C., Blesser, B. and Eden, M. "Graphical Context of Printed Characters", Visible Language, Vol XII No 4, Autumn 1978, pp 428-447 Generative variability: points out existence of "rules" for how to write, even if not conscious [Crane77a] Crane, H.D., and Savoie, R.E. "An On-Line Data Entry System for Hand-Printed Characters", IEEE Computer, pp 43-50, March 1977 On-line handwriting/shape recognition: Uses ULDR chain codes, similar to BLRT chain codes: accelerometer pen? [Crane77b] Crane, H.D., Ostrem, J.S., Wolf, D.E., and Wang, T.N.C. "A Technique for the Input of Handprinted Chinese Characters Based on Sequential Stroke Recognition", Proceedings of International Computer Symposium 1977, Taipei, Republic of China, Vol 1, pp 246-261 Describe drag force, not acceleration, of SRI pen Kanji Chinese has Gaussian distribution of number of



strokes, with 11 average 881 characters in basic Kanji defined by Japanese ministry of education Problem focuses on confusions, since did not have a large enough data base to get valid statistical results on recognition accuracy [Crane77c] Crane, H.D, Wolf, D.E., and Ostrem, J.S. "The SRI pen system for automatic signature verification", Symposium Proceedings NBS Trends and Applications 1977, Gaithersburg, May 1977, pp 32-39 Liu,CN79 Obscured tip on SRI pen SRI dynamic pen design: see Crane75. [CraneHD79a] Crane, H.D. and Wolf, D.E. "Dynamic Re-creation of Signatures", United States Patent 4,156,911, assigned to Stanford Research Institute, Menlo Park, California, May 29, 1979 Signature verification by human eyeball: patent on compressing signature to minimum data without losing detail Use of local extrema/chain codes to store written signatures with minimum memory [CraneHD79b] Crane, H.D. and Wolf, D.E. "Handwritten signature verification system", United States Patent 4,086,567, April 25, 1978, assigned to Stanford Research Institute, Menlo Park, California Signature verification by looking for landmarks (pen lifts) Hew Crane: compressed-data method of storing signatures for human reading [CraneHD80a] Crane, H.D. and Wolf, D.E. "Signal Train Verification System Using Landmarks", United States Patent 4,190,820, February 26, 1980, assigned to Stanford Research Institute, Menlo Park, California Elastic/rubbery matching of signature to segment into sections to be stretched or contracted for template matching Landmarks in signatures which are invariant features [CranmerTV77a] Cranmer, Terrence, V., Mary "Braille display switching matrix and circuit utilizing the same", United State Patent 4,037,200, July 19, 1977 Braille clock display with (fixed) braille dots on insulated conductive pins. Braille characters in vertical columns. Each pin is a touch-controllable non-mechanical switch (by short circuit to surrounding touchstrate: compare with resistive touchscreen) for a set of the terminals single braille character). multitouch? [Crawshaw77] Crawshaw, Martin, and Ottaway, Mary "A Contact-Pencil for Research on Writing", Quarterly Journal of Experimental Psychology, Vol 29, 1977 pp 345-346 Stylus using graphite/pencil lead to sense pen down/up. [CrowFC77a] Crow, Franklin C. "Shadow Algorithsm for Computer Graphics", Proc. SIGGRAPH '77, July 20-22 1977, San Jose, pp. 242-248 Computational techniques for simulated shadows in virtual images: silhouette edges of polygonization. Does not mention ray-tracing. [CrowleyMillingM77a] Crowley-Milling, Michael "How CERN broke the software barrier", New Scientist, 29 September 1977, pp. 790-791 CERN computer science innovations born of necessity: interpretive language with interactive debugging with real-time control, virtual machines matched well to multiple and distributed computers, invention of touch screen with structured menus -- now available commercially -- and "rolling ball" (trackball). [DIN77] Deutsches Institut für Normung e.V. "Schrift B für die maschinelle optische Zeichenerkennung", DIN 66 009, September, 1977 German standard for OCR readable handwritten characters, similar to ANSI standard, but lists several as "not intended for OCR" (too hard to recognize?) [Dasarathy78] Dasarathy, B.V. and Kumar, K.P.B. "Chitra: cognitive handprinted input-trained recursively analyzing system for recognition of alphanumeric characters", International Journal of Computing and Information Science, Vol 7, 1978, pp 253-282 Cited in Li89 [DavisLS76] Davis, Larry S. and Rosenfeld, A. "Applications of relaxation labeling: Spring-loaded template matching", Proceedings of 3rd International Joint Conference on Pattern Recognition, 1976, pp 591-597 Davis,LS77 [DavisLS77] Davis, Larry S. "Understanding shape: Angles and sizes", IEEE Trans. Computers, Vol C-26, March 1977, pp 236-242 Cited in Teh89 [DavisLS79] Davis, Larry S. "Shape Matching using Relaxation Techniques", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-1 No 1, January 1979, pp 60-72 Relaxation process for matching shapes (coastlines) by figures of merits on pairs of angles (segment matching) Cited in Baird84: on template matching in feature space using minimum distance [DavisRL77a] Davis, Robert L and Howells, Joseph A. "Position Determining Apparatus and Transducer Therefore", United States Patent 4,012,588, March 15, 1977 Acoustic digitizer using multiple pick-up "sonar" microphones from sound transducer on stylus: SAC Science Accessories Corporation [DiffieW76a] Diffie, Whitfield and Hellman, Martin E. "New Directions in Cryptography", IEEE Trans. on Information Theory, Vol IT-22, No 6, November 1976, pp. 644..654



General description of public key cryptography, cryptographic (public key) signatures. Authentication by centrally-stored Public File, cryptographycially signed. [DiffieW76b] Diffie, Whitfield and Hellman, Martin E. "Multiuser cryptographic techiques", Proc. NCC, 1976, pp 109..112 General description of public key cryptography/authentication. Private conversations between any two individuals without central key management. Digital signature has non-repudiation (by signer), and unforgeable (by recepient). [Donahey76] Donahey, Alvin V. "Character Recognition System and Method", United States Patent 3,996,557, December 7, 1976, assigned to MI-squared Corporation, Columbus, Ohio On-line character recognition using vertical conductive bar zones/region Says does script, but it is not connected writing script, only for separate/discrete characters. [DonelsonWC77a] Donelson, William C. "Spatial Management of Data", M.S. Thesis, Massachusetts Institute of Technology, September 1977, Department of Architecture See Donelson78 [DonelsonWC78a] Donelson, William C. "Spatial Management of Information", ACM SigGraph Conference, 1978, pp 203-209 Gesture (hand-gesture?) user interface for viewing data using large-screen display. Summagraphics magnetostrictive digitizer tablet with stylus for writing annotation (electronic ink?), Joypad touchscreen (resistive tablet) for user input? Both verbal and electronic ink annotations: how edit? See also Donelson Master's thesis. [DosterW77] Doster, W. "Contextual postprocessing system for cooperation with a multiple-choice character recognition system", IEEE Trans. Computers, Vol C-26, 1977, pp 1090-1101 Spelling dictionary look-up for context correction: in separate character OCR (handwriting) Multiple-Choice Single Character Recognition System and Contextual Postprocessing System [DosterW80] Doster, W. and Schuermann, J. "An application of the modified Viterbi-algorithm used in text recognition", Proceedings of 5th International Conference on Pattern Recognition, 1980, pp 853-855 In Tappert's bibliography [Driscoll77] Driscoll, R.J. et al "Real-Time Reading of Handwritten Symbols and Applications", Proceedings of the Conference eon Digital Processing of Signals in Communications, September 9, 1977, Institute of Electronic and Radio Engineers, pp 293-302 Cited in Sklarew patents [Druse78] Druse, B. and Rao, C.V.K. "A matched filtering technique for corner detection", Proceedings of the 4th International Joint Conference on Pattern Recognition, Kyoto, Japan, November 7-10, 1978, pp 642-644 [Du80] Du, H.C. and Lee, R.C.T. "Symbolic Gray Code as a Multikey Hashing Function", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-2 No 1, pp 83-90, January 1980 Very fast searching using keys (brute-force engineering) [Duerr80] Duerr, B., Haettich, W., Troph, H., and Winkler, G. "A combination of statistical and syntactical pattern recognition applied to classification of unconstrained handwritten numerals", Pattern Recognition, Vol 12, 1980, pp 189-199 Cited in Kerrick88 [Dumas77] Dumas, G. "A Dual Microprocessor Applications: The Alphabec-75 Data Capture System", in "Microcomputer Design and Applications", Academic Press, New York/San Francisco/London, 1977, pp 301-329 Mentions user learning/adaptation/training times and success rate: some learned fast, some never did (!) Xebec/SRI/Crane original character recognition pen product using chain codes and an accelerometer in the writing pen [DunnRankin78] Dunn-Rankin, Peter "The Visual Characteristics of Words", Scientific American, January 1978, pp 122-130 Word recognition in humans (graphical word context). Other Dunn-Rankin papers describe inter-letter confusion in human perception of English alphabet: compare with Functional Attributes of Shillman [Duvernoy79] Duvernoy, J. and Charraut, D. "Stability and stationarity of cursive handwriting", Pattern Recognition, Vol 11, 1979, pp 145-154 In Tappert's bibliography Handwriting variability in cursive script [DymH78a] Dym, Herbert and Krakinowski, Morris "Capacitive two dimensional tablet with single conductive layer", United States Patent 4,087,625, May 2, 1978 Charge-ratio capacitive/electrostatic tablet, triangular electrodes coming in from two sides. Measures capacitive coupling with AC signal. [EachusJJ77a] Eachus, Joseph J. and Graff, Theodore S. "Touch Sensitive Device", United States Patent 4,055,735, October 25, 1977 Touch-button (?) panel/tablet with grid of pressure sensitive material on two sides of a printed circuit board substrate. Cross-wire construction: compare with Kaplow?



[EerNisseEP77a] EerNisse, E.P., et al "Piezoelectric Sensor Pen for Dynamic Signature Verification", Conference 1988 Int. Electron Devices Meeting, Washington, D.C., December 1977, pp 473-476 Cited in Chainer85a Signature verification using pen/stylus with piezoelectric pressure/force sensors for writing pressure, also lateral shear force [EerNisseEP78a] EerNisse, E.P., Land, C.E., and Snelling, J.B. "Input Apparatus for Dynamic Signature Verification Systems", United States Patent 4,078,226, assigned to the United States of America, March 7, 1978 Signature verification stylus pen using piezoelectric force/pressure sensors to sense X and Y forces/acceleration [EhrichRW76] Ehrich, Roger W. "A Readily Computable Decision Rule with Variable Dimensionality", IEEE Trans. Computers, May, 1976, pp 539-542 Bayes and Meyman-Pearson statistical classifiers to select likelihood ratios Features statistically best are generally "specialists" in distinguishing that class [EhrichRW78] Ehrich, Roger W. "Handwriting Recognition", Encyclopedia of Computer Science, Vol 9, 1978, Belzer, Holzman and Kent, editors, Dekker, 1978, pp 180-198 [ElectronicsAndPower79a] Electronics and Power "A British world first in data entry - Micropad Ltd., Quest Automation Lt.", Electronics and Power (UK), October 1979, p. 685 Quest Micropad / Datapad handwriting terminal announcements -- see Micropad files. [Elographics80] Gibson, W., Talmage, J. "Nonplanar Transparent Electrographic Sensor", United States Patent 4,220,815, September 2, 1980, assigned to Elographics, Incorporated, Oak Ridge, Tennessee Resistive-sheet/film digitizer using small spacing dots between a conductive and a resistive layer: Elographics [Engdahl77] Engdahl, Jean "Data Entry and Decoding System for Scripted Data", United States Patent 4,005,400, January 25, 1977, assigned to Societe Suisse pour l'Industrie Horologere Management Services S.A., Bienne, Switzerland Four-segment panel for character recognition, zone-based recognition for handwriting, using a stylus [EngelbrechtRS76a] Engelbrecht, R.S. "Handwriting Identification Technique", United States Patent 3,962,679, June 8, 1976, assigned to RCA Corporation, New York, New York Signature verification using writing speed/velocity [ErmanLD80a] Erman, Lee D.; Hayes-Roth, Frederick; Lesser, Victor R.; and Reddy, D. Raj "The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty", ACM Computing Surveys, Vol 12 No2, June 1980 pp. 213-253 speech-recognition/speech-understanding system, review of similar systems. Solution crosses multiple problem domains: phoneme detection, word segmentation, vocabulary, grammar, problemdomain/discourse semantics. Describes technique as "blackboard" system. [Farag79] Farag, R.F.H. "Word-Level Recognition of Cursive Script", IEEE Trans. Computers, Vol C-28 No 2, February 1979, pp 172-175 Sketch recognition of whole script words vs a table/word dictionary for context Testing involved only a very small sample (ten words) [FeldmanSI79] Feldman, S.I. "Make -- a Program for Maintaining Computer Programs", Software Practice and Experience, Vol 9 No 4, pp. 255-265, 1979 Overview of Unix MAKE build tool with dependencies [FenclGA80a] Fencl, George A. "Low Noise System and Method for Sequentially Sensing Induced Signals in digitizer Grid Conductors", United States Patent 4,185,165, January 22, 1980 Electromagnetic tablet digitizer: states that conductive marks (?) on form can affect performance [Feucht77] Feucht, Dennis "Pattern Recognition: Basic Concepts and Implementations", Computer Design, December 1977, pp 57-68 Overview of pattern recognition: decision functions, multicategory classifiers (type I: simple, type II: pairwise, type III: extended to all classes), hyperplanes and weight vectors in feature space, minimumdistance classifiers Clustering techniques: maximin-distance, K-Means algorithms, general cluster-seeking algorithm. Learning-adaptive recognition: perceptron, absolute increment, multicategory perceptron. [FisherE76] Fisher, E. "The use of context in character recognition", COINS Technical Report 76-12 (Ph.D. thesis, Computer and Information Sciences Department, University of Massachusetts, Amherst), 1976 Cited in DosterW77 on spelling correction dictionary? Morse code recognition [Fitzgerald77] Fitzgerald, William Joseph, Shelton Jr., Glenmore Lorraine, and Wolfe, Robert Nolan "System for converting a rough sketch to a finished drawing", United States Patent 4,058,849, assigned to International Business Machines Corp, New York, New York, November 15, 1977 Editing of handwritten sketches by making the lines more nearly vertical and horizontal, and moving the endpoints of the lines: preserves linkages among strokes Also shows use of an overlay keyboard, presenting a virtual keyboard on the tablet surface Cites Eichen73 patent, and "Instant Draftsman" article from 1972



[FoleyJD80a] Foley, James D; Chan, Peggy; and Wallace, Victor L. "The Human Factors of Graphic Iteration: Tasks and Techniques", Tech. Report 508, US Army Research Institute for the Behavioral and Social Sciences, December 1980 Study of user interface input devices for graphical applications: light pen, buttons/keyboard, chord keyboard, speech recognizer, tablet, mouse, joystick. Tablet most frequently used device for pointing input (1980). [Foster80] Foster, D.H. and Mason, R.J. "Irrelevance of local position information in visual adaptation to random arrays of small geometric elements", Perception, Vol 9, 1980, pp 217-221 (BLRT chain codes) relative position not as significant as local features in human recognition [FranklinRC78a] Franklin, Robert C. and Fuller, Rank I. "Electronic Wall Stud Sensor", United States Patent 4,099,118, July 4, 1978 Interesting example of electrostatic/capacitive sensing: compare with electrostatic touchscreens and digitizers. Detects change in dielectric constant in proximity, rather than higher or lower dielectric constant [Fraser80] Fraser, A. "Comments on 'A Study of Man-Machine Interaction Problems in Character Recognition'", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-10 No 9, September 1980, page 589 Says Suen is full of it Says variability in writing styles is caused by deep, fundamental human individuality Points out "obvious" testing bias in Suen's paper A graphologist speaks out on graphonomics handwriting research Significant comment on dynamic character recognition by a graphologist "neat", constrained handwriting is not realistic [Freeman77] Freeman, H., and Davis, L.S. "A Corner-Finding Algorithm for Chain-Coded Curves", IEEE Trans. Computers, Vol C-26, pp 297-303, March 1977 Adds "corner" detection to chain codes Source of Barry's original "hi-curve" algorithm for corners? [Freeman78a] Freeman, H. "Application of the generalized coding scheme to map data processing", Proceedings of IEEE Pattern Recognition and Image Processing Conference, Chicago, Illinois, May 31-June 2, 1978, pp 220226 [Freeman78b] Freeman, H. and Saghri, A. "Generalized chain codes for planar curves", Proceedings of the 4th International Joint Conference on Pattern Recognition, Kyoto, Japan, November 7-10, 1978, pp 701-703 [Frey77] Frey, P.W. "Chess Skill in Man and Machine", Springer Verlag, New York, 1977 Survey of artificial intelligence ideas in computer chess [Fu77] Fu, K.S. "A Brief Review of Pattern Recognition", Systems, Man, and Cybernetics Review, Vol 6 No 4, August, 1977, pp 3-5 Equivalence of decision-theoretic/statistical and syntactic pattern recognition [Fu80] Fu, K.S. "Recent development in pattern recognition", IEEE Trans. Computers, Vol C-29, October, 1980 Cited in Fu86/Fu83 [FujimotoY76a] Fujimoto, Y., Kadota, S., Hayashi, S., Yamamoto, M., Yajima, S. and Yasuda, M. "Recognition of Hand-printed Characters by Nonlinear elastic Matching", Proceedings of the 3rd International Joint Conference on Pattern Recognition, November 1976, pp 113-118 Quotes Blesser, but says they never built an actual system Uses eight-direction vector/length chain codes for elastic matching Compare to Tappert's paper on elastic matching Goshtasby88 cites that this system used continuity (continuation marks and hooks between characters) to get 99.7% recognition accuracy [FujitaT76a] Fujita, T., Nakanishi, M., and Miyata, K. "The recognition of Chinese characters (kanji) using time variation of peripheral belt patterns", Proceedings of 3rd International Joint Conference on Pattern Recognition, Coronado, California, November 1976, pp 119-121 [Gaertner80] Gaertner, K.P. and Holzhausen, K.P. "Controlling air traffic with a touch sensitive screen", Applied Ergonomics, Vol 11 No 1, 1980, p 17-22 Cited in Beringer89: user-interface on electronic ink, integrated tablet/display. A clamp-on mechanical digitizer [Genesereth79] Genesereth, M.R. "The Use of Semantics in a Tablet-Based Program for Selecting Parts of Mathematical Expressions", MACSYMA User's Conference, 1979. V.E. Lewis, Ed. Citation by Soiffer Kajler on citeseer.nj.nec.com/context/906109/0 mentions use of GUI for circling parts of mathematical expressions, using semantic information [Geographics80] GeoGraphics "GeoGraphics Drafting Board Digitizer", GEOGRAPHICS Incorporated, 1318 Alms Drive, Champaign Ill. 61820, 1986 Drafting board digitizer: usable for whiteboard? [GilmoreGC79a] Gilmore, G.C., Hersh, H., Caramazza, A. and Griffin, J. "Multidimensional letter similarity derived from recognition errors", Perception and Psychophysics, Vol 25, 1979, pp 425-431 Experimental study of confusion matrix with human subjects of pairs of characters, and which features appear to be most relevant to confusion. Compare with Shillman phenomenological attributes? Only suggest results as a guide to picking similar characters, not for determining distinguishing features. Cited in IchikawaS84



[Gould78] Gould, J.D. "How experts dictate", Journal of Experimental Psychology: Human Perception and Performance, Vol 4, 1978, pp 648-661 Cited in Martin89 [Greenaway78] Greenaway, David Leslie "Apparatus and Method for Producing an Electrical Signal Responsive to Handwriting Characteristics", United States Patent 4,122,433, assigned to LGZ Landis and GYR Zug Ag, Zug, Switzerland, October 24, 1978 Pen stylus moves across a ridged/quadrangled surface while writing: resulting vibrations give and indication of pressure force and writing velocity [Greenberg77] Greenberg, M.E. "An investigation of handwritten numeral character recognition", M.S. Thesis, Department of Electrical Engineering and Computer Science, August 1977, Massachusetts Institute of Technology, Cambridge, Massachusetts Throws out data from IEEE handwriting variability Database 1.2.4 as "poorly written" [GroenFCA78a] Groen, F.C.A. and Verbeek, P.W. "Freeman-code probabilities of object boundary quantized contours", Computer Vision, Graphics, and Image Processing, Vol 7, 1978, pp 391-402 Representation of contours (of shapes) using eight-direction Freeman chain codes. Cited in Teh89 [Grossman78] Grossman, Robert "Hybrid pressure/force transducers give robots a sense of touch", EDN, August 5, 1978, pp 29-31 [GroverDJ79] Grover, D.J. "Graphics Tablets - A Review", Displays, July 1979, pp 83-93, IPC Business Press, Great Britain Attempt at complete classification of methods of graphical/tablet interaction. Broad overview of digitizer tablet technologies. [GrummerWL80a] Grummer, William L. and Hendriks, Hendrik B. "Capacitive Touch Switch Array", United States Patent 4,233,522, November 11, 1980 Capacitive touchscreen using grid/array of capacitive sensors. [Guberman76a] Guberman, S. and Litvin, Y. "Implementation of an Algorithm for Recognition of Handwritten Letters", Institute of Applied Mathematics, Academy of Sciences, USSR, TR-22, Moscow USSR (in Russian), 1976 Cited in Goodale83 [Guberman76b] Guberman, S. and Rozentsveig, V. "An Algorithm for Reading Handwritten Letters", (Automation and Remote Control), Vol 37, pp 751 ff Cited in Goodale83 [Haberman76] Haberman, W. and Fejfar, A. "Automatic Identification of Personnel through Speaker and Signature Verification -- System Description and Testing", Proceedings of Carnahan Conference on Crime Countermeasures, May 1976, pp 23-30 [HallPAV80a] Hall, P.A.V. and Dowling, G.R. "Approximate string matching", Computing Surveys, Vol 12 no 4, December 1980, pp 381-402 Cited in Srihari83: on context/spelling correction [Hanaki76] Hanaki, Shin-Ichi, Temma, Tsutomu and Yoshida, Hiroshi "An On-line Character Recognition Aimed at a Substitution for a Billing Machine Keyboard", Pattern Recognition, Vol 8, pp 63-71, January 1976 User-interface: early (1976) Nippon Electric prototype Katakana/Hiragana/Romanji product for data-entry: features chosen by "human intuition" Describes what appears to be oversampling, resampling, noise filter on digitizing tablet (Telemail product) Resampling of handwritten tablet points to be minimum number of feature points for reconstruction/recognition: also used for data reduction/compression of line drawings Character segmentation by stroke time-out, writing in boxes, recognition result Shows shading pattern for boxes to write in 98% recognition rate, using decision tree node Refers to handwriting character segmentation based on recognition result/plausibility [HansonAR76a] Hanson, A.R., Riseman, E.M., and Fisher, E. "Context in Word Recognition", Pattern Recognition, Vol 8, 1976, pp 35-45 (abstract only) Electronic copy is just abstract: see paper file. Low character error rates can give high word error rates. Use context on words to correct errors on single characters. "many theoretical systems are impractical in application". Cites use of "clear text" in context: but many applications are not (data entry) Tri-gram statistical Markovian context probabilities, not linguistic grammar context Cites problems of small training sets (98 characters per class) High (context) correction rates introduce many substitution errors The number of substitution errors is about the same as reject recognition Context after the recognition is done, not before [HardenJC77a] Harden, John Charles and Mastrangelo, Sebastian V.R. "Pressure-sensitive elastic resistor compositions", United States Patent 4,028,276, June 7, 1977 Pressure-sensitive elastomeric conductive rubber, one application is keyboard switches. Metallic-conductive particles in elastomer. [Hattich78] Hattich, W., Tropf, H., and Winkler, G. "Combination of statistical and syntactical pattern recognition - Applied to classification of unconstrained handwritten numerals", Proceedings of Fourth International Conference



on Pattern Recognition, Kyoto, Japan, November 7-10, 1978, pp 786-788 Cited in Kahan87 [HayesRothF77a] Hayes-Roth, Frederick and Lesser, Victor R. "Focus of Attention in the HEARSAY-II Speech Understanding System", Proc. Fifth Int'l. Joint Conf. on Artificial Intelligence, Volume 1, IJCAI, pp. 27-35. 1977 speech recognition / speech understanding system, multiple knowledge sources (phonetic, segmentation, syntax, grammar, problem-domain-specific semantics) included in blackboard of current hypothesized results and alternatives. [HerbstNM76a] Herbst, Noel M., and Morrissey, John H. "Verifier for Handwritten Signatures", United States Patent 3,983,535, assigned to IBM Corporation, September 28, 1976 Cited in Chainer85b: for acceleration on signature verification Signature verification using acceleration: acceleration/accelerometer stylus (no table), compare with Hewitt Crane Stanford acceleromter pen [HerbstNM77a] Herbst, Noel Martin and Morrissey, John Henry "Segmentation Mechanism for Cursive Script Character Recognition Systems", United States Patent 4,024,500, May 17, 1977, assigned to IBM Corporation, Armonk, New York Cursive character segmentation using sharp inflection points (zero velocity) Also makes use of baseline and midline (top of lower-case characters) for character segmentation Criticizes "retrospective" character segmentation in script, such as Harmon [HerbstNM77b] Herbst, N.M., et al "Signature Verification Based on Complete Accelerometry", IBM Technical Disclosure Bulletin, Vol 19 No 12, May 1977, pp 4827-4828 Stylus for digitizer, no tablet: uses acceleration (chain codes? compare to SRI pen) [HerbstNM78a] Herbst, N.M. and Liu, C.H. "Signature Verification Method and Apparatus Utilizing both Acceleration and Pressure Characteristics", United States Patent 4,128,829, December 5, 1978, assigned to IBM Corporation Armonk, New York Signature verification: force/pressure and acceleration. (BLRT chain codes): position features subject to most variability. Separate X and Y acceleration for signature verification. [HerbstNM79a] Herbst, N.M., Liu, C.N., and Panissidi, H.A. "Pressure sensing device and transducer arrangement", United States Patent 4,142,175, February 27, 1979, assigned to IBM Corporation Cited in Chainer85b acceleration on signature verification [HerbstNM79b] Herbst, Noel M., Liu, Chao N., and Panissidi, Hugo A. "Pressure Sensing Device and Transducer Arrangement", United States Patent 4,142,175, assigned to International Business Machines Corporation, Armonk, New York, February 27, 1979 Signature verification: pressure/force sensing stylus, not sensitive to lateral force angle: combined with acceleration for signature verification [HerotCF76a] Herot, Christopher F. "Graphical Input through Machine Recognition of Sketches", Computer Graphics, Vol 10 No 2, Summer 1976, pp 97-102, Proc. SIGGRAPH '76 Sketch recognition: segmentation of tablet input into lines and splines based on speed an pressure. Success of sketch recognition / drafting in part because system is interactive (graphical user interface) for which user may manipulate and modify recognition. Inking stylus. [HerotCF76b] Herot, Christopher F. "Sketch Recognition for Computer-Aided Design", Proc. URODIGS '76, ACM/SIGGRAPH Workshop on User-oriented Design of Interactive Graphics Systems, 1976, pp. 31-35 Sketch recognition: infer user intent (for segmentation, etc.) based on speed, sequence and pressure (tipforce) of strokes. E.g. if drawn more slowly, user probably intended those rounded corners, so do not sharpen them. Overtracings intended to lengthen mark, or make more bold. Tablet large format, has eraser sensor on rear end like a pencil (compare with Wang Freestyle tablet). [HerotCF78a] Herot, Christopher F. and Weinzapfel, Guy "One-Point Touch Input of Vector Information for Computer Displays", Proc. SIGGRAPH '78; SIGGRAPH Newsletter Vol 12 No 3, Aug 1978, pp. 210-216 Pressure-sensitive digitizer (also direction and torque-sensitive) using roughened glass surface and finger as stylus. Friction with finger not undesirable, can be used for multidimensional input. Can detect rotation of a single finger. Finger can be used to give initial push to graphical object, then steer without moving finger from position. Mentions two-finger twist/rotate gesture. Mentions multiple finger / multi-touch touchscreen under development with Elographics. Mentions kinesthetic (haptic) input devices. Compare to Artificial Reality project? [HerotCF80a] Herot, Christopher F. "Spatial Management of Data", ACM Trans. on Database Systems, Vol 5 No 4, December 1980, pp. 493-514 Graphical Data Space (GDS) with world view of data in one display (using small icons), local heirarchy context in another display, detail record in another. Early system for semantic zooming. Prototype uses joystick not mouse or digitizer. [HerroldPA79a] Herrold, P.A. "Alphanumeric CRT Display Having A Plurality Of Display Positions", IBM Technical Disclosure Bulletin TDB 09-79, p. 1338-1338, 01-Sep-1979 CRT display can be rotated 90 degrees (landscape vs. portrait), outline (frame/window borders) of display



and of text is automatically reformatted/regenerated for orientation. [HilbrinkJO78a] Hilbrink, Johan O. "System for Optically Entering, Displaying and Decoding Handwritten Symbols", United States Patent 4,122,415, September 5, 1978 Lightpen and television monitor display, when used for handwriting input and replacing the handwriting input with a generated character [Himmel78] Himmel, David P. "Some Real-World Experiences with Handprinted Optical Character Recognition", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 4, April 1978, pp 288-292 How much effort it takes to train people to handwriting constraints (OCR). Shows samples of constrained writing people were actually trained to. [HoffmanHS80a] Hoffman, H.S. "Exact Position Detection with Finger Point and Feedback", IBM Technical Disclosure Bulletin 22, 1536 (September 1979) capacitive/electrostatic touchscreen grid that can be integrated with a display: interpolation between grid lines to get fine position accuracy writing [Hollerbach78] Hollerbach, J.M. "A Study of Motor Control Through Analysis and Synthesis of Handwriting", PhD Thesis, Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science, August, 1978. Also described in Visible Language, Vol XIII No 3, 1979, pp 252-264 Study of stroke directions based on anatomy of muscles. Points out elimination of clock-wise movement in fast writing. [Hosaka77] Hosaka, M. and Kimura, F. "An Interactive Geometrical Design System with Handwriting", Proceedings of the 6th IFIPS Congress, Toronto, August 1977, pp 167-171 User-interface: handwritten sketches, sketch recognition, gesture input to do drawings Refers to measuring problems of tablet (and of "human behavior") (human factors / variability). Uses second-level fix-up on recognition if first pass is inconclusive. Uses chain codes of inflection points, rotations, straight segments, etc. [HoshinoY77a] Hoshino, Y. "Word recognition apparatus", United States Patent 4,010,445, March 1977 Cited in Suen79 Dictionary of words in character recogntion, look for entry with best/closest match, not exact match. [Hubel79] Hubel, D.H. and Wiesel, T.N. "Brain Mechanisms of Vision", Scientific American, No 241, 1979, pp 150162 Martin,GL90b [IBM76a] IBM "Electronic Calculator Based on Character Recognition of Input from Stylus Acceleration Dynamics", IBM Technical Disclosure Bulletin, December 1976, pp. 2816-2817 Accelerometer writing stylus (no table), used for calculator input. Compare to Hewitt Crane Standfor accelerator pen. [IBM79] IBM "Object Identification and Local Highlighting on Displays", IBM Technical Disclosure Bulletin, Jan 1979, p 3284 Compare with object highlighting via proximity tablet data of "GUI-handles" in a GUI: mentions getting local highlighting of an object on a display as the stylus goes over it, and returning a ID/handle to the application instead of X/Y coordinates. Mentions equivalent functions from lightpen, tablet, joystick, or other positioning (position input) device. [IchikawaT77] Ichikawa, Tadao, Sakamura, Ken, and Aiso, Hideo "ARES - A memory, capable of associating stored information through relevancy estimation", Proceedings of 1977 National Computer Conference, pp 947-954 Seven-direction chain codes/ segment lengths, with an additional approximate string matching function "Lee Distance" from error-correcting codes (in hardware) [IkedoT78] Ikedo, Tsuneo "Automatic Coordinate Determining Device", United States Patent 4,088,842, May 19, 1978 Electromagnetic digitizer tablet that can handle multiple styli (multi-touch), and a cordless/wireless probe or stylus: Can use multiple probes/styli on one device. [ImpedovoS76a] Impedovo, S., Marangelli, B., and Plantamura, V.L. "Real-Time Recognition of Handwritten Numerals", IEEE Trans. Systems, Man, and Cybernetics, Vol 6 No 2, February 1976, pp 145-148 65% to 90% recognition, numerals only (!) Gives examples of handwriting variations: stroke directions [InfoWorld80a] InfoWorld "Write to a Micro", InfoWorld Magazine, October 13, 1980, page 22 MicroPad product announcements: handwriting terminal with two line display, handwriting recognition on pre-printed paper forms. Compare with PenPad from Pencept and CIC HandWriter [IshiiM79a] Ishii, Mitsuo; Yamamoto, Masanari; Iwasaki, Michiko; and Shiraishi, Hiroshi "An experimental input system of hand-drawn logic circuit diagram for LSI CAD", DAC '79 Proc. 16th Design Automation Conference, 1979, pp. 114..120 OCR static recognition of hand-drawn CAD electronic designs/circuits. Requires very neat/constrained handwriting input: connecting lines drawn with rulers, component symbols drawn using a template. [Ito78] Ito, M.R. and Chui, T.L. "On-line Computer Recognition of Proposed Standard ANSI (USASI) Handprinted



Characters", Pattern Recognition, Vol 10, 1978, pp 341-349 Claims 98.3% accuracy on A non-training system (non-adaptive) handwriting recognition. It's not just tablet digitizer noise: unsteadiness in user's pen motion. Eight-type chain code classifications. Straight-line vs curvilinear stroke classes. Testing for performance used "intentional variability" in characters by authors: testing bias? Cites mis-recognition due to missing stroke segments (K-R, J-T). Digitizer error? Delay in recognition same as time for user to look at screen: not a user-interface/human-factors problem. Variability: Shows bizarre four-stroke "Z" shape form. Shows proposed ANSI hand-print standard for static OCR recognition including stroke directions, constrained writing styles. [JarvisJF76a] Jarvis, J.F. "Regular expressions as a feature selection language for pattern recognition", Proceedings of the 3rd International Joint Conference on Pattern Recognition, Coronado, California, November 1976, pp 189-192 [JarvisJF76b] Jarvis, John F. "The line drawing editor, an Experiment in Computer Vision", Proc. SigGraph '76, July 14-16, 1976, Philadelphia, pp. 288-292 Drawing editor / sketch recognition, mark-up commands (edit commands / gestures) written on paper, optically scanned to change drawings (no tablet!). Gesture shapes include digits, X, connector points, carets, arrow tabs. [JarvisJF77a] Jarvis, J.F. "The line drawing editor: Schematic diagram editing using pattern recognition techniques", Computer Graphics Image Processing, Vol 6, 1977, p. 452-484 Drawing editor / sketch recognition [JonesAK78a] Jones, Anita K. and Liskov, Barbara H. "A Language Extension for Expressing Constraints on Data Access", CACM, Vol 21 No 5, May 1978, pp. 358..367 Constraint-definition programming language extensions for fine-grain defining access control/capabilities (rights) of objects -- what other data/objects it can access, what rights another object must have to access it. Control fine-grained to type of operation also. [Kanal77] Kanal, Laveen N. "Current Status, Problems and Prospects of Pattern Recognition", Systems, Man, and Cybernetics Review, Vol 6 No 4, August, 1977, pp 9-14 Overview of pattern recognition: many theories, few products: term covers too many divergent problem areas. Pattern recognition overview: well-constrained problems, sustained efforts lead to successful performance. Realistic performance: "Identifying animals in a zoo is much different from hunting them in the forest." Overview: spatial/temporal/external context, feature extraction (most publicly undocumented: not published), experiment/training design, clustering, no standard data bases [Kaplow76a] Kaplow, Roy and Molnar, Michael "A computer-terminal, hardware/software system with enhanced user input capabilities; the enhanced-input terminal system (EITS)", ACM SIGGRAPH, Vol 10 No 2, Summer 1976, pp 116-124 Dynamically variable keyboard, using digitizer/touch-screen and simulated function keys/buttons: simulated devices, compare with simulated devices and multiple-finger gestures for Schumer, Fingerworks, Buxton. Refers to multiple touches as "chord" mode for simulated keyboard on touchscreen. User-interface where icons/keyboard labels on touch-screen force/pressure-sensitive digitizer/touch-screen change dynamically. Gesture input as chorded/multi-button/multi-touch input. Application Generator? Soft buttons? Note: had multi-touch sensing. [Kaplow80] Kaplow, R. and Molnar, M.K. "Dynamically variable keyboard system", United States Patent 4,202,041, May 6, 1980 assigned to Massachusetts Institute of Technology, Cambridge, Massachusetts, 1980 Dynamically variable keyboard, using digitizer/touch-screen and simulated function keys/buttons: simulated devices, compare with simulated devices for Schumer, Fingerworks, Buxton. User-interface where icons/keyboard labels on touch-screen force/pressure-sensitive digitizer/touch-screen change dynamically. (In chord mode at least) Keys are not selected until release, permitting user to adjust keys (position) before selecting. Supports make-before-break, rolling contact (N-key rollover), and purposeful multiple key contact. Gesture input as chorded/multi-button/multi-touch input? Application Generator? Soft buttons? Note: had multi-touch sensing. [Kato80] Kato, O, Fujita, T., Niwa, M., Morishita, T., Fujii, N., Tanahashi, J. "Handwriting Input System for Japanese", Proceedings of IFIP 80, Tokyo, Japan, October 6-9, 1980, North-Holland publishers, pp 689-694 Science Citation Index. Features are stroke-relative position, then candidate selection, then personal training. Claims 98% recognition rate performance on Japanese Kanji/kana/numerics, after training. [KayAC76a] Xerox Palo Alto Research Center, Learning Research Group (Kay, A.C.) "Personal Dynamic Media", Xerox PARC Learning Research Group, 1976 Dynabook, original report. Musical score capture system OPUS using music keyboard. Turtle geometry with command sequences / scripts. One mention of touch-sensitive screen / touchscreen for future projects, not part of physical Dynabook. [KayAC77a] Kay, A.C. "Microelectronics and the Personal Computer: Personal Dynamic Media", Scientific American, September 1977, page 231



The original user-interface paper: Dynabook (does NOT mention electronic ink or character recognition). See also longer Xerox report. [KayAC77b] Kay, A.C. and Goldberg, A. "Personal Dynamic Media", IEEE Computer, Vol 10 No 3, March 1977 pp 31-41 Dynabook did NOT have stylus or handwriting input: mouse only .. popularly assumed to be the original "electronic ink" handwriting paper: Dynabook, but not so. See also longer Xerox report same title. [Kegel78] Kegel, A.G., Giles, J.K., and Ruder, A.H. "Observations on Selected Application of Optical Character Readers for Constrained Numeric Handprint", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 4, April 1978, pp 282-285 Most handwriting recognition OCR problems are over-printing, 0-vs-6 ambiguity. Also how well human writers match constraints for various commercial OCR handwriting recognition products [KentST76a] Kent, Stephen T. "Encryption-Based Protection Protocols for Interactive User-Computer Communication", MIT Lab. for Computer Scients Technical Report MIT/LCS/TR-162, May 1976 Encrypted communications/networking in Multics: chain based block cipher [KickertWJM76a] Kickert, Walter J.M. and Koppelaar, Henk "Application of Fuzzy Set Theory to Syntactic Pattern Recognition of Handwritten Capitals", IEEE Trans. Systems, Man, and Cybernetics, Vol 6 No 2, February 1976, pp 148-ff [KleyV78a] Kley, Victor "Graphical data entry pad", United States Patent 4,079,194, March 14, 1978 Resistive sheet touch tablet, using two resistive layers: bottom layer top layer scanned separately for X and Y. [KnowltonKC76a] Knowlton, Kenneth Charles "Method and apparatus for using pushbutton telephone keys for generation of alpha-numeric information", United States Patent 3,967,273, June 29, 1976 Use telephone keypad (3 high x 4 wide) with multiple QWERTY alphabetic characters on keys. Use hits a key to select a group, then keys to select position in group, to get a particular character. [KnowltonKC77a] Knowlton, K.C. "Computer Display Optically Superimposed on Input Devices", Bell System Technical Journal, Vol 56 No 3, March 1977, pp 367-384 User-interface: project image on keyboard to change keys: refers to tablet input possibility; compare with Kaplow for soft buttons. Mentions using X-Y tablet (digitizer) with display. [KnowltonKC80a] Knowlton, Ken "Progressive Transmission of Grey-Scale and Binary Pictures by Simple, Efficient, and Lossless Encoding Schemes", Proceedings of the IEEE, Vol 68 No 7, July 1980, pp 885-896 Compression of images by run-length and difference encoding: lossless [KohnfelderLM78a] Kohnfelder, Loren M. "Towards a Practical Public-key Cryptosystem", Undergraduate thesis, M.I.T., May, 1978 Original source for concept of "digital certificate" for PKI infrastructure, trusted authority certifies identity to go with a particular public/private key pair. Discusses Merkle, Merkle-Hellman, and RSA public-key algorithms. Certificate is alternative to Public File / contacting central certifying directory on each use. Public File == CA certification authority. [KruegerMW77a] Krueger, Myron W. "Responsive Environments", Proc. NCC 1977, pp. 423-433 VIDEOPLACE -- optical/video sensing of user outlines in a room-sized environment: non-realistic interactions possible e.g. two users' images must combine to create a window into another scene, etc. Multiple-user, compare optical sensing with multi-touch sensing, interactive desk environment. [KuipersJ76a] Kuipers, Jack "Tracking and Determining Orientation of Object Using Coordinate Transformation Means, System and Process", United States Patent 3,983,474, assigned to Polhemus Navigation Sciences, Incorporated, Burlington, Vermont, September 28, 1976 From a old list of Polhemus patents Patent on correction tables on stylus angle, orientation, position in a 3D tablet digitizer [KuipersJ77a] Kuipers, Jack "Apparatus for generating a nutating electromagnetic field", United States Patent 4,017,858, assigned to Polhemus Navigation Sciences, April 12, 1977 Cited in MaedaK87 three-dimensional / six dimensional electromagnetic digitizer, using X/Y/Z coils. Determines both position and orientation of stylus, thus able to report not only the position on a (threedimensional) surface, but also the plane of the surface at that point. [KuklinskiT77a] Kuklinski, T. "Implementing a Functional Attribute Based Theory of Character Recognition", unpublished manuscript, Pencept, Incorporated, March 21, 1977 Kuklinski voting algorithm for an imperfect decision tree based on pair-wise voting [KuklinskiT79a] Kuklinski, T., PhD Thesis: "Graphical Context as an Aid to Character Recognition", Massachusetts Institute of Technology, Cambridge, Massachusetts, 1979 Graphical context in character recognition: if all the characters are squished down low (eg. K, Y, E, H, etc.), then the mount of low-tail needed to distinguish a pair of similar characters (e.g. V versus Y) is also less. Really gets into similarity of distinguishing features as setting thresholds. [Kurzweil80a] Kurzweil Computer Products "Kurzweil Data Entry Machine", Kurzweil Computer Products, 1980



Kurzweil OCR product: Omni-Font Character Recognition, describes multiple algorithms for recognition, apparently to speed up processing. Includes print-to-braille, special modules for scanning telephone books to detect fields. [Kwan79] Kwan, C.C., Pang, L., and Suen, C.Y. "A Comparative Study of some Recognition Algorithms in Character Recognition", Proceedings of International Conference on Cybernetics and Society, pp 530-535, October 1979, IEEE order number CH1424-1/79/0000-0530 Boundless variability of hand-print handwriting [Lambden78] Lambden, Martin Roy "Electrographic Apparatus and HandRest for Use Therewith", United States Patent 4,129,746 assigned to Quest Automation Limited, Stapehill, Wimborne, England, December 12, 1978. Hand rest for pressure-sensitive / resistive-film tablet, so that hand does not press on tablet surface [Lambden78b] Lambden, Martin Roy "Electrographic Apparatus and Method Of Producing and Electrode Surface Therefor", United States Patent 4,070,544 assigned to Quest Automation Limited, Stapehill, Wimborne, England, January 24, 1978 Description of the resistive-film, plastic pressure sheet digitizer tablet used for the Quest handwriting product [Larsen77] Larsen "Quadrant Check for Signature Verification", IBM Technical Disclosure Bulletin, Vol 20 No 4, September 1977, pp 1538-1539 Accelerometer pen/stylus design (like SRI pen), produces chain codes [Lerner80] Lerner "Understanding speech proves tough task for machines", High Technology, April 1980 Magazine article [Lew79] Lew, J.S. "Optimal Designs of Instrumented Pens for Signature Verification", IBM Technical Disclosure Bulletin, Vol 21 No 8, p. 3415, January 1979 Variation on SRI (non-digitizer) pen: accelerometer pen stylus design [Liljenwall79] Liljenwall, James and Moss, David "Information Entry System", United States Patent 4,139,837, assigned to Creative Ventures, Incorporated, Dayton, Ohio, 1979 On-line character recognition on the face of a wristwatch or calculator: unistroke/single-stroke characters, written with a fingertip Simple on-line character recognition using zones/regions recognition [Lin78] Lin, W.C. and Pun, J.H. "Machine Recognition and Plotting of Hand-Sketched Line Figures", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 1, January 1978, pp 52-57 Symbol recognition and graphical editing: "prettyizing" a sketch : shows recognition of single-stroke circuitdiagram symbols "recognition -- seems to be a problem at times since the user cannot have complete control over the input data when sketching on the tablet" [Lin79] Lin, C.N., Herbst, N.M., and Anthony, N.J. "Automatic signature verification: System description and field test results", IEEE Trans. Systems, Man, and Cybernetics, Vol 9, 1979, pp 35-38 Cited in Teulings86a [LiuCN79] Liu, C.N., Herbst, N.M., and Anthony, N.J. "Automatic Signature Verification: System Description and Field Test Results", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-9 No 1, January 1979, pp 35-38 Signature verification system from IBM using force/pressure and accelerometry Signatures: reference signature modified before storage to deal with variability Signature verification: 1.7% Type I (false rejection) and 0.4% Type II (false acceptance) for deliberate (expert) forgers, false acceptance rate of 0.02% for random (amateur) forgers Signature verification: 1% of users had too much variability for the system to work well: solution is administrative Signature hardware problems: baseline drift in force/pressure sensor, noise in force/pressure sensor, lack of reset, non-uniformity of ink cartridges [MackworthAK77a] Mackworth, Alan K. "On reading sketch maps", Proc. IJCAI '77, 5th Intl'. Joint Conf. on Art. Intelligence, Vol 2 pp 598-606 MAPFREE sketch recognition and sketch editing using graphical data tablet. Segmentation of sketches into chains (of marks) and regions (of area), syntactic rules for joins of chains (junctures) [MadeiraM78a] Madeira, Marion "Gesture Recognition and the Use of Touch Sensitive Color Displays for Simple Diagramming", Master's Thesis, M.I.T, August 11, 1978 GESTUR gesture recognition on touch-sensitive device digitizer, using adaptive/trainable recognizer by Heckbert. Drawing program for entering diagrams/sketches during typing. Rubout gesture for erase, "L" angle gesture for rectangle. [MatsushitaT78] Matsushita, T. et al "Personal Computer-Aided Composition and Editing System", 3rd USA-Japan Computer Conference, 1978, 10/10-12/78 cited for text-editing of handwritten marks, graphical editing [MattesHG76a] Mattes, Hans George and Torok, Gabor Peter "Graphic Input Terminal", United States Patent 3,959,585, May 25, 1976 Resistive sheet touch panel using two sheets with voltages applied in X and Y directions on the two sheets, with an unenergized third sensor sheet between. [Maugh78] Maugh, Thomas H. II "Holographic Filing: An Industry on the Verge of Birth", Science, Vol 201,



August 4, 1978, pp 431-432 Cited in Sziklai84 on storing optical images, with signatures. Holographic storage would be more reliable, as losing part of holographic image leaves full image recoverable, albeit at lesser signal/noise ration. Analog storage, not digital. [McDermott80] McDermott, J. "R1: An Expert in the Computer Systems Domain", Proceedings of the First Annual National Conference on Artificial Intelligence, 1980, pp 269-271 Expert A.I. system to configure orders for DEC VA computers [MerkleRC80a] Merkle, Ralph C. "Protocols for Public Key Cryptosystems", Proc. 1980 IEEE Symp. on Security and Privacy, pp. 122..134 Review of weaknesses of key distribution / key management for public-key cryptography. Vulnerability of Kohnfelder certificates is compomise of central keys. [Mihelic77] Mihelic, F.; Pavesic, Nokola and Gyergyek, Ludvik "Recognition of Writer of Handwritten Texts", Proceedings of the International Conference on Crime Countermeasures, Kentucky University, Lexington, 1977, pp. 237..240 Cited by Marlin Eller, Microsoft Pen Computing group [Milloy78] Milloy, D.G. "Comment on recognition and confusion of the lower-case alphabet", Perception and Psychophysics, 1978, Vol 24 No 2, pp 190-191 Substitution errors in human recognition of lower-case characters Criticizes defects in everybody's studies of lower-case character confusion matrices for human recognition [Moayer76] Moayer, B. and Fu, K.S. "A tree system approach for fingerprint pattern recognition", IEEE Trans. Computers, Vol C-25, 1976, pp 262-274 [MorenoR78a] Moreno, Roland "Systems for Storing and Transferring Data", United States Patent 4,092,524, May 30, 1978 Chip-card/chip-karte, defined as electronic object. Combine shorter password (i.e. PIN) with cryptographic chip-card to get security equivalent to longer password. [MoriS80] Mori, S. and Suen, C.Y. "A Study of the Zero and Letter Oh Problem", Proceedings of 5th International Conference on Pattern Recognition", December 1980, pp 842-847 Cited in Litvin82 [Morrissey76] Morrissey, J.H. "Electronic Calculator Based on Character Recognition of Input from Stylus Acceleration Dynamics", IBM Technical Disclosure Bulletin, Vol 19 No 7, December 1976, pp 2816-2817 Disclosure of US patent 3,983,535 for signature verification, but says extension to input for handwriting recognition is obvious. On user-interface of writing on an electronic calculator (using chaincodes/acceleration dynamics) [MullerAM77a] Muller, Alex Marcel and Tyrrell, Joseph Leslie "Electrical Sensing Writing Pen", United States Patent 4,034,155, July 5, 1977 Pen for electrostatic/capacitive grid tablet [Muth77] Muth, F.E. and Tharp, A.L. "Correcting human error in alphanumeric terminal input", Information Processing Management, Vol 13, 1977, pp 329-337 Cited in Srihari83: on context/spelling correction [MyAppleSpace79a] MyAppleSpace.com "Apple Graphics Tablet 1979 Summagraphics Magnetostrictive", MyAppleSpace.com, fetched 2012 Summagraphics magnetostrictive tablet for Apple computers, 1979. Filed under MyAppleSpace11a. [Myers80] Myers, C.S., Rabiner, Lawrence R., and Rosenberg, A.E. "Performance tradeoffs in dynamic time warping algorithms for isolated word recognition", IEEE Trans. Acoustics, Speech and Signal Processing, Vol ASSP28, 1980, pp 622-635 Cited in Kruskal83 [Narasimhan80] Narasimhan, M.A., Devarajan, Venkat, and Rao, K.R. "Simulation of Alphanumeric Machine Print Recognition", IEEE Trans. on Systems, Man, and Cybernetics, Vol SMC-10 No 5, May 1980, pp 270-275 Shows performance results in a confusion matrix for alphabet from OCR data, compare with Shillman, who uses a confusion matrix to show which features are distinguishing [Naus76] Naus, M.J., and Shillman, R.J. "Why a Y Is Not a V: A New Look at the Distinctive Features of Letters", Journal of Experimental Psychology: Human Perception and Performance, Vol 2 No 3, 1986, pp 394-400 Shillman article on functional attributes [NeedhamRM78a] Needham, Roger M. and Schroeder, Michael D. "Using encryption for authentication in large networks of computers", CACM Vol 21 no 12 December 1978, pp. 993..999 PKI authentication techniques in communications: pre-dates SSL, certificates. Authentication protocols (exchange of keys), signed communication, off-line authentication. Presumes secure computing platform (trusted system). Database of public keys/trusted authentication authorities instead of certificates, but uses heirarchical naming of trusted authorities. [NewScientist79a] New Scientist "Computer becomes a secretary: Quest Automation Micropad", New Scientist, 18



October 1979, page 190 Quest Automation Micropad (based on earlier Datapad) handwriting recognition terminal. Based on work from National Physical Laboratory in Teddington. Mentions that recognition uses stroke direction also, e.g. to distinguish sloppy 8 from sloppy B. Advertizing plays on keyboard reluctance of managers compared to secretarial staff. [Niemann77] Niemann, Heinrich "Classification of Characters by Man and Machine", Pattern Recognition, Pergamon Press 1977, Vol 9, pp. 173-179 Human performance on OCR recognition of handwritten characters compared with machine performance, under various noise conditions: machine works better on typed fonts, human on handwritten characters [OBrien79] O'Brien, M.T. "A Network Graphical Conferencing System", Rand Corporation, Stanta Monica, California, August 1979 (N-1250-DARPA) Real-time freestyle user interface for electronic ink: a central blackboard/whiteboard conferencing / drawing communally, separate windows for each user for typing (words). [Odaka79] Okada, Jazumi, and Masua, Isao "Some Results on Stroke Order Free On-line Character Recognition", PRL Vol 79 No 29, pp 77-84 (in Japanese) stroke-order independence by grouping strokes which are close together (?) [Odaka80] Okada, Jazumi, and Masua, Isao "Online Recognition of Handwritten Characters by Approximating Each Stroke with Several Points", Trans. IECE, 1980, Vol 63-D, No 2, pp 153-160 (in Japanese) See same-title paper in IEEE SMC, Nov/Dec 1982 [OkaM78] Oka, Masatomo and Yasuhara, Makoto "Signature Verification Experiment based on Nonlinear Time Alignment: A Feasibility Study", IEEE Trans on Systems, Man and Cybernetics, Vol SMC-7, No 3, pp 212-216 Dynamic time-warping / non-linear time alignment via dynamic programming for signature verification [Okuda76] Okuda, T., Tanaka, E. and Kasai, T. "A method for the correction of garbled words based on the Levenshtein metric", IEEE Trans. Computers, Vol C-25, February, 1976, pp 172-178 DosterW77 on spelling dictionary? [Pallishusky76] Pallishusky, Silvia Minka "Method and Means for Teaching Alphabet Recognition and Learning Handwriting", United States Patent 3,950,863, April 20, 1976 Overlaying transparencies to teach children the order of strokes for learning handwriting Refers to nine basic shapes, or segments, from which all 26 letters (52 with lower case) are constructed [Pavlidis77a] Pavlidis, Theo "Structural Pattern Recognition", Springer-Verlag, New York, 1977 Defines pattern recognition as "identification of the ideal which a given object is made after" Definition: classification theory vs boundary theory [Pavlidis77b] Pavlidis, Theo "Comments on Current Perspectives in Pattern Recognition", Systems, Man, and Cybernetics Review, Vol 6 No 4, August, 1977, pp 8-9 Just because humans have good pattern recognition performance, and poor arithmetic, does not mean pattern recognition is easier than computer arithmetic [Pavlidis77c] Pavlidis, Theo "Polygonal approximations by Newton's method", IEEE Trans. Computers, Vol C-26, 1977, pp 800-807 [Pavlidis80] Pavlidis, Theo "Algorithms for shape analysis and waveforms", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PARM-2, July 1980, pp 301-312 Chain codes, corner detection, curvature, Fourier shape, shape analysis for OCR image recognition (segmented pictures) [Pencept80a] Pencept, Inc. "Pencept: New Concepts in Source Data Capture", Pencept, Inc. 1980 (hardcopy book) Pencept business plan for financing, handwriting recognition tablet. Cites figures on productivity gain (faster) for data capture with handwriting than with typing/keyboarding. [PepperW78] Pepper, William Jr. "Human-Machine Interface Apparatus", United States Patent 4,071,691, January 31, 1978, assigned to Peptek, Incorporated, Bethesda, Maryland Phase-detection digitizer, resistive film touchscreen responds to finger touch: compare to Scriptel? [PepperW78a] Pepper, William Jr. "Human-Machine Interface Apparatus", United States Patent 4,129,747, December 12, 1978 Cited by Sklarew88b on user interfaces on touchscreens. touch-screen digitizer using phase delay of a signal in a resistive film with capacitive coupling [PepperW80] Pepper, William Jr. "System for producing electric field with predetermined characteristics and edge terminations for resistance planes therefor", United States Patent 4,198,539, April 15, 1980 Resistive-film digitizer using non-linear and three-dimensional electric fields, two voltage sources. Fields can be curved as desired, or corrected for pin-cushion distortion. Improvement to 4,071,691: capacitive digitizer, cited in Microtouch patent as sensor for pen or finger controller [Persoon77] Persoon, E. and Fu, K.S. "Shape discrimination using Fourier descriptors", IEEE Trans. Systems, Man and Cybernetics, Vol 7, 1977, pp 171-179



Taxt90: Pavlidis80 says 95% on IEEE handwriting recognition database [PetersonJL80a] Peterson, J.L. "Computer Programs for Detecting and Correcting Spelling Errors", CACM, Vol 23 no 12, December 1980, pp 676-687 Cited in Sinha88: on spelling context correction Has many reference on spelling correction, detection, bestmatch to strings, limits of lack of syntactic and/or semantic context [Price79] Price, K. and Reddy, R. "Matching segments of images", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-1, January, 1979, pp 110-116 Cited in Burr81 [PrughRW80a] Prugh, R.W. and Fadden, B.J. "Graphic Digitizer", United States Patent 4,206,314, June 3, 1980, assigned to GTCO Corporation, Rockville, Maryland Digitizer with grid as transmitter, loop as the receiver [PurcellSC77a] Purcell, Stephen C. "Understanding hand-printed Algebra for computer tutoring", Massachusetts Institute of Technology, Artificial Intelligence Laboratory Memo No 445, February 1977 (also available as S.M. Thesis) 2D programming language with tablet and symbol/handwriting recognition user interface. Uses domainspecific knowledge (i.e. mathematical expressions) / context in parsing and recognizing mathematical expressions with on-line handwriting recognition. "chalkboard languages" - two-dimensional doodles, writing, pictures Points out that overlap is inadequate as the only criterion for parsing/segmentation in handwriting (two-dimensional) Uses a lattice representation of possible stroke groupings/segmentations of characters, showing ambiguities of parsing (vs. of recognition) Features: local geometry (curvature, etc.) not enough: need global geometry (closure of endpoints on "U" vs. "O", etc.) Variability: greater when humanswritein context, vs one character at a time Recognizer mentioned by Alan Kay in email posting on www.squeakfoundation.org, November 2000 [Rabiner78] Rabiner, Lawrence R., Rosenberg, A.E., and Levinson, S.E. "Considerations in dynamic time warping for discrete word recognition", IEEE Trans. Acoustics, Speech, and Signal Processing, Vol ASSP-26, 1978, pp 575582 Cited in Kruskal83 [Rabiner79] Rabiner, Lawrence R., and Wilpon, Jay G. "Speaker-Independent Isolated Word Recognition for a Moderate Size (54 Word) Vocabulary", IEEE Trans. Acoustics, Speech, and Signal Processing, Vol ASSP-27 No 6, pp 583-587 Discrete word speech recognizer using statistical clustering templates: a carefully-trained speakerindependent limited recognizer can work as well as casually-trained speaker-dependent Speech recognition on a professionally trained actor with very human-clear diction was worse Clustering procedure for voice templates much better than casual enrollment Describes many enrollment/training methods for speakerindependent/dependent recognition system Training/enrollment data was "cleaned up" to remove breathing sounds, clicks, etc. [Rabiner80] Rabiner, Lawrence R. and Schmidt, C.E. "Application of dynamic time warping to connected digit recognition", IEEE Trans. Acoustics, Speech, and Signal Processing, Vol ASSP-28, 1980, pp 337-388 Voice recognition. Dynamic time warping to locate images of digits in a string: gives probabilistic results for full string, depends on post-processing to select final result. Segmentation not known in advance, nor how many digits in the utterance. Cited in Kruskal83 [RadicePF80a] Radice, P.F. "Personal Verification Device", United States Patent 4,234,868, November 18, 1980, assigned to Penwalt Corporation, Philadelphia, Pennsylvania Signature verification using force/pressure biometric only. "force/pressure only" digitizing tablet using piezoelectric or pyro-electric sheet or film [RahuelJC80] Rahul, Jean-Claude and Dagnelie, Jean-Paul "Graphics table particularly for a telewriting system", United States Patent 4,225,750, September 30, 1980 X/Y electromagnetic grid tablet digitizer operating at 10 Megahertz, modulated at 100 Kilohertz. Cites application as a telewriting (telautograph?) device. Cites to 1973 references for electronic whiteboard/blackboard systems. [RaoK80a] Rao, Kameswara and Balck, Kenneth "Type Classification of Fingerprints: A Syntactic Approach", IEEE Trans. PAMI, vol PAMI-2 no 3, May 1980, pp. 223-231 Classification of fingerprints using syntactic chain codes / BLRT patterns: loop, whorl, double-whorl, etc. [Reenskaug79] Reenskaug, Trygve "A note on Dynabook requirements", Xerox PARC memo, 22 March 1979: partial copy available on the Internet Automatic layout / structuring of graphical objects with connections: note that it does not mention handwriting recognition or character recognition? Refers to Prokon Anticipates market for Dynabook-like products about 1982-1986, programming in a higher-level language (e.g. APL, Pascal) [ReevesW78a] Reeves, William; Buxton, William; Pike, Robert; and Baecker, Ron "Ludwig: an Example of Interactive Computer Graphics in a Score Editor", Structured Sound Synthesis Project (SSSP), U. Toronto, Canada,



November 1, 1978 Graphical editor for music scores, using digital tablet. Has a drop-down (see-through/transparent?) tool, vertical menu of note durations, user drags it to put a particular duration over a note and release. [RetzDL76a] Retz, David L. and Schafer, Bruce W. "Structure of the ELF operating system", Proc. NCC National Computer Conference, 1976, pp. 1007-1016 Multi-user operating system for DEC PDP-11 computers, used in Arpanet development. USERLOADER utility to load (download) programs from another system on the Arpanet network, and install (map into virtual address space). Mentions TelNet, control-c EXEC character to interrupt programs. [RivestRL78a] Rivest, R.L.; Shamir, A.; and Adleman, L. "A Method for Obtaining Digital Signatures and PublicKey Cryptosystems", CACM, Jan 1983, Vol 26 No 1, pp. 96-99 Original RSA paper on PKI public key cryptography / public key cryptosystems. Example application is secure, signed email. Does not mention CAs. [RosenfeldA78a] Rosenfeld, Azriel "Survey: Picture Processing: 1977", (source not known) 1978 Not sure of source: Check Kuklinski's thesis? Large bibliography on picture processing, filtering, edge detection, image compression, character recognition and other kinds of pattern recognition (fingerprint, face, etc.), machine vision [RosenfeldA80a] Rosenfeld, A. "Picture processing: 1980", Computer Graphics and Image Processing, Vol 16, May, 1981 Fu86/Fu83 [RubincamDP79a] Rubincam, David P. "Electronic Book", United States Patent 4,159,417, June 26, 1979 Cited in Sklarew88b Portable electronic book, with user interface (no computer applications): book text contained on memory card, shown in display. Hinged notebook-like design. Compare with Dynabook by Alan Kay? [SIGGRAPH79] SigGraph "General Methodology and the Proposed Core System", report of the Graphics Standards Planning Committee, in Computer Graphics (Proc. of SIGGRAPH 79), Vol 13 No 3, August 1979 [SMC78] (various authors) "Special section on hand-print OCR", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 4, pp 279 ff, April 1978 Special section on real-world application experience with hand-print OCR, vs laboratory results [Saich77a] Saich, William George "Data Terminals having Interactive Keyboards and Displays and Data Processing Apparatus Incorporation such Terminals", United States Patent 4,028,695, June 7, 1977 Soft button keyboard: touch of key causes change in appearance to indicate that it is selected, press (release?) of key causes function to be performed. Two gestures, one of which has a highlight? Or One gesture, with two kinds of feedback? [SakoeH76a] Sakoe, Hiroaki "Automatic Character Recognition Device Employing Dynamic Programming", United States Patent 3,979,722, September 7, 1976, assigned to Nippon Electric Co. Limited, Tokyo Japan Refers to "standard patterns" for template matching (which it is not). Eight-direction chain codes. Uses dynamic matching of vector maxima/minima for DCR handwriting recognition (BLRT chain codes). Chaincode BLRT points are absolute, vary distances between for degree of match. [SakoeH78a] Sakoe, Hiroaki and Chiba, S. "Dynamic programming algorithm optimization for spoken word recognition", IEEE Trans. Acoustics, Speech and Signal Processing, Vol ASSP-26, 1978, pp 43-49 Cited in Kruskal83 dynamic programming method for speech recognition, discrete words: only categorized that error rate is one-third less, not overall accuracy or success. [SakoeH79a] Sakoe, Hiroaki "Two-Level DP-Matching -- A Dynamic Programming-Based Pattern Matching Algorithm for Connected Word Recognition", IEEE Trans. Acoustics, Speech and Signal Processing, Vol ASSP-27, NO 6, 1979, pp 588-ff dynamic programming method for speech recognition, continuous speech but ver limited vocabularly: digits only, "sentences" of one to four digits, only five speakers in test data. [ScanData76a] Scan-Data Corporation "ScanData 2250/1 OCR System with handprint recognition", Scan-Data Corporation, 1976 OCR product for constrained handprint recognition, using very stylized block writing style. Recommended size, shape, style of handprint constraint boxes in designing forms, which must be in particular color separate from ink. [SchmandtC80a] Schmandt, Christopher "Some Applications of Three-Dimensional Input", Master's Thesis, M.I.T., January 1980 Study of using Polhemus three-dimensional digitizer for pointing and body position. The original work on "Put-That-There", combining speech recognition and hand pointing. [SchultzJ79a] Schultz, Jan and Davis, Layton "The Technology of PROMIS", Proc of IEEE, Vol 67 No 9, September 1979, pp. 1237-1244 PROMIS: Early Digiscribe SAW touch-screen user interface system for medical records, 1967, user interface mostly point-and-click plus keyboard for text input.



[SeelbachHE80a] Seelbach, H.E. "Input Device for Input of Alphanumeric Characters into a Computer", United States Patent 4,184,147, January 15, 1980 Simple four-grid "character recognition" device: sequence of crossings [Sethi77] Sethi, I.K. and Chatterjee, B. "Machine recognition of constrained handprinted Devanagari", Pattern Recognition, Vol 9, pp 69-75, 1977 Cited in Biswas81 [Seuffert77] Seuffert, P. "An Application of Line and Character Recognition in Cartography", Proceedings of IEEE Conference on Pattern Recognition and Image Processing, pp 337-342, June 1977 Cited in Goodale83 [ShillmanR76a] Shillman, R. "Automatic Recognition of Thick Stroke Characters", Quarterly Progress Report No 118, Appendix IX, Massachusetts Institute of Technology Research Laboratory of Electronics, July 1976 How "legs" appear shorter as they get thicker [ShillmanR76b] Shillman, R. and Naus, G. "The Distinctive Features of the Letters O and D", Quarterly Progress Report No 118, Appendix X, Massachusetts Institute of Technology Research Laboratory of Electronics, July 1976 Functional attributes in character recognition: distinguishing characters O vs D is (one-sided) symettry. also "leg" attribute for Y an V, F and C, P and D, H and U, and A from O. [ShillmanR76c] Shillman, R., Kuklinski, T. and Blesser, B. "Psychophysical Techniques for Investigating the Distinctive Features of Letters", International Journal of Man-Machine Studies, Vol 8, pp 195-205, 1976 Classical letter shapes do not occur frequently in hand-printing Shillman article on functional attributes [ShillmanR77a] Shillman, R.J. and Babcock, R.T. "Preliminary Steps in the Design of Optical Character Recognition Algorithms", unpublished manuscript, Cognitive Information Processing Group, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, 1977 Physical to functional rules (functional attributes) for 2 and Z at Massachusetts Institute of Technology (see Babcock) Example of 2-Z space, one space invading the other Good introductory example of physical to functional rules, how to pick the dominant feature for a character recognizer [ShillmanR78a] Shillman, Robert J. "The Application of Optical Character Recognition to Machine Translation", Manuscript draft, Massachusetts Institute of Technology, 1978 Manual re-typing of text (especially with different alphabets: Cyrillic, etc.) largest bottleneck on machine translation of printed documents. Review of OCR static optical character recognition for text conversion. [ShillmanR78b] Shillman, Robert J. "Character Recognition and the Data Entry Problem", Manuscript draft, Massachusetts Institute of Technology, 1978 Focuses on OCR static optical character recognition for data entry, versus DCR on-line dynamic character recognition. Manual re-typing of text (especially with different alphabets: Cyrillic, etc.) largest bottleneck on machine translation of printed documents. [ShinghalR79a] Shinghal, R. and Toussaint, G.T. "A bottom-up and top-down approach to using context in text recognition", International Journal of Man-Machine Studies, Vol 11, 1979, pp 201-212 Cited in Sinha88: on spelling/dictionary context correction [ShinghalR79b] Shinghal, R. and Toussaint, G.T. "Experiments in text recognition with modified Viterbi algorithm", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-1, 1978, pp 184-192 Cited in Sinha88: on spelling/dictionary context correction [Shortliffe76] Shortliffe, E.H. "Computer-Based Medical Consultation: MYCIN", American Elsevier, New York, 1976 Expert A.I. system to diagnose infectious blood diseases Cited in FoleyJD84 [Simmons80] Simmons, Robert M. "An On-line Character Recognizer", Interface Age, March 1980, pp 110-114 Very simple on-line handwriting character recognition program running on a 1980 Z-80 CP/M system effectively trims ends of stroke by looking for matching chaincode sequence within longer chain-code sequences Zone-based recognizer, using chain of codes for the zones visited: single-stroke characters? [SimonJC77a] Simon, Jean Claude and Rosenfeld, Azriel "A short survey of character recognition", Digital Image Processing and Analysis, 1977 Overview of character recognition concepts: preprocessing, feature extraction, classification. Mostly literature reference list. [SipplCJ80a] Sippl, Charles J. and Sippl, Roger J. "Computer Dictionary, Third Edition", Howard Sams, 1980 (hardcopy book) Technical dictionary 1980 [SmithAR78a] Smith, Alvy Ray "Color Gamut Transform Pairs", Proc. SIGGRAPH 78, Aug 1978, pp. 12-19. Reprint, "Tutorial: Computer Graphics" IEEE Computer Society, 2nd Ed., 1982, pp. 376-383 Early paper related to Alpha blending: mathematical treatment of color blending / teinting in digital graphics, in RGB space and HSV space. More than 8 bits per color imperceptible to human user eye. Hexcone model for color perception. Gamma correction / gamma compensation for nonlinearity of color scale. [SmithAR79a] Smith, Alvy Ray "Tint Fill", Proc. SIGGRAPH 79, 1989, pp. 276-298. ACM SIGGRAPH Computer



Graphics Newsletter, Vol 13 No 2, August 1979 Area fill algorithm for shaded images, where boundaries are not sharp. Only changes tint of pixels, not brightness. [SnyderEJ77a] Snyder, Edward J. and Domyan, Stephen L. "Position coordinate determination device", United States Patent 4,018,989, assigned to Summagraphics Corporation, April 19, 1977 Cited in Lukis87 on digitizer tablets Ratiometric measurements in magnetostrictive grid tablet, to correct for errors that increase with tablet size. [Spanjersberg78] Spanjersberg, A.A. "Experiments with Automatic Input of Handwritten Numerical Data into a Large Administrative System", IEEE Trans. on Systems, Man, and Cybernetics, Vol SMC-8, No 4, April 1978, pp 286-288 Shows various constraint methods on handwriting styles (boxes, zones, write around the dots, markedsense card) to make handwriting more recognizable by machine, gives some performance results [Spohrer80] Spohrer, James C., Brown, Peter F, Hochschild, Peter H., and Baker, James K. "Partial Traceback in Continuous Speech Recognition", IEEE publication 0360-0913/80/000/0036 Verbex Corporation, formerly Dialog Systems Inc, Two Oak Park, Bedford MA 01730 use of context in continuous speech recognition [Srihari76] Srihari, S. and Zack, G. "Document Image Analysis", Proceedings of 8th International Conference on Pattern Recognition, Paris, France, October 1986, pp 434-436 Cited in Baird86a: on document skew alignment angle for OCR [Stallings76] Stallings, W. "Approaches to Chinese Character Recognition", Pattern Recognition, Vol 8, 1976, pp 87-98 [StockholmE79a] Stockholm, E. "Recognition of a writer as a function of his method of writing", Perception and Motor Skills, Vol 49, 1979, pp 483-488 (partial copy) Identify user by handwriting characteristics. Teulings86a cites for invariant spatial features in handwriting, regardless of writing conditions for subject (blackboard, paper, etc.) [StumpeB77a] Stumpe, Bent "A New Principle for an X-Y Touchscreen", CERN technical report, March 16 1977 Capacitive/electrostatic touchscreen digitizer using X and Y planes of conductive traces: earlier work goes back to 1973. Additional papers in file regarding conductive polymers for a transparent touchscreen. Senses on multiple traces and determines "gravity point" of touch location. Compare with Scriptel. [StumpeB78b] Stumpe, Bent "Experiments to find a manufacturing process for an x-y touch screen: Report on a visit to Polymer-Physik GmbH", CERN technical report, 6 February 1978 Manufacturing process for x-y touch screen based on StumpeB77a. Clear and insulating conductive lacquers to make conductive/resistive films, also with antireflective properties. Mentions putting lacquer between wires for better optical properties. [Suen76a] Suen, C.Y. and Radhakrishnan, T. "Recognition of hand-drawn flowcharts", Proc. 3rd Intl. Joint Conf. on Pattern Recognition, pp. 424-428 OCR Recognizing flowchart symbols: standardized shapes, strict syntax (chain codes) [Suen77a] Suen, C.Y. "Alphanumeric Hand-prints with Stroke Directions and Sequences", internal report for Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1977 Original collection of writing samples for PenVerter Partners (Pencept) [Suen77b] Suen, C.Y. and Shillman, R.J. "Low Error Rate Optical Character Recognition of Unconstrained Handprinted Letters Based on a Model of Human Perception", IEEE Trans. Systems, Man, and Cybernetics, Vol 7 No 6, June, 1977, pp 491-495 Suen support (with Shillman) use of psychologically base features Suen claims the computer did better recognition than human on U-V [Suen78a] Suen, Ching Y. "Advances in Optical Character Recognition", Proc Canadian Computer conf, Edmonton, May 23-25, 1978, pp. 1-6 Handprinting OCR recognition, uses templates [Suen79a] Suen, C.Y. "A Study on Man-Machine Interaction Problems in Character Recognition", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-9 No 11, pp 732-737, November 1979 Refers to "users do not follow models, but revert to usual writing" on training/constrained handwriting recognition Ergonomics / user-interface problems in handwriting recognition [Suen79b] Suen, C.Y. "n-Gram statistics for natural language understanding and text processing", IEEE Trans. Pattern Analysis and Machine Intelligence", Vol PAMI-1, 1979, pp 164-172 Cited in Sinha88: for statistics on language (one million word samples) to use for context correction Contains several lists of English-language word frequency dictionaries [Suen80a] Suen, C. "Computer Analysis of Handprinted Characters", Proceedings of the 5th International Conference on Pattern Recognition, Miami Beach, Florida, December, 1980, pp 398-401 Human recognition success rate is 7.6% substitution error. Examples of handwriting variability from 1500 student enrollment cards, with distribution frequency Constraints: examples of how humans give cursive,



lower-case, etc. even when asked specifically to print styles Handwriting recognition by machine with better performance than humans requires understanding of how humans recognize characters Taxonomy of different approaches to handwriting recognition: template matching, correlation; distribution of points, moments, crossings, distances; transformations and series expansions; geometrical and topological features, syntactic and structural analysis Describes Suen's research on distinctive features (functional attributes) Lists of constraints typically tried to reduce handwriting variability [Suen80b] Suen, C., Berthold, M. and Mori, S. "Automatic Recognition of Hand-printed Characters - The State of the Art", Proceedings of the IEEE, pp 469-487, Vol 68 No 4, April 1980 Survey of handwriting recognition, both static (OCR) and dynamic (on-line). Used as citation in several patents. Follow-up by Tappert and Suen in 1990, by Plamondon in 2000. Specifically mentions decision trees at several points. [Suen80c] Suen, Ching Y. "The Role of Multi-Directional Loci and Clustering in Reliable Recognition of Characters", Proc. 6th Intl. Conf. on Pattern Recognition, Munich, Oct 19-22, 1982, pp 1020-1022 Fine angular resolution in OCR recognition of handwritten character by small features (characteristic loci). [Suenaga78] Suenaga, Yasuhito "Automatic text editing using handwritten mark recognition", Proceedings of the 9th Conference on Image Science and Engineering, No 3-1, pp 63-66, Dec. 1978 (in japanese) Cited in Suenaga83 [Suenaga80a] Suenega, Y. and Nagura, M. "A facsimile based manuscript layout and editing system by auxiliary mark recognition", Proceedings of 5th Int Conf on Pattern Recognition, Dec 1-4, 1980, pp 856-858 Cited in Kankaanpaa87, Goodale83 See also Nagura83, same paper [Summagraphics78a] Summagraphics / Scriptographics "Summagaphics data tablet digitizer", Summagraphics, 1978 Electromagnetic tablet digitizer line. Shows large separate electronic controller: dual system HW-11/48 combines two tablets (large for digitizing, 11"x11" small tablet for menus), one puck, on one controller and therefore half the data rate, with identifying bit/flag for tablet. Former name was Scriptographics. [Talmage78] Talmage, John E. and Bates, L. Dexter "Nonplanar Transparent Electrographic Sensor", United States Patent 4,071,689, January 31, 1978 Resistive-sheet digitizer using small spacing dots [TanLJ79a] Tan, Lu-Jan "Keyless electronic calculating pen", United States Patent 4,141,073, February 20, 1979 Accelerometer stylus/pen, recognition by zone-like directions of strokes (chain codes). Recognizes/evaluates mathematical/arithmetic expressions. Calculator display build into side of pen. Compare with Crane accelerometer pen, Anoto pen. [TappertCC78a] Tappert, C.C. and Das, Subratta K. "Memory and Time Improvements in a Dynamic Programming Algorithm for Matching Speech Patterns", IEEE Trans. Acoustics, Speech, and Signal Proc., Vol. ASSP-26, pp 583586, 1978 Cited by Marlin Eller, Microsoft Pen Computing group [TappertCC78b] Tappert, C.C., and Kurtzberg, J.M. "Elastic Matching for Handwritten Symbol Recognition", Proc. IBM Int. Conf. Image Processing Pattern Recognition, 1978. Also, IBM Res. Rep. RC9988, May 1983 Cited by Marlin Eller, Microsoft Pen Computing group [Tersoff78] Tersoff, Abraham I. "Man-Machine Considerations in Automatic Handprint Recognition", IEEE Trans. on Systems, Man, and Cybernetics, Vol SMC-8 No 4, April 1978, p 279 short review: reliable OCR for hand-print is nowhere in sight [Teulings79] Teulings, H.H.M., and Thomassen, A.J.W.M. "Computer-Aided Analysis of Handwriting Movements", Visible Language, Vol XIII No 3, 1979, pp 218-231 Describes digitizer set up to study handwriting: was tablet performance any good? [ThackerCP79a] Thacker, C.P.; McCreight, E.M.; Lampson, B.W.; Sproull, R.F.; and Boggs, D.R. "Alto: A Personal Computer", Xerox Corporation, 1979. Appeared in "Computer Structures: Principles and Examples", 2nd Edition, pp. 549-572, Siewiorek, Bell and Newall, eds. Original Alto personal computer with mouse and graphical user interface. Included BCPL language. Also mentions XY input tablet as input device, better than mouse for drawing, but tablet "interfered with keyboard" (no particulars given). [Thornburg79] Thornburg, David D. "Capacitive Transducer", United States Patent 4,177,421 Capacitive / electrostatic digitizer [Thornburg80b] Thornburg, David D. "The PrestoDigitizer Tablet: A Low-Cost Device for the Real-Time On-Line Entry of Handprinted Characters", Proceeding of 3rd ACM SIGSMALL Symposium and the first SIGPC symposium on Small Systems, pp 179-183 PrestoDigitizer product: seven-zone handwriting feature recognition. stroke sequence and direction information, simple dictionary to identify/recognize the handwritten character. Refers to learning algorithms (training of system?) [Thornburg80c] Thornburg, David D. "A Low-Cost Alternative to Data Entry Keyboards", Recreational Computing,



January 1980, pp 16-18 PrestoDigitizer product: seven-zone handwriting feature recognition [ThorntonRW79a] Thornton, R.W. "The Number Wheel: A Tablet Based Valuator for Three-dimensional Positioning", Proceedings of SIGGRAPH '79, published as Computer Graphics, Vol 13 No 2, August 1979, pp 102107 Essentially a one-directional trackball, two can be combined for simultaneous X and Y positioning (or angle) of 3D simulated object. Advantage over regular sliders is can do wrap-around (e.g. 0..359 for rotational angle), or implement hard limit stops at any value desired. Cited in FoleyJD84. Not 3D positional input. Implemented on a pen digitizer or touchscreen. [Toussaint77] Toussaint, G.T. "The Use of Context in Pattern Recognition", Proceedings of IEEE Computer Society Conference on Pattern Recognition and Image Processing, Renselear Polytechnic Institute, Troy, New York, June 6-8, 1977, pp 1-10 Context lies not just in the data, but in the perceiver Syntactical vs statistical use of context Statistical context: huge number crunching to reduce unmanageable probability distribution Types of word-feature context: graphological, phonological, statistical, syntactic, semantic, pragmatic Markov statistical methods for very large context dictionaries Modified Viterbi algorithm (statistical context) not so hot Goshtasby88 cites for using spelling dictionary look-up for context correction [Toussaint78a] Toussaint, G.T. "The use of context in pattern recognition", Pattern Recognition, Vol 10, 1978, pp 189-204 Essay on fundamental need to consider context in pattern recognition: e.g. optical illusions in human perception; length of descenders on characters "P" and "D"; A vs. H, 8 vs. B; Roman "U" and "V" Cited in HullJJ83 [Toussaint78b] Toussaint, G.T. and Shinghal, R. "Cluster analysis of English Text", Proceedings of IEEE Conference on Pattern Recognition and Image Processing, Chicago, IL, 1978, pp 164-172 Cited in HullJJ83 on context dictionary [TroxelDE76a] Troxel, D.E. "Feature Selection for Low Error Rate OCR", Pattern Recognition, Vol 8 No 2, April 1976, pp. 73-76 Scanned OCR character recognition (typed characters): need for strongly separating features, because of difficulty of dealing with long tails of probability distributions. Suggests placing character in confusion matrix to determine quality of classification by features used. [TsurutaS80a] Tsuruta, Shichiro; Sakoe, Hiroaki; Chiba, Seibi; and Nakada, Toshio "Connected Speech Recognition System DSP-100", NEC Research and Devleopment, No 56, January 1980, pp. 88-94 Speaker-independent continous speed recognition system from Nippon Electric: dynamic programming, vocabulary 200..400 words (Japanese), simple word matching then warping to match continuous speech. Tested on small number of subjects (7). [Tversky77] Tversky, A. "Features of similarity", Psychological Review, Vol 84, 1977, pp 327-352 Cited in IchikawaS84: on human recognition [UllmannJR77a] Ullmann, J.R. "A binary n-gram technique for automatic correction of substitution, deletion, insertion and reversal error in words", Computer Journal, Vol 20, 1977, pp 141-147 Spelling correction for transposed missing, and added characters in words. Correction of up to two errors by comparison to dictionary, special compression format for dictionary. Cited in Hull83a: on spelling correction [Viviani80] Viviani, P. and Terzuolo, V. "Space-time invariance in learned motor skills", in "Tutorials in motor behavior", Stelmach, G.E and Requin, J., editors, New York, North Holland, publishers, pp 525-533 Cited in Schomaker86 Schomaker86 cites that overlearned/over-trained writing strokes (signatures?) show great invariance in relative stroke timing [Wakamatsu79] Wakamatsu, Shuichi "Character Writing System", United States Patent 4,144,405, March 13, 1979 Method for generating Kanji characters making use of the sequential order of writing parts of the characters with an input unit Shows tablet input of handwritten Kanji characters, but does not refer to character recognition per se [Wallace76] Wallace, Victor L. "The Semantics of Graphic Input Devices", "Proceedings of SIGGRAPH/SIGPLAN Conference on Graphics Languages, published as Computer Graphics, Vol 10 No 1, April 1976, pp 61-65 Co-worker with Foley: Mentions virtual/simulated devices: relevant to Schumer, though not cited? Virtual devices (for semantic purposes) are Pick, Button, Keyboard, Locator, Valuator: specifically refers to character recognition for keyboard input: compare with Buxton virtual devices on a tablet. Early reference to logical equivalence of locator devices (tablet, touchscreen, mouse, joystick, etc.). Interactions/gestures include drag, rubber-banding, stroke (electronic "ink"). [Waltz78] Waltz, David L. "On the Interdependence of Language and Perception", U. Penn Technical Report T781020 Partly an essay on the ambiguity of perception as a function of context



[Ward76] Ward, J.R., Nelson, G.E., Desch, S.H., and Kaplow, R. "Two New Strategies for Computer-Assisted Language Instruction (CALI)", Foreign Language Annals, Vol 9 No 1, February 76, pp 28-37 CAI Computer-aided instruction, grammatical/syntax analysis of German sentences: Gordon Eugene Nelson, Jean Renard Ward, Robert Kaplow (see Kaplow patent for UI item) [WhetstoneA76a] Whetstone, Albert L. and Phillips, Stanley C. "Digitizing_graphic_system_using_magnetostrictive_transducers", United States Patent 3,956,588, May 11, 1976 Summagraphics magnetostrictive tablet technology on drafting table, two magnetostrictive wires one each along each arm of mechanical X/Y drawing holder [WhiteGM78] White, G.M. "Dynamic programming for the Viterbi algorithm, and low cost speech recognition (abstract only)", Proceedings of the 1978 IEEE International Conference on Acoustics, Speech, and Signal Processing, pp 413-417 Cited in Kruskal83 [WhiteJE76a] White, James E. "A High-Level Framework for Network-Based Resource Sharing", IETF RFC707: Early paper on making RPCs easier to use by standardizing message mechanisms for remote procedure calls: uses term Procedure Call Protocol. Separate error return message (e.g. exception). Marshalling is done by standardized parameter types: character strings, bit strings, integers, lists Cited in Kruskal83 [Wing78] Wing, A.M. "Response time in handwriting", in "Information processing in motor control and learning", New York: Academic Press, G.E. Stelmach, editor, 1978, pp 153-172 Cited in Teulings86 as that up/down stroke pairs may be basic grapheme unit of writing [Wing79] Wing, A.M. "Variability in Handwritten Characters", Visible Language, Vol XIII No 3, 1979, pp 283-298 Confusable forms of script b-f, i-r Printing slower than script, but gets faster with practice People have same writing style on blackboard and on paper [WrightSJ78a] Wright, Jr., S.J., Anderson, P.T., and Grimes, R.S. "Multi-Modal Data Input/Output Apparatus and Method Compatible with Bio-Engineering Requirements", United States Patent 4,070,649, January 24, 1978 Accellerometer stylus, Character recognition using a four segment panel, diagonally. Reference similar to BLRT chain codes. Used for wrist-worn portable calculator with separate (wireless) accellerometer stylus. [Xerox76a] Xerox Corporation "Alto User's Handbook 15a", Xerox Corporation, 1976 User manual for Xerox Alto personal computer: three-button mouse, refers to operations when "hold it down" on a mouse button, prior to moving: press-and-hold? [Xerox78a] Xerox Corporation "The Alto User's Primer", Xerox Corporation, June 1978 Overview of operational procedures in Alto operating system: backups, initializing disk volumes, etc. [Xerox79a] Xerox Corporation "Alto User's Handbook September 1979", Xerox Corporation, September 1979 User manual for Xerox Alto personal computer. Sections Alto Non-programmer's Guide; Bravo Manual - text editor; Laurel Manual - e-Mail electronic messages; Markup User's Manual, Draw manual - illustrations and drawing; FTP manual; Neptune manual - file management. Printing support. [Xerox80a] Xerox Corporation "Xerox 5700 Electronic Printing System Reference Manual", Xerox Corporation, 1980 Touch-panel / touch-screen interface for commercial printing system Xerox 5700. Virtual buttons: function of button depends on sequence of user input e.g. tapping on "help" button first changes function of next button user presses. Buttons displayed change depending on function. [Xerox80b] Xerox Corporation "Welcome to Alto Land: Stanford ALTO User's Manual", Xerox Corporation, 1980 User manual for Xerox Alto personal computer. Includes descriptions of "chat window" user interface in FTP GUI. [Yacyk78] Yacyk, J. "Alphabetic Handprint Reading", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-8 No 4, pp 279-282, April 1978 Constraints do not work well on handprinting in practice: Users tend to omit serifs from O and S in ANSI74 standard of constrained writing styles. Review of OCR for ANSI-standard handprints, shows Universal Subset standard neographic shapes for hand-printed characters and symbols. Tersoff paper is on same page [YamamotoK78] Yamamoto, K. and Mori, S. "Recognition of hand-printed characters by outermost point method", Proceedings of Fourth International Joint Conference on Pattern Recognition, Kyoto, Japan, November 7-10, 1978, pp 794-796 Cited in Kahan87 [YamamotoK80] Yamamoto, K. and Mori, S. "Recognition of Hand-printed Characters by an Outermost Point Method", Pattern Recognition, Vol 12., pp 229-236, March 1980 Similar to BLRTs chain codes, but uses 16 directions, not 4 Refers to dis-ambiguation of Katakana and Roman characters [YasuharaM76] Yasuhara, M. "On Distinctive Features in Character Recognition -- Some Evidence from Reactiontime Measurements", Working Paper 102, Cognitive Information Processing Group, Research Laboratory of Electronics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, August 15, 1976



Mentions lack of consensus on what features to use in any given pattern recognition problem [YasuharaM77a] Yasuhara, M. and Yasumoto, Yasuhiko "Measurement of Handwriting Pressure - PressureSensitive Pen", Annual Report of RLCS Avoid feature extraction in signature verification by time-template matching: compare elastic matching [YasuharaM77b] Yasuhara, M. and Oka, M. "Signature Verification Experiment Based on Nonlinear Time Alignment: A Feasibility Study", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-7 No 3, pp 212-216, March 1977 Signature verification using pressure-sensitive stylus on tablet: not clear that it uses pressure information, focuses on elastic time matching to allow for speed/velocity variations while writing. [YasuharaM78] Yasuhara, M. and Kuklinski, T. "Category boundary effect for grapheme perception", Perception and Psychophysics, Vol 23 No 2, 1978, pp 97-104 Discrimination function peaks at the category boundary, just as speech does Boundary classification for character recognition Ted Kuklinski and Yasuhara: says say "physical attribute" and "functional attribute", since "feature" is used sloppily Shows ambiguity of feature "leg" along character trajectory, instead of character along feature trajectory Cultural bias: l vs r in speech by Japanese, and by Americans (extension to characters: European 7 vs F (?)) [Yoshida78] Yoshida, Masumi; Iwata, Kiyoshi; Yamamoto, Eiichiro; Masui, Takeshi; and Kabuyama, Yukikazu "Pattern Recognition Processing System", United States Patent 4,105,998, assigned to Fujitsu Limited, Kawasaki, Japan, August 18 1978 Recognizing handwritten characters by their intersections within a rectangle with horizontal and vertical reference lines; used the distances and positions of the intersection points, such as to get at the internal "open" space within a character [Yoshida80] Yoshida, M. et al "Recognition System for Design Chart Drawn on Section Paper", Proceedings of 5th International Joint Conference on Pattern Recognition, 1980, pp 127-130 Cited in Kato80: user-interface for prettying up drawings [YouKC79] You, K.C. and Fu, K.S. "A syntactic approach to shape recognition using attributed grammars", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-9, 1979 biblio75 (Prev page)

Main Page







Main Page


[ABAJournal81a] ABA Journal "Computer Add-Ons: Micropad handwriting terminal", ABA Journal, May 1981, Vol 67, page 625 Product information announcement about Micropad handwriting-recognition terminal (see file) from Micropad, Inc.: Micropod is a local or remote input terminal which captures handwritten data at the time of writing, translating It into machine readable form. [ANSI81] ANSI "ANSI X3H31 Status Report on the H31 Strawman Proposal on Input Functionality", Document X3H31/31-11R1, April 20, 1981 [ActaPsychologica83] Acta Psychologica "Special issue on motor aspects of handwriting", Vol 54, 1983 [Agui81a] Agui, T., Matsubara, K., and Nakajima, M. "Sequential Computer Processing of a Collection of Closed Curves and Its Application to Pattern Recognition", Trans. of the IECE of Japan, Vol E64, No 10, October 1981, pp 661-666 (abstract only) Chain-code transformation, for matching similar curves [Amin82] Amin, Adnan, and Masini, Gerald "Machine recognition of cursive Arabic words", SPIE Vol 359, Applications of Digital Image Processing IV, 1982, pp 286-292 Dictionary lookup for word-by-word handwriting recognition of Arabic Segmentation: some Arabic letters are discontinuous, so Arabic words are 1 to 7 strokes (not counting dots?) Handwriting spelling dictionary look up by number of strokes, number of dots, number of intersections in Arabic Cited in FoleyJD82 Input model: precursor to PHIGS and CGI, postcursor to SIGGRAPH CORE? [Anson82] Anson, E. "The Device Model of Interaction", Computer Graphics, Vol 16 No 3, July 1982, pp 107-114 Critique of CORE graphics standard, need for composite devices. Compare with proposal by Pencept for handwriting recognition input to PHIGS standard. Model supports multiple (simulated) devices, two-handed input, contrasted with ping-pong input of one device at a time. Refers to user-interface of simulating a virtual soft keyboard /function buttons on a tablet [Apple83a] Apple Computer "Apple Modem 300/1200 User's Manual", Apple, Inc. 1983 (hardcopy book) Modem command set for Hayes-compatible Apple modem [Applicon83] Applicon "AGL User's Guide", Applicon Incorporated, Burlington, Massachusetts, June 1983



cited in RubinSM84: Ledeen recognizer? [ArakawaK83a] Arakawa, K. "On-line recognition of hand-written characters -- Alphanumeric, hiragana, katakana, kanji", Recognition, Vol 16 No 1, pp 9-16, 1983. Published earlier as (Arakawa78) Fourier coefficients of strokes as feature points For Roman alphabet, 29 single-stroke and 7 two-stroke character shapes For Hiragana alphabet, 10 single-stroke and 18 two-stroke character shapes, 13 3stroke, 5 4-stroke [BabbGR82] Babb, Gerald R. and Kuklinski, Theodore T. "Pattern Algorithm Permits Freehand Printed-Data Entry", Computer Technology Review, Winter 1982 Shows basic features of first Pencept product: single-stroke and multi-stroke forms, boxed input [Badie82] Badie, K. and Shimura, M. "Machine Recognition of Roman Cursive Scripts", Proceedings of the 6th International Conference on Pattern Recognition, pp 28-30, 1982 Refers to similarity/variability of loop and arc in script handwriting Script recognition using clock-wise vs counter-clockwise loops and arcs Refers to corner(cusp)/loop transition in script writing [Bahl81] Bahl, L.R. and Cocke, J. "Font-Independent Character Recognition by Cryptanalysis", IBM Technical Disclosure Bulletin, Vol 24 No 3, August 1981, pp. 1588-1589 Cites Baum-Petree algorithm for decoding Font- and language-independent recognition by doing cryptanalysis on whatever categories and arbitrary recognition comes up with? [Bahl83] Bahl, L.R., Jelinek, F., and Mercer, R.L. "A Maximum Likelihood Approach to Continuous Speech Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5 No 2, March 1983, pp 179-190 Realistic testing: artificial tasks vs natural tasks (for a priori grammar for speech) Speech: acoustic/phonetic encoding vs communication theory model [BarachDR81] Barach, David R, Taenzer, David H, Wells, Robert E. "Design of the PEN Video Editor Display Module", ACM conference proceedings: ACM 0-89791 050 -8/81/0600/0130 Video text editor to emulate different computer keyboard terminals, with separation of data and display modules: does not involve pointing/pen or character recognition input (Copy provided by Gary Odom) [BarkerPG82] Barker, P.G. "Data base interaction using a hand print terminal", International Journal of ManMachine Studies, Vol 17, 1982, pp 435-458 DCR Dynamic Character Recognition devices allow several novel kinds of user interactions / interfaces. Says needs much higher resolution digitizer touch-pad than is available. [BartelsRH83a] Bartels, Richard H.; Beatty, John C.; Booth, Kellogg S; and Field, Daniel E. "Computer Graphics Laboratory Fall 1983 Review, Univ. of Waterloo", University of Waterloo, Computer Science Department, Report CS-83-33, December 1983 GTCO Digi Pad 5 Pressure Pen Tablet; comments on lack of smarts used in microprocessors for functionality of pointing devices: example is buffering of data in pointing device to avoid data over-run from host buffering-- does not comment on latency. Labanotation input for dance transcription (Benesh editor) [BewleyWL83a] Bewley, William L.; Roberts, Teresa L.; Schroit, David; and Verplank, William L. "Human Factors Testing in the Design of Xerox's 8010 "start" Office Workstation", Proc. CHI '83, December 1983, pp. 72-77 User interface usability testing of Xerox Star GUI, primarily on use of mouse buttons, 1/2/3-button mouse: separate button for drag/copy/select, multiple clicks (double click, etc.) Drawthrough gesture for selection: not quite press-and-hold. [Biswas81] Biswas, Prasenjit and Majumdar, Arun K. "A Multistage Fuzzy Classifier for Recognition of Handprinted Characters", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-11 No 12, December, 1981, pp 834-838 Test results on 12 of the 35 Devanagari alphabet handwritten characters (all work done by hand: did not have a real computer?) Fuzzy set classifier for Devanagari (Indian) handwriting recognition Data collection: tested on the 70% "satisfactory" sample writing sheets: bias in sample? points out subjectiveness of training? (pairwise comparison?) syntactic method only applied to handwritten characters which were confusable [Blesser83] Blesser, B. "Multistage Digital Filtering Utilizing Several Criteria", United States Patent 4,375,081, February 22, 1983 Low-pass digital filter for a tablet to pre-process handwritten character to eliminate wobble/digitization noise before recognition [Bozinovic82] Bozinovic, Radmilo and Srihari, Sargur N. "A string correction algorithm for cursive script recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 4, November 1982, pp 655-663 Handwriting cursive script recognition using channel model of dictionary: allows for splitting, merging, and substitution (segment parsing errors). Cited in Bozinovic89 for context: probabilistic spelling corrector based on a distance metric for strings. [BrodyH83a] Brody, Herb "Machines that Read Move Up a Grad", Hight Technology, Feb 1983, pp 35-ff Review of commercial OCR products and applications (document sorting), including Kurzweil, [BrownF83] Brown, Frances "The acquisition of handwriting in the UK", Oral presentation at the Criminalistic



Institute in Prague, 1983. Written form available at http://www.bham.ac.uk/english/bibliography/handwriting/new_web_pages/acquisition.htm Report on handwriting styles taught and learned in the U.K. Points out that there is no single style of handwriting taught in copybooks in the U.K., unlike some other countries. Advisor, Tom Davis, comments on the diversity of writing styles in signature verification and the detection of forgeries [BrownMK83] Brown, M.K. and Ganapathy, S. "Preprocessing Techniques for Cursive Script Word Recognition", Pattern Recognition, Vol 16 No 5, pp 447-458, 1983 adaptive/trainable recognition, suggests "closed loop" verification (feedback to user) is helpful Much DCR/CSR handwriting/gesture recognition research is commercial/proprietary, therefore not published. DCR (gesture/handwriting input) picking up as result of human factors in Man/Machine interface. Writer (author) independent recognition would be more desirable than present trained recognition. In development, necessary to hand-correct for real-world errors in input data collection. [BrownRM83] Brown, Robert M., and Cheng, C.F. "Optical Character Recognition for Automated Cartography: The Advanced Development Handprinted Symbol Recognition System", Naval Ocean Research and Development Activity, NSTL Station, Mississippi, Report No NORDA-TN-187, March 1983 NTIS citation index [Buckle81] Buckle, Derek and Strand, Timothy D. "Processing of Information", United States Patent 4,262,281, assigned to Quest Automation Limited, Dorset, England, April 14, 1981 Quest Automation / Datapad product patent on handwriting recognition: Claims on alignment/registration of paper on digitizer tablet by marking on pre-printed alignment targets. Pressure-sensitive tablet with floating hand rest to keep hand from pressing on tablet. See also Micropad [BurrDJ81a] Burr, D.J. "Elastic matching of line drawings", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-3, March, 1981, pp 708-713 Handwritten sketch/character input recognition: refers to IEEE handwriting data base collection 1.2.4: (24x36 binary OCR) Like Greenberg77, throws out handwriting samples from IEEE Database 1.2.4 as "poorly written" [BurrDJ83a] Burr, D.J. "Designing a Handwriting Reader", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5 No 5, pp 554-559, September 1983 Context by matching grouped letters into words using Unix "spell". Separate "training dictionary" for each user: single-stroke characters, discrete characters. Segmentation by pen-lift only, therefore unistroke only: "t" "i" etc. must be made with one stroke. [Buxton82] Buxton, William "An Informal Study of Selection Positioning Tasks", Graphics Interface '82, pp 323-ff trainable, short-hand single-stroke symbols for graphical shapes, graphics editing: Unistroke, terminated by pen lift at the end. Gestures are checkmark, downward stroke, and upwards stroke. [Buxton83] Buxton, William, Fiune, Eugene, Hill, Ralph, Lee, Alison, and Woo, Carson "Continuous hand-gesture driven input", Proceedings of Graphics Interface 83, page 191-195, 1983 Gesture input: tap/circle gesture. Says velocity-sensitive gestures difficult Cited in Rhyne86: Digitizer stylus must be held vertical to get good results -- reference to stylus tilt problem, often miss-called parallax. [Buxton83a] Buxton, William "Lexical and Pragmatic Considerations of Input Structures", Computer Graphics, January 1983, pp 31-ff Lexical (chunking and phrasing) aspects of compound gestures [Buxton83b] Buxton, William "Towards a Comprehensive User Interface Management System", Computer Graphics, Vol 17, No 3, July 1983 MENULAY/MAKEMENU system: User-interface editor, user can sketch user interface and then associate actions with user interface elements. Uses four-button puck on graphics tablet. [BuxtonWB83c] Buxton, William "Etch: A Study in Marking-Based Interaction", University of Toronto, 1983: available at billbuxton.com Video demonstration to accompany "Continuous Hand-Gesture Driven Output". Complexity of marking menus can be reduced by structuring the dialogs. [BuxtonWB83d] Buxton, William "Menulay: A UIMS for Rapidly Sketching Prototyping and Implementing User Interfaces", University of Toronto, 1983: available at billbuxton.com Video demonstration to accompany "Towards a Comprehensive User Interface Management Systems. System for defining user interfaces using sketching and marking menus. [Byte82a] Williams, Greg "Title Article: The IBM Personal Computer -- A Closer Look at the IBM Personal Computer", BYTE Magazine, Vol 7 No 1, January 1982 Overview of IBM personal computer. IBM support both CP/M-86 and MSDOS. "A Closer Look at the IBM Personal Computer" (Gregg Williams). IBM Basic (GW/Basic). Also articles on Cromix (Unix OS clone), Atari graphical smooth scrolling implementation. HIPAD tablet digitizer from Houston Instruments, with keyboard simulation. [Byte83a] BYTE Magazine "Special Issue: Inside the IBM PC", BYTE Magazine, Vol 8 No 11, November 1983



Detailed hardware and software description of features of the IBM personal computer. MS/DOS 2.0 installable drivers, background tasking. Pencept PenPad text/graphics with handwriting recognition. IBM Kana/Japanese keyboard / IBM 5550 system keyboard. "Big Blue goes Japanese" Has pictures of the complexity of keyboards for Japanese and Chinese characters. Article on Concurrent CP/M operating system [CTS81] CTS Recognition "Telepad product information", 13-14 Golden Square, London W1R 3AG, England, 1981 Early Micropad-like British handwriting recognition product. [CalComp83] CalComp "CalComp 2000 Series Digitizer Operator's Manual", 50218-1, page 15, CalComp Incorporated, January 1983 [Carau81] Carau, F., Hetzel, H. and Tremblay, M. "Travelling Wave Digitizer", United States Patent 4,255,617, assigned to Hewlett-Packard Co, Palo Alto, California Digitizer measuring X and Y sequentially in time. Mentions velocity correction for bowing of diagonal lines, corrections for non-orthogonality, course and fine measurements in two stages. X/Y Grid, puck with magnifying glass shown. [CardSK83a] Card, Stuart K.; Moran, Thomas P.; and Newell, Allen "The Psychology of Human-Computer Action", Lawrence Erlbaum Associates, 1983 (hardcopy book) User interfaces discussed in terms of keyboard interactions. One section on Grafcon tablet: section on pointing/selection speed only compares joystick vs. mouse. [Carey83] Carey, Tom "User Differences in Interface Design", IEEE Computer, November 1982, pp 14-ff Early essay on user interface design for varied users: range to be dealt with is not just beginner/intermediate/experienced users, also range of learning styles (hands-on versus read the documentation, visual learners versus text learners). [Carroll82] Carroll, John M. "The Adventure of Getting to Know a Computer", IEEE Computer, November 1982 User interfaces better if friendly: no unfriendly error messages. Also mentions discoverability in user interface design. [Casey82] Casey, R.G. and Nagy, G. "Recursive Segmentation and Classification of Composite Character Patterns", Proceedings of the 6th International Conference on Pattern Recognition, Munich, Germany, October 1982, pp 1023-1026 Combine character segmentation with classification in adaptive decision tree Optical resolution good enough for recognition may be too low to segment characters (like cursive/connected vs discrete characters segmentation): some touching characters in OCR can only be segmented by recognizing component characters [Casio83] Casio "Module No. 658 Wrist-watch Controller", Casio, 1983 Touchscreen controller for Casio wristwatch. Electronic file includes additional material on VDB-1000, Casio Databank with small keyboard on wristwatch. Very early Smart-Watch. User manual for Casio wristwatch: may be the model with touchscreen input and handwriting recognition, manual describes small keyboard input. [ChalkBoard83a] Devlin, Joe "The legend of the pad of power", Creative Computing, Vol 9 No 10, October 1983, page 52 Chalkboard Power Pad: touch-sensitive membrane graphics tablet responding to more than one touch at a time, can image contact of entire hand. Resolution 120x120. Sold as educational toy. multi-touch capable? [ChalkBoard83b] Crowell, Gregory William "Chalkboard PowerPad graphics tablet", Chalkboard Inc, 1983: at Personal Computer Museum, Brantford Ontario, Canada (2011) Chalkboard Power Pad: pressure-sensitive membrane graphics tablet responding to more than one touch at a time. Resolution 120x120. Sold as educational toy. Pressure-switch touch graphics tablet, 14,400 sensor points for 10x10 resolution on 12x12 inch surface. "Has no problem resolving simultaneous multiple inputs" (multi-touch), "software stretching" to interpolate (?) to higher resolution. [Cognex83a] Cognex Corporation "Cognex Corporation presents DataMan", Cognex Corporation, 1983 Date approximate: OCR product for reading serial numbers and other identifying numbers in manufacturing and shipping, such as product IDs on boxes. [ComputerGW82] Computer Graphics World "Digitizer Survey", Computer Graphics World, July 1982, pp 66-69 Vendor survey in 1982 for digitizing tablets, video scanner digitizers. [Computerworld81a] Computerworld "Micropad: Data Entry Unit Accepts Writing", Computerworld, July 6, 1981, p. 52 Announcement of Buffered Micropad handwriting recognition input terminal, 512 characters, built-in oneline LED display. [Computerworld82a] Computerworld "Image Data Tablet System", Computerworld, January 18, 1982, page 14 Image Data Tablet System: Tablet and monitor with handprint (handwriting) recognition, mathematical calculation, graphics mode input, on-tablet keyboard icons. May be Pencept PenPad 200? [ConklinD83] Conklin, Dick "PC Graphics (excerpt)", Wiley IBM PC services, ISBM 0-471-89207-6



Early reference to digitizing tablets with IBM PC: rotational transformation, digitizer resolution much higher than display: typical resolution of tablet 2048 points (sic) [CooperLN81a] Cooper, Leon N., and Elbaum, Charles "Information Processing System using Threshold Passive Modification", United States Patent 4,254,474, March 3, 1981, assigned to Nestor Associates, Stonington, Connecticut Nestor Graphics patent [CooperLN82a] Cooper, Leon N. and Elbaum, Charles "Curve Follower", United States Patent 4,319,331, March 9, 1982, assigned to Nestor Graphics, Providence, Rhode Island Nestor Graphics patent Curve follower to turn OCR data into DCR character data [CooperLN82b] Cooper, Leon N., Elbaum, Charles, and Reilly, Douglas L. "Self Organizing General Pattern Class Separator and Identifier", United States Patent 4,326,259, April 20, 1982, assigned to Nestor Graphics, Providence, Rhode Island., also European Patent Application 81300559.2 Nestor Graphics patent [Cordella83] Cordella, L.P. and Sanniti di Baja, G. "Structural description of silhouettes", Proceedings of the 3rd Scandinavian Conference on Image Analysis, Copenhagen, Denmark, 1983, pp 73-78 Cited in Arcelli85: about line-thinning and reconstruction [Cox82] Cox, C.H. III, Coueignoux, P., Blesser, B., and Eden, M. "Skeletons: A Link Between Theoretical and Physical Letter Descriptions", Pattern Recognition, Vol 15 No 1, PP 11-22, 1982 Barry's group: functional attribute (cognitive) vs synthetic (generative) Deal with embellishments separates from base pattern (in OCR) [CraneHD82a] Crane, H.D. and Wolf, D.E. "Dynamic Creation of Signatures", United States Patent 4,344,135, August 10, 1982, assigned to Stanford Research Institute, Menlo Park, California Hew Crane: amendment to Crane79: patent 4,156,911 (?) Human-reading for signature verification [CraneHD83a] Crane, H.D. and Ostrem, J.S. "Automatic Signature Verification Using a Three-Axis Force-Sensitive Pen", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-13 No 3, May 1983, pp 329-337 Writing pen with obstructed view: accelerometer (no touch screen) [DahlJP81a] Dahl, Jerome P.; Epley, Phillip R.; and Fox, Jon E. "Program Controlled Capacitive Keyboard Variable Threshold Sensing System", United States Patent 4,305,135, December 8, 1981 Capacitive key matrix keyboard, references touch-sensing circuit in IBM journal [DamerauFJ82a] Damerau, Frederick J. "Guess-ahead feature for a keyboard-display terminal data input system", United States Patent 4,330,845, May 13, 1982 Auto-completion of typed text: if what is typed so far matches text in a list, monitor shows rest of that text without advancing cursor, user can type over or hit a function key (right arrow) to accept. Compare with auto-completion in MAC LISP, word prediction in accessibility on-screen keyboard software? [DenningDE81a] Denning, Dorothy E. and Sacco, Giovanni Maria "Timestamps in Key Distribution Protocols", CACM vol 24 no 8, August 1981, pp. 533-536 Use of timestamps in key-exchange algorithms and signatures to prevent re-plays. (Compare with use of nonce). [DooijesEH83a] Dooijes, E.H. "Analysis of Handwriting Movements", Acta Psychologica, Vol 54, 1983, pp 99-114 Frequency-domain analysis of handwriting movements in X and Y: Lissajous analysis [DosterW83a] Doster, W. and Oed, R. "Zur Bildanalyse bei der Handschriftlichen Direkteingabe", Proceedings of Mustererkennung 1983, October 11-13, 1983, Karlsruhe, West Germany Best description of AEG's segmentation/parsing algorithm for handwritten characters Blithely claims that rasterization of dynamic on-line character data would reduce this with OCR to the same problem (but the OCR segmentation problem is harder!) [DosterW83b] Doster, W. and Schuermann, J. "A Step Towards Intelligent Document Input to Computers", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 19-23, 1983, Washington, D.C., pp 515-516 Scan in documents with OCR, use on-line handwriting recognition user-interface to edit them [Downton83] Downton, A.C., Baker, R.G., Lewis, S.M., and Cooper, P.J. "Readability measurements of Palantype transcription for the Deaf", International Journal of Man-Machine Studies, Vol. 19, pp 575-594, 1983 [DunnK81a] Dunn, K. "Choose Digitizer Technology and Features to Suit Applications", Computer Technology Review, Fall/Winter 1981, pp 171-175 Many limiting characteristics of digitizers (width of pencil line, etc.). List of what can be spec'ed for a digitizer performance (but not the trade-offs). [DunnK81b] Dunn, K. "Understanding digitizer resolution and accuracy", Mini-Micro Systems, December 1981 Differential linearity error is a digitizer's worst enemy; Clumping and stretching of digitizer co-ordinates for X-vs-X non-linearity; Digitizer accuracy vs digitizer stability; General review of digitizer characteristics and proper specifications [Elliot82] Elliott, B.J. "Apparatus for Determining Pen Acceleration", United States Patent 4,345,239, August 17,



1982, assigned to International Business Machines, Armonk, New York Capacitive sensor for measuring pen/stylus acceleration for signature verification [Embley81] Embley, D.W. and Nagy, G. "Behavioral Aspects of Text Editors", A.C.M. Computing Surveys, Vol 13 No 1, March 1981, pp 33-70 Cites gaps in human factors/engineering and cognitive psychology literature re text editing user-interface; human factors of using digitizing tablets for text editing Cites "taste for federal funding" as reason some human factors studies have skipped various interests: research selection bias Users frequently will not use mouse in text editing: (we say "3-handed monkey" effect): three-handed monkey Tablet: pointing with light pen fatigueing, and not accurate enough [EvansKB81] Evans, Kenneth B.; Tanner, Peter P.; and Wein, Marceli "Tablet-based Valuators that Provide One, Two, or Three Degrees of Freedom", Computer Graphics, Vol. 15 No. 3, August 1981, pp. 91..97 Digitizer tablet simulation of virtual input devices: turntable, multiple number wheels, three-axis trackball [Fairhurst82a] Fairhurst, M.C. "Image Characteristics as Assessment Criteria for an Electronic Writing Aid", Proceedings of International Conference on Man-Machine Systems, July 1982, pp 191-195 Claims better test results for machine recognition than for human reader (!) [Fairhurst82b] Fairhurst, M.C, and Maia, M.M. "An Approach to Machine Reading of Text with a Memory-based Character Recognition System", Colloquium Proceedings "Coding of Documentary Information", University of Kent, Canterbury, England, May 1-2, 1982 Discusses pairwise discriminators to reduce memory requirements for OCR of typed text: Similar to Pencept? [FieldsC83] Fields, Craig I. "Virtual Space Teleconference System", United States Patent 4,400,724, August 23, 1983 Not a whiteboard system: video conferencing arrangement with each user facing one monitor and one video camera per other user. [Filipski81] Filipski, Alan J. "Critical-point Representation of Hand-printed Numerals", Proceedings of International Conference on Cybernetics and Society, IEEE 0360/8913/81/0000-0198, 26-28 October, 1981, Atlanta, Georgia, pp 198-202 96% correct on handprinted OCR recognition: skeletonizing, "critical point" strokes with initial and final slope features for 18 types of stroke segments. Admits to and spells out several weaknesses: 4 vs 9, patching OCR skeletonization breaks Handwritten OCR recognition: simple template matching works well if feature extraction is good Critical points for features and segmentation in OCR handwriting: points of high curvature, then Freeman chain codes Refers to Knoll Database of hand-printed numerals, 21x25 binary grid, IEEE Pattern recognition data base 1.2.2) [FoleyJD81] Foley, J.D., Wallace, V. and Chan, P. "The Human Factors of Interaction Techniques", George Washington University, Institute for Information Science and Technology Technical Report GWU-IIST-81-03, Washington, D.C., 1981 Precursor to FoleyJD84 [FoleyJD82] Foley, J.D. and VanDam, A. "Fundamentals of Interactive Computer Graphics", Addison-Wesley, Reading, Massachusetts, 1982 (hardcopy book) Three-point calibration, general graphics reference Comparison of tablet and mouse "locator" devices, absolute position versus relative motion. Section 5.3 "Technique for simulating logical devices": "locator device simulation" / virtual devices, simulated keyboard, character recognizers. Says Ledeen recognizer simplified version of Teitelman 1964, described in Newman 199 [Freyd83] Freyd, J.J. "Representing the dynamics of a static form", Memory and Cognition, Vol 11, 1983, pp 342346 Tappert's bibliography [Fu81] Fu, K.S. "A survey on image segmentation", Pattern Recognition, Vol 13, 1981, pp 3-16 Fu86/Fu83 [Fukushima83] Fukushima, K., Miyake, S., and Ito, T. "Numeral Character Recognition by the Algorithm of the Neocognitron", Trans. of the Institute of Electronic and Communications Engineers of Japan, Vol J66D No 2, February 1983, pp 206-213 NTIS index: neural network for handwritten numerals recognition, using a scanner: learning-with-a-teacher better than learning-without-a-teacher (training set makes a difference, but what?) [Furuta82] Furuta, Richard, Scofield, Jeffery and Shaw, Alan "Document Formatting Systems: Survey, Concepts, and Issues", Computing Surveys, Vol 14, No 3, September 1982, pp 417-ff Very little treatment of UI issues or pointing devices: primarily deals with keyboard commands and display of monofont text [GTCO82] GTCO "Digi-Pad 5 Family Data Sheet", DP5-L104-0782, GTCO Corporation, Rockville, Maryland, 1982 GTCO Electronic grid digitizer, serial interface: pressure-sensitive pen/stylus, 16-button puck/cursor [Gander83] Gander, Stephen Joseph "A proposed method of handwriting recognition", S.B.E.E. Thesis, M.I.T.,



1983 (abstract only) Not from Blesser group [Gehani82] Gehahi, N. "The Potential of Forms in Office Automation", IEEE Trans. Communication, Vol 30 No 1, January 1982 Gives many reasons for using forms as fundamental office automation model. (taken from Hekmatpour86). Use of forms eases transition from manual to office automation. [Geyer81] Geyer, L.H., and Gupta, S.M. "Recognition/confusion of dot matrix vs conventional font capital letters", Perception and Psychophysics, Vol 29, 1981, pp 280-282 Cited in Suen86 Suen86 cites this on what matrix resolution needed for OCR on hand-print [GlickmanD82a] Glickman, David, Greanias, Evon C., Repass, James T., and Rosenbaum Walter S. "Stem Processing for Data Reduction in a Dictionary Storage File", United States Patent 4,342,085, assigned to International Business Machines Corporation, Armonk, New York, July 27, 1982 Stem processing for data reduction in dictionary storage file storing word list file with prefix and suffix truncated so that only unique root element remains [Goodale83] Goodale, T.S., Goyal, S., and Litvin, Y. "Designing a Text Editor with Graphic and Handwritten Input", Report TR 83-401.1, GTE Laboratories Incorporated, 40 Sylvan Road, Waltham, Massachusetts 02254, November 1983 (partial copy) Cited in Litvin89 Gesture recognition for a text editor? Handwritten text input? Gesture-based user interface with handwriting recognition for editing text with electronic ink: first reference for prior art? Early reference to visual parallax on electronic ink with integrated tablet/display. [Gould83] Gould, J., Conti, J., Tovanyecz, T. "Composing Letters with a Simulated Listening Typewriter", CACM, pp 295-308, Vol 26 No 4, April 1983 Simulation of "perfect" speech recognition, showing that there is indeed a user-interface problem in addition to recognition problems. [Greanias82] Greanias, Evon C. and Yhap, Ernesto F. "Chinese/Kanji On-line Recognition System", United States Patent 4,365,235, December 21, 1982, assigned to International Business Machines Corporation, Armonk, New York Chinese recognition by recognizing component strokes/radicals Refers to "spelling" of Chinese/Kanji: order strokes/radicals are written in States that strokes are classified into 42 categories for segmentation, and that Chinese/Kanji characters are made up of only 72 basic symbol elements/alphabet Cites problem with sonic/acoustic digitizers: sensing point offset from writing point If Kanji/Chinese character is not recognized, user can add it to the prototype set for recognition on the fly [Greer83] Greer, K.L., and Green, D.W. "Context and Motor Control in Handwriting", Acta Psychologica, Vol 54, 1983, pp 205-215 Digitizer specifies +-.015" on 0.001 resolution, but also +-.01 accuracy [Gu83] Gu, Y.X., Wang., Q.R., and Suen, C.Y. "Application of a Multilayer Decision Tree in Computer Recognition of Chinese Characters", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5 No 1, January 1983, pp 83-89 Features for Chinese character recognition are Walsh coefficients, projected onto X and Y axes: character alignment (normalization), 99.5% accuracy on 3000 characters OCR on Chinese/Kanji characters Using a binary decision tree for pattern recognition of a large number of classes (Chinese characters OCR) Performance: noisy characters only: clean characters do not occur in real life [GurolIM83a] Gurol, I. Macit and Bang, Gary M. "Touch Sensitive Transparent Switch Array", United States Patent 4,423,299, December 27, 1983 Transparent switch matrix for touchscreen, conformable to a display: grid of conductors separated by spacer dots, pressing on top sheet/membrane closes switch of two conductors. [HP83a] Hewlett-Packard "HP-150 touchscreen Personal Office Computer product", (various) File contains reference for HP-150 touchscreen computer: MS/DOS personal computer (8088) with infrared touchscreen on 9-inch video display. Also known as Touchscreen MAX. No OS support, applications had to be written specifically for touchscreen input. [Haber81] Haber, R.N. and Haber, L.R. "Visual components of the reading process", Visible Language, Vol XV No 2, 1981, pp 147-181 Cited in Bozinvic89 Bozinovic cites for human recognition by outline of word (as shown in WrightG52) [Hagita83] Hagita, N., Naito, S. and Masuda, I. "Handprinted Chinese Characters Recognition by Peripheral Direction Contributivity Feature", Trans. of IECE of Japan, Vol J66D No 10, October, 1973, pp 1185-1192 (in Japanese: abstract only) OCR for Chinese using features of stroke structure, complexity, direction, connective relation, and relative location for 95.4% of block-style handprinted data [Hanaki81a] Hanaki, S. and Yamazaki, T. "On-line recognition of handprinted Kanji characters", Pattern Recognition, Vol 12, 1980, pp 421-429



Tappert's bibliography -- Chinese recognition [Hanaki81b] Hanaki, S., Temma, T., Yoshida, H., Arakawa, T., Suziki, M., Seki, T., and Kikuchi, Y. "Online Realtime Character Recognition System", Trans. of IECE of Japan, Vol E64 No 5, p 374, May 1981 (Abstract only) Science Citation Index Handprinted recognition: feature is break into piecewise segments, compare with decision tree: 32 handwriting terminals on one computer (cluster) [Hemenway82] Hemenway, K. "Psychological Issues in the Use of Icons in Command Menus", Proc. Human Factors in Computer Systems Conference, Washington, D.C., A.C.M., March 1982, pp 20-24 Cited in FoleyJD84 [HewlettPackard83a] Hewlett Packard "Hewlett Packard HP-150 Touchscreen Computer", wikipedia.org, fetched 2011 Early (DOS-age) touchscreen computer product. [HillisWD81a] Hillis, William Daniel "Active Touch Sensing", M.I.T. A.I. Memo 629, April 1981 High-resolution touch sensor, several hundred sensing points per square inch. Uses grid/matrix of aanisotropic elastomeric material: crosspoint problem addressed by grounding all other lines than the ones being sensed, so that no current flows. Shear forces determined by force needed to move the probe across whatever it is touching. [HillisWD82a] Hillis, W. Daniel "A High-Resolution Imaging Touch Sensor", Int'l Jnl. of Robotics Research, June 1982, Vol 1 No 2, pp. 33-44 (Abstract only) Imaging touchpad sensor for robotics using small pressure/force sensors. [HollerbachJM81a] Hollerbach, John M. "An Oscillation Theory of Handwriting", Biological Cybernetics, SpringerVerlag, Vol 39, 1981, pp 139-156 (see also Hollerbach78) Unclear fancier curve fitting to handwriting models gives any more insight Maximum writing speed/velocity is 25 mm/sec Acceleration peaks in handwriting not just a tablet/pen artifact Variation in slant in handwriting (vertical) is about 10 degrees Cusp/loop: substitution of clockwise vs counter-clockwise motion in handwriting Variability of corner shapes for single writer Says reduction in concentration on handwriting needed to think and write at same time [Honeywell81a] Honeywell Information Systems "Level 68 Introduction to Programming on Multics", Honeywell Information Systems AG90-03, July 1981 Multics operating system programming tutorial: special section on dynamic linking. All files are memorymapped into address space, no separate I/O. Mentions directe file types stream/sequential: stream is ascil file? APIs for access control ACLs. [Honeywell83a] Honeywell Information Systems "MULTICS Common Commands", Honeywell Information Systems G58-0 February 1983 Multics operating system CLI command-line commands: set_acl; copy command preservices ACLs access control lists; send_mail over ArpaNet. [Honeywell83b] Honeywell Information Systems "Inter-MULTICS File Transfer Facility Reference Manual CY7301", Honeywell Information Systems Order Number CY73-01, December 1983 Multics file transfer facility: describes I/O Daemon process. On target system for file transfer, ring access must be at higher numbered access level (more restrictive), I/O daemon must have ACL access rights, user on target/foreign system must grant access. Read/write access can be restricted by setting zero length for segment. AIM (capability) permission must be less or equal. [Hong82] Hong, T.H., Shneier, M., Hartley, R. and Rosenfeld, A. "Using pyramids to detect good continuation", University of Maryland, Computer Science TR 1185, 1982 Cited in Lipscomb91 [Hopfield83] Hopfield, et al "Unlearning Has a Stabilizing Effect in Collective Memories", Nature, Vol 304 pp 158159, 1983 Cited in Faggin89 [Hosaka81] Hosaka, M. and Kimura, F. "Use of handwriting action in construction of models", in Scientific Information Systems in Japan, H. Inose, editor, 1981, pp 83-90 Tappert's bibliography [Hosaka82] Hosaka, M. and Kimura, F. "Using Handwriting Action to Construct Models of Engineering Objects", Computer, Vol 15 No 11, November 1982, pp 35-47 User interface (fill in charts) for handwriting recognition input of engineering drawings (mechanical drafting, maps, NC control drawings) Features are passage through 3x3 template grid, then chord lengths and directions (cusp, stroke, rotation, straight): cusps turn into small loops, vice versa is a source of error [HowbrookE83a] Howbrook, E. "Apparatus and Methods for Recognizing Handwritten Signs", United States Patent 4,369,431, January 18, 1983, assigned to National Research Development Corporation, London, England Signature recognition/verification using first twelve initial segments, segmenting by zero velocity in Y Notes that a frequency cut-off of 20Hz is o.k. for signature signal [Hsu82] Hsu, W.S., Takahashi, K., Ozawa, S., and Fujita, H. "Ordered stroke extraction method for printed



Chinese character recognition", Trans. of IECE of Japan, Vol E65 No 2, February 1982, p. 140 (abstract only) NTIS abstract: fix skeletonization/line-thinning breakdown at intersections by simulating writing motion in OCR [Huber83] Humber, William A. "Interactive Map Information Exchange System", United States Patent, 4,420,682, December 13, 1983 Optical scanner which read digital co-ordinates from a map, using co-ordinates in magnetic ink, but also describes optical scanning [Huh82] Huh, Y.K. and Beus, H.L. "On-line recognition of hand-printed Korean characters", Pattern Recognition, Vol 15, 1982, pp 445-453 Tappert's bibliography [HullJJ82a] Hull, Jonathan J. and Srihari, S.N. "Experiments in text recognition with binary n-gram and viterbi algorithms", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-4, 1982, pp 520-530 Viterbi and binary n-gram for context in optical character recognition: most efficient implementation: Viterbi algorithm is based on probabilities of confusion of pairs of characters [HullJJ82b] Hull, Jonathan J. and Srihari, S.N. "Comparison of two contextual post-processing algorithms for text recognition", Proceedings of 1982 IEEE Computer Society Conference on Pattern Recognition and Image Processing, 1982, pp 146-151 (abstract only) NTIS abstract: looks very similar to other 1982 paper by same authors. [HullJJ83b] Hull, Jonathan J., Srihari, S.N., and Choudhari, R. "An integrated algorithm for text recognition: comparison with a cascaded algorithm", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5, 1983, pp 384-395 Letter substitution errors on text recognition corrected by various means of context: combining bottom-up and top-down (syntactic and semantic) context works better than separately Context for spelling correction: goodness measure extended to probability that it is a corrupted form of another letter Context for spelling correction: letter probabilities: digrams/pairs and trigrams (bottom/up context) (may involve a false assumption about input language) Context for spelling correction: dictionary lookup (lexicon for top/down context) Many sources for spelling errors: typographical in original text, keying/writing errors, character recognition error [HullsLR83a] Hulls, L. Robin "On-Board Intelligence Increases Accuracy of Plotters and Digitizers", Computer Technology Review, Summer 1983, pp 129-133 Numonics article on digitizers and stepper-motor plotters Digitizers: no point being more accurate than the application needs. accuracy may be affected by pen angle/tilt. two-phase coarse/fine position determination using two characteristics (interpolation?) [IBM81a] IBM Technical Disclosure Bulletin "Improved Parameter Set for Adaptive Symbol Recognition", IBM Technical Disclosure Bulletin, June 1981, pp 769-771 Copy on file does not give name of author: since it mentions "elastic matching", most likely it is C.A. Tappert Use offset of character from its center of gravity as a substitute for the writing baseline Add horizontal positions of strokes to each other as an additional factor in recognition: computer with Functional Attributes of Shillman/Blesser (example is "A" and three-stroke "asterisk") Also filed under Tappert and under Karnaugh [IBM81b] IBM "Liquid Crystal Display and Touch Panel Keyboard Input", IBM Technical Disclosure Bulletin, September 1981, pp 1888-1890 Cited in More93 [IEEE83a] IEEE CG&A "New Products: Software brings CAD capability to microcomputers", IEEE Computer Graphics and Applications, April 1983 Product brief on AutoCad: light-pen, or Sun-Flex touch pen (touchscreen digitizer) [Ikeda81] Ikeda, K. et al "On-line Recognition of Hand-Written Characters Utilizing Positional and Stroke Vector Sequences", Pattern Recognition, Vol 13 No 3, pp 191-206, 1981 Discrimination of similar characters: boundary recognition. Uses different methods for characters of different number of strokes [ImageData82] Image Data Products "Image Data Tablet System product information", Bristol, England, 1982 Digitizer tablet [Inforite82] Cadre Systems Limited "Inforite Hand Character Recognition Terminal product information", Cadre Systems Limited, 1 Wilkinson Road, Cirencester, Glos., GL7 1YT, England Early British handwriting recognition product: single line display, paper forms (shop invoices) fit into device, shown at Comdex 1982 Note: Inforite used as product name by other companies [IshiiK83a] Ishii, K. "Generation of Distorted Characters and Its Applications", Denshi Tsushin Gakkai Ronbunshi, Vol 66-D No 11, November 1983, pp 1270-1277 (in Japanese), translated in Systems, Computers and Controls, Vol 14 No 6, 1983, pp 19-27 Cites problem of needing very large training samples to get all variations Claims "only samples of low



quality can improve dictionary" (training) (vs. ambiguous?) Claims "recognition rate tells you nothing about how performance goes down with quality of characters" Shows artificial forms used in Japanese JIS handprint standard Not enough to read good characters, but how well does it do on bad ones? Claims 98.5% recognition rate Use artificial variability instead of real handwritten data (!) [JainR82a] Jain, R. and Haynes, S. "Imprecision in Computer Vision", Computer, Vol 15 No 8, August 1982, pp 39-48 Goodale83. Application of fuzzy set theory to visual recognition / OCR: imperfect lighting, other imprecise measurements. Variation in connected regions, thinning / edge detection. [JohnsonEO81] Johnson, E.O. and Tosima, S. "Visual-Perception-Related Effects in Chinese-Japanese Written Characters", RCA Review, Vol 40, March 1981, pp 60-ff (partial copy) Human recognition features: visual groups of strokes in Chinese/Japanese characters consist of subgroups of strokes, usually about four: visual group count corresponds to letter count in Western languages [JordanDM83a] Jordan, David M. "Multics Data Security", Scientific Honeyweller vol 2 no 1, June 1981. Derived version 1983, Honeywell document GA01 Describes finer control and finer parsing of access control in Multics with AIM Access Isolation Mechanism (nondiscretionary access control) in stricter set of rules for matches between segment and user attributes. AIM is to prevent user from releasing data to which user may have access via ACL: User's access is the more restricted of the two: clearance (of user) vs. classification level (of data). [Kaleyeh83] Kaleyeh, J.M. and Landgrebe, D.A. "Predicting the Required Number of Training Samples", IEEE Trans. Pat. Anal. and Mach. International, November 1983, pp 664-667 Cited in Meisel85 [Kamran83] Kamram, A. and Feldman, M.B. "Graphics Programming Independent of Interaction Techniques and Styles", Computer Graphics, Vol 17 No 1, January 1983, pp 58-66 Describes GMU's Information Display Systems project (see Foley). Critique of CORE (and GKS) input device model, lack of extensibility [Karnaugh81] Karnaugh, M., Kurtzberg, J.M., and Tappert, C.C. "Improved Parameter Set for Adaptive Symbol Recognition", IBM Technical Disclosure Bulletin, Vol. 24, No. 1B, pp. 769-771, 1981 Cited by Marlin Eller, Microsoft Pen Computing group. 2006 available at www.prioartdatabase.com / ip.com [Kato82] Kato, O., Iwase, H., Yoshida, M., and Tanahashi, J. "Interactive Handdrawn Diagram Input System", Proceedings of IEEE Computer Society Conference on Pattern Recognition and Image Processing, 14-17 June, 1982, Las Vegas, Nevada, pp544-549 Interactive user-interface for handwriting recognition, sketching and sketching, using angle variation and stroke type (straight line, angled line, ellipse, circle, arc) for features Contains user interface for text entry to pretty up character spacing and alignment, fixing sketches (without recognition) [KimJ83a] Kim, J. "Baseline Drift Correction of Handwritten Text", IBM Technical Disclosure Bulletin, Vol 25 No 10, March 1983, pp 5111-5114 See also Tappert papers Correct cursive script for baseline drift [KimS81] Kim, S. "Inversions - a catalog of calligraphic cartwheels", BYTE Books, Peterborough, New Hampshire, 1981 Very large catalog of reversible and ambiguous text to show variability of human recognition and perception. Has informal but very interesting bibliography on variability of human recognition [Kirsch82] Kirsch, Steven T. "Electro-Optical Mouse", United States Patent 4,364,035 Optical mouse of surface having passive, position-related marks in a pattern of two colors / Sekendur? (Shading optical tablet) [Kirsch83] Kirsch, Steven T. "Electronic Mouse", United States Patent 4,390,873, June 28, 1983 Optical mouse using checkerboard square pattern on the mouse-pad / tablet: Sekendur? [KnappAG83a] Knapp, Alan G. "Touch sensor array systems and display systems incorporating such", United States Patent 5,270,711, December 14, 1993 Capacitive touch tablet: an array of diode capacitors cells (can be transparent and overlaid on a display, or stand-alone graphics tablet) are charged, touching the cells with a stylus or finger discharges the capacitor. Pressure-threshold membrane switches may be used to make touch "pressure sensitive". Identify multiple sensor elements which have touched and their Coordinates. (multi-touch) [KnoxKT82a] Knox, Keith T. "Image Processing Method and Apparatus Having a Digital Airbrush for Touch Up", United States Patent 4,345,313, assigned to assigned to Xerox Corporation, Stamford, Connecticut, August 17, 1982 For brush-type user-interface in graphics drawing [Krouse83] Krouse, John K. "Selecting a Graphic-Input Device for Cad/Cam", Machine Design, October 6, 1983, pp 74-80 Voice/speech/tablet/mouse/joystick input overview, 1983 [KrummeDW82a] Krumme, D.W., and Ackley, D.H. "A Practical Method for Code Generation Based on Exhaustive



Search", Proceedings of SIGPLAN '82 Symposium on Compiler Construction, A.C.M. SIGPLAN Notices, Vol 17 No 6, June 1982 Fortran compilers beat the heck out of "C" compilers for code efficiency [KruskalJB83a] Kruskal, Joseph B. "An Overview of Sequence Comparison: Time Warps, String Edits, and Macromolecules", SIAM Review, Society for Industrial and Applied Mathematics, Vol 25 No 2, April 1983, pp 201237 Distance metric for differences in linear strings: Levenshtein distance: mutations in chromosomes, UNIX "diff" files, string matching, minimal mutation distance etc. Clustering /boundary definition using relative difference (dyadic), not absolute position (monadic) Levenshtein distance easier to use than probabilistic/statistical estimate of how much change would be required Common sense: if your pattern recognition algorithm works better, it is better Describes boundary comparison using monadic variables/features (absolute value) vs dyadic (relative comparison only) Different approaches for dyadic comparison: common sense, adapt monadic variables, and Levenshtein relative distance [KuhlFP82a] Kuhl, Frank P. and Giardina, Charles R. "Elliptic Fourier Features of a Closed Contour", Computer Graphics and Image Processing Vol 18, pp. 235-258, 1982 Recognize object outlines by Fourier analysis of 2D chain-code outline, lowest mode of fit is an ellipse. Mentions quantization / pixelation of images, show effects. [KuklinskiT82a] Kuklinski, T. and G. Babb "Pattern Algorithm Permits Freehand Printed-Data Entry", Computer Technology Review, Winter 1982 Pencept pendpad terminal product, handwriting recognition on tablet. [Kurtzberg82] Kurtzberg, J.M. and Tappert, C.C. "Segmentation Procedure for Handwritten Symbols and Words", IBM Technical Disclosure Bulletin, Vol 25 No 7B, December 1982, pp 3848-3852 Mentions reduction of "dots" as a stroke type Makes disparaging remarks about "boxed" input for discrete recognition Discusses delayed strokes, "t"-crossings for discrete recognition [Kuzunuki83] Kuzunuki, Sochiro "Online Handwritten Input Graphic Editing Device (translation)", Japanese patent application JP60-75980, March 1983 (translation) Japanese patent showing segmentation, handwriting, sketch input, drawing extents Cited in Sklarew patents [Lai81] Lai, M.T.Y. and Suen, C.Y. "Automatic recognition of characters by Fourier descriptors and boundary line encodings", Pattern Recognition, Vol 14, 1981, pp 383-393 Cited in Kerrick88, Li89 [Lambden81] Lambden, Martin R. "Electrographic Apparatus", United States Patent 4,289,925, September 15, 1982 Quest Automation MicroPad, handwriting recognition product, small stand-alone terminal: pressuresensitive tablet using flexible membrane under tension, resistive film [LampsonBW83a] Lampson, Butler W. "A Description of the Cedar Language: A Cedar Language Reference Manual", Xerox PARC technical report CSL-83-15, December 1983 Reference manual for Cedar object-based language, system, and GUI, successor to Smalltalk. Cedar kernel language is low-level subset, with direct access to primitive or OS types not permitted in Cedar. Strong type checking. No description of Cedar GUI. [LandauerTK83a] Landauer, T.K., Galotti, K.M. and Hartwell, S. "Natural Command Names and Initial Learning: A Study of Text-Editing Terms", CACM, Vol 26, July 1983, pp 495-503 Cited in Rhyne86 Rhyne86 cites this that for verbal command names, very poor agreement on informal name subjects give for text editing commands (mnemonicity for gestures?) "naturalness" in user-interface not helpful: for example, better if different names for options with similar semantics, but different syntax Human factors / user-interface: different methods of eliciting preferred command names get different results for same subject -- in particular, naive users make poor choices for command names [LeeA83a] Lee, Allison and Lochovsky, F.H. "Enhancing the Usability of an Office Information System Through Direct Manipulation", Proceedings of the CHI 1983 Conference on Human Factors in Computing Systems, Boston, 1983, pp 130-134 Cited in Kankaanpaa87 gesture/handwriting user interface on digitizing tablet (using puck/stylus). Text editing gestures for add (caret), delete (underline), replace (inverted pigtail), paragraph, change font, etc. Trash can, filer, other icons (compare with Wang Freestyle?). [LeeB81] Lee, Bum C., Kim, Jung G. and Yi, Seung, K. "Improvement on Korean Character Recognition by Resolving Ambiguity Problem", Proceedings of International Conference on Cybernetics and Society, IEEE 0360/8913/81/0000-0193, 26-28 October, 1981, Atlanta, Georgia, pp 193-197 Korean recognition: six types of Korean characters (first consonants, second consonants, vertical vowels, horizontal vowels) using syntactic recognition and 8-direction chain codes [Lemone82] Lemone, Karen A. "Similarity Measures Between Strings Extended to Sets of Strings", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol PAMI-4, No 3, May 1982, pp 345-347



substrings of chain-codes [Leroux81] Leroux, J., Miclet, L., Bonnet, A., Delarue, X. and Tormos, S. "Segments Detection in Binary Pictures for the Representation and the Syntactic Recognition of Hand Written Characters", Proceedings of the 6th International Conference on Pattern Recognition, IEEE CH1801-0/82/0000/0692 1981, pp 692-695 Chain code string comparison in OCR for handwriting recognition? [Litvin82a] Litvin, Y. "Segmentation of Handwritten Text by the Analysis of Two-Element Connectors", unpublished manuscript, G.T.E. Research Laboratory, Waltham Massachusetts, 1981 Cited in Litvin82b [Litvin82b] Litvin, Y. "Principles of evaluation for hand-printed and cursive text recognition methods", G.T.E. Technical Note 401.1, April 1982 See also Recognitive Sciences and Skylight Software (Yuri's consulting business names). Mentions retrace removal (page 7) User interface: points out difference between errors understandable to user, and errors not ("qualitative errors"). [Litvin82c] Litvin, Y. "Two Implementations of Data Reduction in Graphics Input", unpublished manuscript, December 1982 Cited in Goodale83 [LoomisJ83a] Loomis, Jeffrey; Poizner, Howard; Bellugi, Ursula; Blakemore, Alynn; and Hollerbach, John "Computer Graphic Modeling of American Sign Language", Computer Graphics, Vol 17 No 3, pp. 105-ff Uses LED lights and digital cameras as three-dimensional 3D digitizer sensor to recognize hand gestures of ASL American Sign Language [Loy82] Loy, W.W. and Landau, I.D. "An On-Line Procedure for Recognition of Handprinted Alphanumeric Characters", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-4 No 4, July 1982, pp 422-427 Reduce handwritten characters to a polygon / Freeman chain codes, then compare syntactic feature vector, then statistics on segment lengths Serif/hook removal, preprocessing/smoothing, retrace collapsing on on-line character recognition Handwritten samples "only" constrained to one of 69 writing styles/shapes: 99% and 97%: 20Kbytes memory, 500 Ms on 8086 Asserts for on-line recognition that learning new shapes is more important than accuracy / recognition rate [MallosJB82a] Mallos, James B. "Touch Position Sensitive Surface", United States Patent 4,346,376, August 24, 198 Touchscreen over CRT display using total internal reflection of light generated by CRT, reflected where finger is in contact with screen. Like a lightpen, detection of position by knowing timing of CRT raster position, therefor unaffected by CRT raster drift. [Mantas83] Mantas, J. and Heaton, A.G. "Handwritten character recognition by parallel labelling and shape analysis", Pattern Recognition Letters, Vol 1, July 1983, pp 465-468 Refers to problem of encountering a shape system not trained to for adaptive recognition OCR of handwriting recognition using polygonal approximation (chain codes), fuzzy labelling: thinning, tail-removal, fuzzy sets. [Marr82] Marr, D. "Vision: A computational investigation into the human representation and processing of visual information", San Francisco, Freeman Press, 1982 Cited in IchikawaS84. Human recognition of figures by "wire-frame" model. [MartinC81a] Martin C.; Jelinsky, P.; Lampton, M.; Malina, R.F.; and Anger, H.O. "Wedge-and-strip anodes for centroid-finding position-sensitive photon and particle detectors", Review of Scientific Instruments, Vol 52, 1981, p. 1067 ff Charge-ratio electrostatic configuration for particle detection: resolution not limited by dimensions of wedge anodes and different-width strip anodes. Different geometries: dual wedge anodes between strip anodes, wedges in opposite directions of taper; radial configuration; zig-zag pattern with no vias / through holes in substrate. [Matsuda83] Matsuda, Ryouchi "Present Status and Future Trends of Japanese Language Information Processing Systems", Proc. of 1983 International Conference on Text Processing with a Large Character Set, Tokyo, Japan, October 17-19, 1983, pp 436-446 Survey of problems and technologies for processing Japanese Characters: standardized keyboards, speech recognition, handwriting recognition [Matsukawa83] Matsukawa, Junko "Naming and recognition of random shapes", Japanese Journal of Psychology, Vol 54 No 1, 1983, pp 62-66 (In Japanese) Humans recognize (recall? identify?) shapes better if the are recognizable as a familiar object [Maurer82] Maurer, H.A., Rozenberg, G., and Welzl, E. "Using String Languages to Describe Picture Languages", Information and Control, Vol 54, 1982, pp 155-185 (partial copy) Chain codes: something similar to BLRTs for describing images [McDermott83] McDermott, Drew "Contexts and Data Dependencies: A Synthesis", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol 5, No 3, May 1983, pp 237-246



General paper on information dependencies and context [McDonaldWW82a] McDonald, Wylie W. "System and method for providing an audible sound and a tactile feedback in an electronic data processing system", United States Patent 4,334,280, June 8, 1982 Give a beep periodically (one second intervals) while a device is operating, or give an audible and tactile feedback each time a key is pressed / device is operated. For calculators, beeps on the keys? [Meads83] Meads, Jon A. "Defining the Ergonomic Buzzwords", Proceedings of the 1983 Annual conference of the A.C.M What is user-friendly? friendly to a beginner may be bad for expert [Meguro82] Meguro, S. and Umeda, M. "Recognition of multi-font Chinese Characters", Trans. Inst. Electronic and Communications Engineers, Japanese Section, Vol E65, No 8, p 513, August 1982 Multi-font OCR off-line recognition [MehtaN82a] Mehta, Nimish "A Flexible Machine Interface", M.A.Sc. Thesis, Dept. of E.E., U. Toronto, 1982 Early multi-touch input device, optical detection of fingers on frosted glass, but no rear-projection (separate display). Also recognize shapes and objects on surface. Cited by Buxton as historical note. [MehtaN82b] Mehta, Nimish; Smith, Kenneth C.; and Holmes, F.E. "Feature extraction as a tool for computer input", IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '82. May 1982, pp. 818 - 820 multi-touch/multi-hand input tablet (camera underneath, optical) with projection of keyboard or template images on top. Reconfigurable keyboard/touch service. Compare with Kaplow (reconfigurable keyboard) and Jeff Han (FTIR multi-touch) and Wellner DigitalDesk. [Meyrowitz82a] Meyrowitz, Normand and van Dam, Andries "Interactive Editing Systems: Part I", Computing Surveys, Vol 14 No 3, September 1982 Text editing: primarily keyboard-based editing systems, long treatment of Xerox STAR with mouse pop-up menus, but no use of gesture (other than drag?), digitizer, stylus, or handwriting editing. [Meyrowitz82b] Meyrowitz, Normand and van Dam, Andries "Interactive Editing Systems: Part II", Computing Surveys, Vol 14 No 4, 1982 Text editing: primarily keyboard-based editing systems, long treatment of Xerox STAR with mouse pop-up menus, but no use of gesture (other than drag?), digitizer, stylus, or handwriting editing. [Micropad82] (no author) "Micropad Product Information", Quest International, American Sales Office for Micropad Inc., LaGrange Illinois handwriting-terminal using a digitizer with two conductive sheets, sheets covered by a hand-rest; singlecharacter discrete character recognition also showed a GUI application picking from a diagram of replacements parts, plus character recognition Press release included, data-entry use in New Scotland Yard: "gets write to the point"; Joe Crivello, National Sales Manager, Illinois [MoogRA82a] Moog, Robert A. "A Multiply Touch-sensitive Clavier for Computer Music Systems", Proc. International Computer Music Conference, 1982, pp. 601-605 Musical touch keyboard that detects X and Y position of fingers, key velocity (via position sensor), and force/pressure. Z/Force detected with conductive rubber with greater capacitance as it is conformed by force to a curved bar capacitor plate, four-wire resistive sheet touchpad for X and Y. [MoranT81a] Moran, T. "The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems", International Journal of Man-Machine Studies, Vol 15, 1971, pp 3-50 (abstract only) Grammar and semantics for user-interfaces (commands). Example input devices are keyboard and display. [Morasso83] Morasso, P., Mussa Ivaldi, F.A., and Ruggiero, C. "How a Discontinuous Mechanism can Produce Continuous Patterns in Trajectory Formation and Handwriting", Acta Psychologica, Vol 54, 1983, pp 83-98 Generative variability in handwriting [Mullin81] Mullin, James K. "Reliable Indexing Using Unreliable Recognition Devices", IEEE Trans on Pattern Analysis and Machine Intelligence, Vol PAMI-3, No 3, May 1981, pp 347-350 Indexing document automatically with OCR recognition by substituting likely-confused characters to same pseudo-character , similar to the Soundex system by Davidson for similar-sounding English names. Electronic ink / sketch indexing? [Murase83] Murase, H., Wakahara, T., and Umeda, M. "Online Recognition Algorithm for Hand-Sketched Flowchart by Candidate Lattice Method", Denshi Tsushin Gakkai Ronbunshi, Vol 65-D No 6, June 1983, pp 675682 (in Japanese), translated in Systems, Computers and Controls, Vol 14 No 3, 1983, pp 37-46 Claims 97.9% recognition rate on 120 samples (small sample size) Refers to segmentation errors in recognizing two-dimensional flowchart symbols Sketch recognition - flowcharts [MurrayJM82a] Murray, John M. and Klingenstein, Kenneth J. "The Architecture of an Electronic Book", IEEE Trans. Industrial Electronics, Vol. IE-29 No 1, February 1982, pp. 82-91 Estimate of hardware requirements (memory, processor, display) for an electronic book including graphics, with search and indexing functions. Reference works (PDR, etc.), educational textbooks, coupled with



speech synthesizers for the blind. Exemplary UI would be keyboard/buttons. [MyersCS81a] Myers, C.S. and Rabiner, Lawrence R. "Connected digit recognition using a level building DTW algorithm", IEEE Trans. Acoustics, Speech, and Signal Processing, Vol ASSP-29, 1981, pp 351-363 Cited in Kruskal83: Dynamic time warping for recognition [NEC82] NEC: "Terminal that accepts handwriting lets the uninitiated use computer", Electronics Magazine, June 30, 1982, p 76 NEC handwriting terminal product for personal PCs, low-cost [NEC83] NEC "News Update: NEC 2100", Electronics Magazine, June 16, 1983, page 32 NEC 2100 kanji and hiragana symbols ... product NEC handwriting terminal product, high-end version of personal PC product [NTT81] NTT "NTT 1900: System reads kanji characters into word processors", Electronics Magazine, June 16, 1981, page 64 NTT 1900 kanji and hiragana symbols product "Aesop" on-line handwriting recognition, Nippon Telephone and Telegraph See also list in CIC folder on NTT [NTT82] NTT: "System edits handwritten copy, finishes sketches", Electronics Magazine, June 30, 1982, pp 73-74 (Date may be wrong) sketch/scribble/gesture input and editing system. Mechanical digitizer(!). [Nagura83] Nagura, Masakazu and Suenaga, Yasuhito "A Facsimile-Based Graphics Editing System by Auxiliary Mark Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5 No 4, July 1983, pp 433441 See also Suenaga80, same paper Handwriting mark recognition, OFF-LINE scanned OCR user interface for graphics editing Mark-up OFF-LINE scanned handwriting recognition user interface for on-line changes to scanned line drawings Has many Japanese citations for OCR of handwritten drawing and character recognition [NagyG82a] Nagy, G. "Optical Character Recognition - Theory and Practice", in "Handbook of Statistics", Vol 2, Krishnaiah, P.R. and Kanal, L.N., editors, North-Holland, 1974, pp 621-649 Testing: substitution error rates for OCR in practice two to three orders of magnitude lower than reported in academic literature Optical digitizer resolution of 0.004" sufficient for typed text Human adaptation reasons for some handwriting recognition systems' success Cursive writing not as useful as speech, or discrete writing Most optical scanners for OCR barely have resolution adequate for recognizing ideal characters, much less real ones Optical scanning digitizer characteristics: geometric, photo-metric, control Optical scanning digitizer characteristics: cite for tablet digitizer as comparison Testing: cites work by Chow on statistical relation of substitution vs reject error rate Kahan87 cites this as saying Duda72 binary Bayesian statistical classifier is widely used in OCR Optical digitizer characteristics: no vendor willing to be pinned down on performance [NagyG83a] Nagy, G. "Optical Scanning Digitizers", IEEE Computer, May 1983, pp 13-24 Optical scanning digitizer characteristics: geometric linearity Optical scanning digitizer characteristics: stability/repeatability Optical scanning digitizer characteristics: cite for tablet digitizer as comparison [NagyG83b] Nagy, G. "Candide's Practical Principles of Experimental Pattern Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-5 No 2, March 1983, pp 199-200 Tongue-in-check list of how researchers fudge pattern recognition statistics/results to bias them in their favor [Nakajima81] Nakajima, K., Kida, H., and Arakawa, H. "Handprinted Character Recognition Techniques on Commercial-Based Facsimile Input", Electrical Communications Laboratories Technical Journal, Vol 30 No 9, 1981, pp 2361-2372 (in Japanese) (abstract only) Handwritten numerals and katakana recognition, coping with facsimile distortions and variability [Nakamura83] Nakamura, Y. "Character Reading Apparatus", United States Patent 4,389,634, June 21, 1983, assigned to Tokyo Shibaura Denki Kabushiki Kaisha, Kawasaki, Japan, and Tokyo Electric Company, Limited, Tokyo, Japan Hand-held scanner for character recognition [NakataniLH83a] Nakatani, Lloyd H. and Rohrlich, John A. "Soft machines: A philosophy of user-computer interface design", Proc. CHI '83, pp 19-23 Simulate real machines / controls for devices using real-time computer graphics display and a touch screen digitizer: refers to switches and controls appearing on display in response to input from other switches or controls, such as a calculator changing modes and keys. Compre with Kaplow? [NelsonBJ81a] Nelson, Bruce Jay "Remote Procedure Call", XEROX PARC Report CSL-81-9, May 1981: Ph.D. dissertation CMU report CMU-CS-81-119 Generally credit with coining term RPC Remote Procedure Call. RPCs integrated into computer, or proeprocessor for source file. Marshalling, exceptions, stub functions. Cites to other references for PKI authentication, authorization mechanisms. See also RFC 707, 1976. [Newbower81] Newbower, R.S.; Cooper, J.B.; Edmondson, J.E.; and Maier, W. Reynolds "Graphics-tablet for Data



Entry in Computer-assisted Anesthesia Record-keeping", ?? Conference Proceedings, IEEE 01954210/81/0000/0139, 1981, pp 139-142 User-interface application: special symbols and markings with handwriting recognition for application involving anesthesia record keeping User-interface: shows forms with combinations of writing, handwriting recognition, drawing, check-off menu areas, etc. [Nihei83] Nihei, Y. "Developmental Change in Covert Principles for the Organization of Strokes in Drawing and Handwriting", Acta Psychologica, Vol 54, 1983, pp 221-232 Change in writing styles as kids grow up [Numonics82] Numonics "DigiBit Product Description", Numonics Incorporated, 418 Pierce Street, Lansdale PA 19446, 1982 Vendor of electromagnetic digitizer tablets: small vendor, mostly custom products. [OCR81a] (various) "OCR/scanner products circa 1981", (various) Folder of OCR products circa 1981: Burroughs 1200 and Burroughs 1205 Series, AEG Formularleser PFL6160 POLYFONT, ECRM Concept 1 Pagereader; Burroughs PS100; DEST Corporation WorkLess Station; Datapro Research "All About Optical Readers" 1978; Datacopy CIR Software Character Image Recognition; Kurzweil KDEM 1200 Intelligent Character Recognition; Compuscan Alphaword Series 80 [Odaka81a] Odaka, Kazumi, Wakahara, Toru, and Hashimoto, Shin'ichiro "Online Handwritten Character Recognizer - An Application to Japanese Word Processor", EC Vol 81 No 20, pp 33-44 (in Japanese) Japanese handwriting recognition / text editing, boxed (large format) input on tablet integrated into desk surface. [Odaka81b] Odaka, K. "On-line Pattern Recognition System for Hand-written Characters", United States Patent 4,284,975, August 18, 1981, assigned to Nippon Telegraph and Telephone Public Corporation, Tokyo, Japan NTT character recognition patent: Use of "feature points" in character recognition vs octants, etc. [Odaka82a] Odaka, K. and Masuda, I. "Pattern Recognition System for Hand-written Characters Operating on an On-Line Real-Time Basis", United States Patent 4,317,109, February 23, 1982, assigned to Nippon Telegraph and Telephone Public Corporation, Tokyo, Japan NTT character recognition patent Patent on stroke-order independent recognition for Kanji/Chinese Feature points for Kanji/Kana are lengths of segments [Odaka82b] Odaka, Kzaumi, Arakawa, Hiroki and Masuda, Isao "On-line Recognition of Handwritten Characters by Approximating Each Stroke with Several Points", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-12 No 6, November 1982, pp 898-903 See 1980 paper of same title in Japanese Better than 99.8% correct recognition on Chinese/Kanji, hiragana, katakana, and alphanumerics/Romanji. Three to six feature points on each stroke for on-line handwriting recognition: points just resampled to a minimum distance? end-points of (Chinese) straight strokes only? [Odaka82c] Odaka, K.; Wakahara, T.; and Masuda, I. "Stroke Order Free On-line Character Recognition Algorithm and Its Application", Trans. Inst. Electonics and Commun. Eng. of Japan, Vol E65 no 6, June 1982, pp. 379-ff (partial copy) Shape matching for handwriting recognition, followed by specific algorithms for ambiguous cases. Uses distance measurements and not stroke-order information. [Odaka83a] Odaka, K., Wakahara, T., Masuda, I., and Hashimoto, S. "Stroke Order Free Online Character Recognition Algorithm and Its Application", Electronic Communications Laboratory Technical Journal, Vol 32 No 10, 1983, pp 2145-2158 (in Japanese) Japanese handwriting recognition: features are inter-stroke distance pairs: claims 99.5% accuracy on 2057 Kanji Refers to AESOP user-interface for handwriting text/script editing. Measures accumulated distance along each stroke of an input with the strokes of an ideal sample -- compare with Tappert elastic matching? [Ogawa81] Ogawa, H. and Taniguchi, K. "Stroke resolution and segmentation of character string based on relaxation techniques", Trans. of IECE Japan, Vol PRL80-5, 1981 Cited in Leung87 [OkaR83] Oka, R. "Studies on Recognition of Handwritten Chinese-Japanese Characters by Using Cellular Features", Research of the Electrotechnical Laboratories, Report No 834 1-109, September 1973 (in Japanese) Cellular automoton for character recognition (?) [Okamoto83] Okamoto, N., Nakamura, O., and Minami, T. "Character Segmentation for Mixed-Mode Communication", Proceedings of IFIP 9th World Computer Congress, 19-23 September, 1983, Paris France, pp 681-685 Character and word segmentation in OCR documents using overlapping rectangles [Okamura83] Okamura, K., Morita, K., Kanaoka, T., Okada, T., and Tomita, S. "Syntactic Pattern Recognition for Handwritten Katakana Characters by a Bottom-up Parser", Trans. of the Institute of Electronics and Communications Engineers of Japan, Vol J66D No 2, February 1983, pp 222-223. (Abstract only)



NTIS abstract of Japanese publication: Katakana Japanese handwriting recognition [Pavlidis82a] Pavlidis, Theo "An asynchronous thinning algorithm", Computer Graphics Image Processing, Vol 20, 1982, pp 133-157 Cited in Arcelli85 Arcelli85 cites as general reference on line thinning, and its applications [Pavlidis82b] Pavlidis, Theo "Algorithms for Graphics and Image Processing", Berlin, West Germany: SpringerVerlag, 1982 Cited in Arcelli85 Arcelli85 cites chapter 9 as general reference on line thinning, and its applications [Pavlidis83a] Pavlidis, Theo "Effects of Distortions on the Recognition Rate of a Structural OCR System", Proceedings of Computer Vision and Pattern Recognition '83, Washington, D.C., June 1983, pp 303-309 Cited in Lam86 3-point calibration on digitizer tablets (not four points) [Pencept82a] Pencept "Hand Print Recognition Technology Provides A New Way To Use Computers", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1982 handwriting-input terminal product: DCR dynamic character recognition (a.k.a. on-line handwriting recognition). Mentions boxed input for stroke parsing/segmentation. Tablet 11" high by 15" wide: standard paper with printed form on left, control area on right. [Pencept83a] Pencept "Pencept Penpad (TM) 200 Product Literature", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1983 handwriting-input terminal product [Pencept83b] Pencept "PENPAD Reference Manual", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1983 (hardcopy manual) Handwriting terminal with digitizing tablet and handwriting recognition Handwriting recognition terminal from Pencept: VT-100 terminal emulation, with internal language for forms management. P. 3-7 keyboard emulation mode for input. p. 3-36 virtual tablets, virtual areas with independent programming, coordinate origin/rotation/scaling. [PepperW81a] Pepper, William Jr. "Touch panel system and method", United States Patent 4,293,734, October 6, 1981 Resistive-film touchpanel, finger may either be source or sink of signal. Signal flowing through user's body to ground, or signals generated by ambient noise (e.g. 60-Hz hum). Pressure-sensitive touch panel, transparent touch panel using ITO or transparent polyester film. Triangular resistive sheet touchpanel with three sides. [PepperW81b] Pepper, William Jr. "Video game apparatus and method", United States Patent 4,302,011, November 24, 1981 Touch-screen digitizer that senses intensity of touch force or pressure [PepperW82] Pepper, William Jr. "Touch panel system and method", United States Patent 4,353,552, October 12, 1982 Resistive-film touchpanel, finger may either be source or sink of signal. Signal flowing through user's body to ground, or signals generated by ambient noise (e.g. 60-Hz hum). Pressure-sensitive touch panel, transparent touch panel using ITO or transparent polyester film. Triangular resistive sheet touchpanel with three sides. [PepperW83a] Pepper, William Jr. "Edge terminations for impedance planes", United States Patent 4,371,746, February 1, 1983 Edge connectors to linearized distortions at edges of resistive film digitizer / touch-screen [PetersC83a] Peters, Chris "The Microsoft Mouse: Letting the mouse out of the bag", BYTE Magazine, June 1983, pp 130-143 Microsoft mouse: article compares mouse to light pen and to joystick. Piano demonstration program. [PetersenP83a] Petersen, Peter "Man-Machine Communication: CERT Thesis", PhD Thesis, CERT / Aalborg Universitetscenter, 11 January 1983 Review of CERN user-interface (man-machine interface) developments, including capacitive touchscreen. Capacitance measurement by charge-time detector (did not work well), and by voltage divider (worked better). 16-button touchscreen with heirarchical menus. Cites to touchscreens based on cross wires, infrared light, acoustic waves, resistive, capacitive. [PickHL83a] Pick, Herbert L. Jr., and Teulings, Hans-Leo "Geometric Transformations of Handwriting as a Function of Instruction and Feedback", Acta Psychologica, Vol 54, 1983, pp 327-340 Tell users to change their handwriting: changing slant works, but if say to make taller, users also make wider, and vice versa. (Handwriting variability) [Plamondon83] Plamondon, R. and Brault, J-J. "A System for Signature Analysis and Verification Based on an Accelerometer Pen", Proceedings of International Carnahan Conference on Security Technology, Zurich, Switzerland October 4-6, 1983, pp 157-163 Uses tilt angle in signature verification [PopularScience81a] Popular Science "Electronic printer transmits mail, makes copies, too", Popular Science,



February 1981 Xerox 5700 printer with touch-screen interface showing simulated buttons and switches, touch-to-begin [Quest82] Quest Automation Limited "Micropad User's Guide", Quest House, Prince's Road, Ferndown, Dorset BH22 9HQ, United Kingdom Micropad dynamic on-line character recognition product: Resistive film digitizer? [Quest83] Quest Automation Limited "Q-Sign Terminal product literature", 10 Whittle Road, Wimborne, Dorset BH21 7SD, United Kingdom. Tel: 0202 891518, 1983 Quest Automation: Micropad signature verification product [Ray81] Ray, A.K. and Chatterjee, B. "An Algorithm for the Recognition of Constrained Handwritten English Numerical Characters", Journal of the Institute of Electronics and Telecommunications Engineers of India, Vol 27 No 9, September 1981, pp 297-299 NTIS abstract: OCR on handwriting recognition of numerals: features are true endpoints, true group points, and true cross points [RecycledGoods83a] Recycled Goods, Inc. "Pencept M200 Penpad Tablet Computer - Vintage Collectible", www.labx.com/v2/spiderdealer, fetched Dec 2007 Pencept Penpad M200 handwriting terminal, circa 1983, showing system box, digitizer, etc. [Rediffusion82] Rediffusion Computers Limited "WRITAWAY product literature", Kelvin Way, Crawley, Sussex RH10 2LY, England, 1982 Micropad-like handwriting recognition product from England, using two resistive sheets and an air separator on the tablet. [Reilly81] Reilly, Douglas L., Cooper, Leon N., and Elbau, Charles "A Neural Model for Category Learning", Center for Neural Science and Department of Physics, Brown University, Rhode Island date approximate Neural (net) model for supervised learning, later applied to handwriting recognition [RivestRL83a] Rivest, Ronald L.; Shamir, Adi; and Adleman, Leonard M. "Cryptographic Communications System and Method", United States Patent 4,405,829, September 20, 1983 Patent on original RSA public-key encryption/decryption algorithm. [RocheleauRT81a] Rocheleau, R.T. "Coarse Position Digitizer", United States Patent 4,242,843, January 6, 1981, assigned to Summagraphics Corporation, Fairfield, Connecticut Digitizer with loop as the transmitter, grid as the receiver [Romein81] Romein, J.J. "Acoustic Writing Combination, Comprising a Stylus with an Associated Writing Tablet", United States Patent 4,246,439, January 20, 1981, assigned to US Philips Corporation, New York, New York Acoustic digitizer design: two ultrasonic sound sources on the stylus, permitting tilt / parallax correction Accuracy: pencil lines are 0.005", Ink is 0.01" wide, visual acuity is 0.0005" max [SAC82] SAC "Digitizer Terminology and Comparability", Science Accessories Corporation, 1982, Southport, Connecticut [Saghri81] Saghri, J.A. and Freeman, H. "Analysis of the precision of generalized chain codes for the representation of planar curves", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-3, September 1981, pp 533-539 Line-segment approximation shows average quantization error a function of grid size, not angle resolution: gives formula for grid size vs matching accuracy [Sakoe82] Sakoe, Hiroaki "A Generalized Two-Level DP-Matching Algorithm for Continuous Speech Recognition", Trans. of the I.E.C.E. of Japan, Vol E65 No 11, November, 1982, pp 649-656 Dynamic programming, time-skipping, dynamic time warping, for speech (author has also published on handwriting character recognition) [Salkeld82] Salkeld, Robert J. and Sklarew, Ralph C. "Closed Space Structures", United States Patent 4,318,517, March 9, 1982 Same Ralph Sklarew of Grid computer: structures in space the form of a ring around a planet or other orbiting body: cites Larry Niven "Ringworld" as prior art [Salter83] Salter, L. "Variability of Japanese Characters", internal report, Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, September 1983 Analysis (no user studies) of variability expected in Japanese handwriting recognition, based on what had been found in ASCII alphabetic handwriting and stylus dynamics [Samet81] Samet, Hanan "An Algorithm for Converting Rasters to Quadtrees", IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol PAMI-3, No 1, January 1981, pp 93-ff Image processing: group lines/regions in scanned raster image / bitmap into quadtrees, where tree nodes are maximal blocks (e.g. minimal fragmentation of regions) [SasakiL81a] Sasaki, L.; Fedorkow, G.; Buxton, W.; Retterath, C.; and Smith, K.C. "A Touch-Sensitive Input Device", Intl. Computer Music Conference Proceedings, 1981, pp. 293-297 Single-touch pressure sensitive (contact area) capacitive touchscreen/tablet for musical instrument input. Adjust two values at once using X/Y input of tablet, into SSSP digital synthesizer music program. Multi-



touch input would be better. [SatoT82] Sato, T. and Toja, A. "Recognition and Understanding of Hand-drawn diagrams", Proceedings of the 6th International Conference on Pattern Recognition, pp 674-677, 1982 Cleaning up graphical drawings using low-level symbol recognition: (see sketch editing, prettifying) [SatoY82] Sato, Yuichi and Nakamura, Taichi "Predictive Encoding Method for Handwriting Signals", Trans. of the IECE of Japan, Vol E 65 No 2, February 1982, p. 133 (abstract only) 30 Hz sampling of handwriting on a tablet, DPCM/PCM coders to send handwriting at 200-300 bits/second, 100-500 bit buffer: tablet performance/handwriting signal extraction/frequency response [Schaeken82] Schaeken, B. and Verschueren, W. "A Recognition System for Handwritten Numerals", Proceedings of the 6th International Conference on Pattern Recognition, Munich, Germany, October 19-22, 1982 NTIS: Supervised learning on statistical recognizer [SchmandtC81a] Schmandt, Christopher and Hulteen, Eric A. "The Intelligent Voice-Interactive Interface", Proc. CHI'82, pp 363-366 Put-That-There project: hand-pointing gestures and voice command input [SchulzeLJJ83a] Schulze, Lawrence J.J. and Snyder, Harry L. "A Comparative Evaluation Of Five Touch Entry Devices", Virginia Polytechnic Institute report AD-A240 114, October 1, 1983 Comparison of touchscreen digitizer / touch entry devices (TED) technologies: touch wire, cross wire, capacitive, conductive film (resistive), acoustic ranging, infrared (optical beam) and pressure sensitive. Cites to E.A. Johnson 1960's for invention of touch entry device. [Shillman81] Shillman, R.J. "Dynamic Character Recognition: An Emerging Technology", invited paper, Proceedings of COMPCON 81, 22nd IEEE Computer Society International Conference, IEEE Catalog No 81CH1626-1, February 23-26, 1981 Handwriting input overview paper by Bob Shillman Gives more recent address for Micropad Limited, Image Data Products Limited, handwriting commercial vendors [ShinghalR82a] Shinghal, R. and Suen, C.Y. "A Method for Selecting Constrained Hand-Printed Character Shapes for Machine Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-4 No 1, pp 74-78 Shows lots of collected base forms (with Suen) [ShinghalR82b] Shinghal, Rahan "An Experimental Investigation of Four Text Recognition Algorithms", IEEE Trans. System Man and Cybernetics, Vol SMC-12, No 4, July/August 1982, pp. 573-577 Context helps on character recognition: specifically modified Viterbi algorithm, even on "unconventional" English text. [ShinghalR83a] Shinghal, R. "A hybrid algorithm for contextual text recognition", Pattern Recognition, Vol 16, 1983, pp 261-267 Context via Markov method, plus a dictionary, better than predictor-corrector method Cited in Sinha88: on spelling/dictionary context correction [ShiraiY82a] Shirai, Yoshiaki "Image Processing for Data Capture", IEEE Computer, November 1982 pp. 21-35 Review of extracting 3D information from 2D images: shading, binocular imaging: mostly machine vision. Also sketch recognition for hand-drawn (sanned) electronic circuit diagrams. [Shneiderman83] Shneiderman, B. "Direct Manipulation: A Step Beyond Programming Languages", IEEE Computer, Vol 16 No 8, pp 57-69, August 1983 Refers to IBM direct-manipulation office desktop user interface "Pictureworld" (like Microsoft's "Bob"?), with file cabinets, mailboxes, notebooks, phone messages Direct manipulation: gesture/command symbols user interface. States that direct manipulation UIs are more like actual objects people already comprehend, therefore easier to use (e.g. VisiCalc spreadsheet and a paper spreadsheet). Defines direct manipulation as manipulating digital objects on a screen without the use of command-line commands. Has definition of direct manipulation, page 64 [Shoukry83] Shoukry, Amin and Amin, Adnan "Topological and statistical analysis of line drawings", Pattern Recognition Letters, Vol 1, July 1983, pp 365-374 On-line handwriting recognition using two-dimensional graph (chain codes) using slope/angle of lines, intersections, labyrinthology, for features [Siddiqui83] Siddiqui, K.J. and Shinghal, R. "Using Contextual Postprocessing to Improve Machine Recognition of Text", IEEE International Symposium on Information Theory, 26-30 September, 1983, St. Jovite, Quebec (abstract only) Recognition performance 71% on Munsun's OCR handwriting data set, improved to 86% with Viterbi context algorithm [SinghB83] Singh, Baldev, Beatty, John C., Booth, Kellogg S., Ryman, Rhonda "A Graphics Editor for Benesh Movement Notation", Computer Graphics, Vol 17 No 3, pp 51-ff Digitizer tablet with four-button puck, experiment in GUI design with floating (pop-up?) and dynamic menus [SmithAR81] Smith, A. Richard and Erman, Lee D. "Noah -- A Bottom-Up Word Hypothesizer for Large-Vocabulary



Speech Understanding Systems", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-3 No 1, January 1981, pp 41-51 Context: using low-level speech context, hypothesizes word for higher-level context analysis. Context to resolve co-articulation on continuous speech recognition. Cites experiments to determine human use of context in speech. On speech recognition, discusses possible biases and justifications for them in training and test data collection. [SmithDC82a] Smith, David Canfield; Irby, Charles; Kimball, Ralpha; Verplank, Bill "Designing the Star User Interface", BYTE Magazine, 1982 No 4, pp 242-282 Design of Xerox Star GUI with mouse and keyboard: mouse uses trackball rather than SRI wheels. Development philosophy: throw away first implementation, it was always a prototype. Properties and application sheets for graphical objects. Move is by click-source/click-MOVE-button/click-target, not drag/move gesture. Keyboard windows / on-screen keyboard / virtuelle tastatur, similar to soft input panel SIP? [SmithDC82b] Smith, David Canfield; Irby, Charles; Kimball, Ralpha; Harslem, Eric "The Star User Interface: an overview", Proc. NCC AFIPS '82, June 7-10, 1982, pp. 515-528 Desktop user interface patterned after physical office: forms/records have no physical counterpart. Data icons and function icons, e.g. icons for printers, file cabinets, folders, property sheets. Keyboard windows / on-screen keyboard display, similar to soft input panel SIP? press-and-hold? Earlier references back to 1977 [SnellJM83a] Snell, John M. "Sensors for Playing Computer Music with Expression", Proc. Int'l. Computer Music Conf., 1983, pp. 113-126 Touch-sensitive touchpads in music for expression: add touch surfaces to clavier keyboard keys to send finger-force during a note, or movement in in Y direction (X is pitch, need the discrete keys). Back of black and white keys merge to a flat surface for continuous pitch control in X direction. Linear touch sensor may be sub-divided into sections for multiple fingers on same pitch (multi-touch). Reducing motion travel of key allows faster playing: substitute with force sensor. Cites multi-touch to Bill Buxton in 1983 for musical instrument input. Musicians rely on haptic feedback (feel of the physical instrument) in playing. Optical multi-touch input using "wine glass effect" of total internal reflection on a glass surface. [SnowberryK83a] Snowberry, K. et al "Computer Display Menus", Ergonomics, Vol 26 No 7, 1983, pp 699-712 Hierarchical menu breadth / wide menus gives faster user actions than tall / narrow menus. [SpenceR82a] Spence, Robert and Apperley, Mark "Data base navigation: an office environment for the professional", Behavior and Information Technology, Vol 1 No 1, 1982, pp. 43-54 Paper study (as in paper-based prototype) for a interface for database use (office) combining pointing, gesturing, touching, and spoken voice recognition commands. Anticipates using wall-sized display, in-air pointing by ultrasonic or capacitive sensor to point to icons. Refers to in-air pointing as Teletouch, "fingerprint" confirming cursor to indicate icon is being pointed to. Cites to Micropad handwriting recognition terminal, and Bolt "Put-that-there" gesture input. [Srihari82] Srihari, Sargur N. and Bozinovic, Radmilo "A String Correction Algorithm for Cursive Script Recognition", Proceedings of the 6th International Conference on Pattern Recognition, IEEE CH18010/82/0000/0232 1981, pp 232-234 Not sure about conference name Spelling dictionary string correction, using a posteriori computation after recognition is done [Srihari83a] Srihari, S.N., Hull, J.J., and Choudhari, R. "Integrating diverse knowledge sources in text recognition", A.C.M. Trans. Office Information Systems, Vol 1 no 1, 1983, pp 68-87 Context from bottom-up (probability from previous letter sequence), channel (probability of "A" corrupting to "B"), and top-down (lexicon). Spelling correction for substitution errors: same as Hull83a [Srihari83b] Srihari, Sargur N. and Bozinovic, Radmilo "Use of Knowledge in the Visual Interpretation of Cursive Script", International Conference On Systems, Man and Cybernetics, pages 187 - 191. IEEE Computer Society Press, 1983 Cited by Marlin Eller, Microsoft Pen Computing group [Stentiford83] Stentiford, Frederick W.M. and Mortimer, R.G. "Some New Heuristics for Thinning Binary Handprinted Characters for OCR", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-13 No 1, January/February 1983, pp 81-84 Line-thinning for handwritten data Skeletonization/thinning: side effects are spurious projection, necking, tail generation, noise holes, with heuristics to fix them. [StewartTJ81] Stewart, Theodor J. "An Interactive Approach to Multiple Criteria Decisionmaking Based on Statistical Inference", IEEE Trans. on Systems, Man, and Cybernetics, Vol SMC-11, No 11, November 1981, pp 733-ff feature selection critical in pattern recognition (see other references on choice of features) [Stonham82] Stonham "Networks of Memory Elements -- A Processor for Industrial Automation", Digital Systems



for Industrial Automation, Issue No 1 pp 2-3, 1982 Cited in Faggin89: for pattern recognition on human faces Cited in Faggin89: in connection with Wizard/Wisard adaptive image classifier [Suen82] Suen, C.Y. and Shinghal, R. "A Method for Selecting Constrained Hand-Printed Character Shapes for Machine Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol Pami-4 No 1, January 1982, pp 74-78 Describes Suen's data base of 174 types of sample characters [Suen83] Suen, C.Y. "Handwriting Generation, Perception, and Recognition", Acta Psychologica, Vol 54, 1983, pp 295-312 Describes printing, script, and manuscript styles and their legibility to humans. Says writing speed for printing increases with practice. Digitizer spec is for 3/8" proximity sensing range. [Summagraphics81a] Summagraphics "Bit Pad One User's Manual", Summagraphics Corporation, August 1981 User's manual for Summagraphics Bit Pad One magnetostrictive tablet digitizer. Describes (accurately?) magnetostrictive technology. Has Schematics of control electronics. [Summagraphics82a] Summagraphics "Summagrid User's Guide", page A-3, Summagraphics Corporation, 1982, Fairfield, Connecticut, 1982 [Summagraphics82b] Summagraphics "An Application Note: How the Summagraphics data tablet digitizer is used", Summagraphics, 1982 Application note for Summagraphics data tablet, pattern recognition for handwritten characters at University of New Brunswick Computing Centre, handwritten characters: pre-processing and classification subroutines, with menu-picking areas (rectangles) on side of tablet. [Summagraphics82c] Summagraphics "Bit Pad One: The versatile Data Tablet Digitizer for your computer system Product Information", Summagraphics, 1982 BitPad-1: de facto serial interface standard for digitizing tablets, expecially with stylus. Sales literature mentions data entry, cursor control (mouse/locator input), elimination of typing. [Summagraphics83a] Summagraphics " MM 1201 and MM 961 Technical Reference (Preliminary for Preproduction Product)", Summagraphics Corporation, 1983 Electromagnetic digitizer: describes increment mode, with minimum distance to record movement in stream mode. Compare with press-and-hold? [TanakaHa82] Tanaka, Hatsukazu; Hirakawa, Yutaka; and Kaneku, Seiko "Recognition of Distorted Patterns Using the Viterbi Algorithm", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-4 No 1, January 1982, pp 18-25 Viterbi algorithm plus a trellis of partial patterns, to do recognition of whole from context of parts [TanakaHi82] Tanaka, Hideo; Uejima, Satoru; and Asai, Kiyoji "Linear Regression Analysis with Fuzzy Model", IEEE Trans. Systems, Man, and Cybernetics, Vol SMC-12 No 6, November 1982, pp 903-907 Fuzzy sets applied to pattern recognition where classifications overlap. [Tanner83] Tanner, Peter, P. and Buxton, William A.S. "Some Issues in Future User Interface Management System (UIMS) Development", Technical Report, University of Ontario, also in "User Center System Design: New Perspectives on Human-Computer Interaction", Norman, Donald A. and Draper, Stephen W (Ed.), 1986, Lawrence Erlbaum Associates, New Jersey / London Joystick, track-ball, digitizer tablet, with two-handed input [TappertCC82] Tappert, C.C. "Cursive Script Recognition by Elastic Matching", IBM Journal of Research and Development, Vol 26 No 6, November 1982, pp 765-771 Handwriting cursive recognition by point-by-point distance measurements against "ideal" cursive writing. Does not do well with loops vs cusps, which are forms that transition into each other in handwriting motion and variability. Refers to hook effects due to weak tablets as a big problem: compare with Ward patent and article on digitizer tables. Combines letters in connected script to match whole words (context/dictionary.) [Taylor83] Taylor, I. and Taylor, M.M. "The Psychology of Reading", New York: Academic Press, 1983, pp 183193 Cited in Bozinvic89 Bozinovic89 cites for human recognition by outline of word (as shown in WrightG52) [Telesis81a] Telesis Systems Corporation "Telesis Systems Corporation, Business plan", Telesis System Corporation, Concord MA, January 1981 (hardcopy book) Business plan for CAD/CAM company. Product used two displays, one for displaying menus and using lightpen/touchscreen input, the other for the CAD display. [ThomasJJ83] Thomas, James J. and Hamlin, Griffith "Workshop summary: Graphical Input Interaction Technique", printed in Computer Graphics, Vol 5, January 1983, pp 279-304 Workshop report on graphical input interaction user interfaces: including handwriting recognition GUI [Thornburg82] Thornburg, Dvid D; Flegal, Robert M.; and Lam, Tat C. "Graphics Pen for Soft Displays", United States Patent 4,318,096, March 2, 1982



Tablet pen with pressure transducer in the tip, to measure side-force on the stylus, for graphic arts rendering [ToddR81a] Todd, Robert "Apparatus and Method for Recognized a Pattern", United States Patent 4,259,661, assigned to Burroughs Corporation, March 31, 1981 Cited in Bokser88 OCR hardware: recognition via template comparison [Toida83] Toida, T. et al "A Method for Correcting Characters in On-Line Handwritten Input", Showa 58-nendo Denshi Tsushin Sogo Zenkoku Taikai, April 1983 Cited in Sklarew99 patent [Torok82] Torok, G.P. and White, A.B. "Remote Chalk-board automatic Cursor", United States Patent 4,317,956, March 2, 1982 Remote chalkboard / whiteboard / telautograph, display remote cursor showing where the stylus/marker is in proximity (?) so that things can be "pointed to" remotely without marking. Digitizer tablet patent on automatic cursor, mark erasure (recognition strokes removed in electronic ink user-interface), GUI display of eraser in remote whiteboard system. Uses "telautograph" systemas term for whiteboard system, but unidirectional whiteboard, only communicating one way. Use of a display/proximity cursor on whiteboard to help other user notice where things are being written, and to "point to" images. [Tsichritzis82] Tsichritzis, D. "Form Management", CACM, Vol 25 No 7, July 1982, pp 453-478 Quoted in Hakmatpour86 (missing from this list?) Deals with forms in office management, says flow of forms is important Mentions non-paper "forms" for voice Refers to operations on forms: automatic calculation, etc. [Tsuruoka83] Tsuruoka, S., Kimura, F., Yoshimura, M., Yokoi, S., and Miyake, Y. "Thinning Algorithms for Digital Pictures and Their Application to Handprinted Characters Recognition", Trans. of IECE of Japan, Vol J66D No 5, May 1973, pp 525-532 (in Japanese: abstract only) Skeletonization / line-thinning [Turba81] Turba, T.N. "Checking for spelling and typographical errors in computer-based text", in Proceedings of A.C.M. SIGPLAN SIOGA Symposium on Text Manipulation", Portland, Oregon, June 1981, pp 51-60 Cited in Hull83a spelling/context correction using structural information: invalid prefix ("in" for "im", joining of suffix (beginner vs beginer), etc. rather than just comparison with word list. [Voiers83] Voiers, W.D. "Evaluating Processed Speech using the Diagnostic Rhyme Test", Speech Technology, January/February 1983, pp 30-39 Speech: how contextual factors are controlled in test protocols affects recognition results. Speech recognition failures completely explained by a limited set of underlying phonemic (human recognition) features [Wakahara83] Wakahara, T. and Umeda, M. "Stroke-number and Stroke-order Free On-line Character Recognition by Selective Stroke Linkage Method", Proceedings of ICTP '83, Tokyo, October 17-19, 1983, pp 157-162 Combinatorial solution to stroke connection and stroke order variations. Break strokes into equal-length pieces, then match them as a feature [WalkerJ83] Walker, John "The Autodesk File: Crisis Letter", www.fourmilab.ch/autofile From collection of internal documents on the history of AutoCad: June 21, 1983. cites special concern on marketing deal with Sun-Flex (touch-pen touchscreen digitizers), Touch-pen considering alternative vendor P-CAD. [WallaceBA81a] Wallace, Bruce A. "Merging and Transformation of Raster Images for Cartoon Animation", Computer Graphics, vol 15 No 3, August 1981, pp 253-262 opacity mask and image blending (similar to alpha blending) for anti-aliasing [Wang83] Wang, C. Sun, H., Yada, S., and Rosenfeld, A. "Some experiments in relaxation image matching using corner features", Pattern Recognition, Vol 16, 1983, pp 167-182 Cited in Yu90: on context for line thinning using chain codes [Watanabe83] Watanabe, Y., Gyoba, J., and Maruyama, K. "Reaction time and eye movements in the recognition task of hand-written Katakana-letters", Japanese Journal of Psychology, Vol 54 No 1, pp 58-61, 1983 (in Japanese) Uses eye fixation to determine what features are cognitively important Repeats and continues Blesser et al's early work from Massachusetts Institute of Technology [Watari83] Watari, Masao, Sakoe, Hiroaki, Chiba, Seibi, Ishizuka, Hisao, Kawakami, Yuichi, and Iwate, Toshiki "A DP-Matching LSI for Speech Recognition", NEC Research and Development, No 70, pp 71-78, July 1983 Dedicated dual processor chip used for speech recognition [WeltyC81a] Welty, Charles and Stemple, David W. "Human Factor Comparison of a Procedural and a Nonprocedural Query Language", ACM Trans. Database Sys. 6(4):626-649 (Dec 1981) TABLET - Query language, more procedural than SQL. Human Factors (learnability) of a programming language. TABLET is more verbose, Cobol-like syntax. [WhiteJM83] White, J.M., and Rohrer, G.D. "Image Thresholding for Optical Character Recognition and Other



Applications Requiring Character Image Extraction", IBM Journal of Research and Development, Vol 27 No 4, pp 400-411, July 1983. (Abstract only) Handwriting recognition using nonlinear adaptive procedure: thresholding of scanned OCR images Preprocessing to clean up OCR scanner images of carbon copy forms, bank checks, smudges, scenic backgrounds, etc. [WhitfieldD83] Whitfield, D., Ball, R.G. and Bird, J.M. "Some comparisons of on-display and off-display touch input devices for interaction with computer generated displays", Ergonomics, Vol 26, 1983, pp 1033-1053 Beringer89, electronic ink hardware? Includes definition of touchscreen and touchpad which may be separate from display, review of touch-screen digitizer technologies. Mentions position accuracy errors at light touch or at end of a stroke as finger is lifted. Touchpad uses mylar sheets held apart by air pressure: micropad? [WillisR83a] Willis, Richard "Big Blue Goes Japanese", BYTE Magazine, Novmeber 1983, pp. 144-163 IBM 5550 Japanese (Katakana and Kanji) display terminal, 24x24 matrix characters. See also CIC Handwriter handwriting recognition tablet for 5550. [Wing83] Wing, A.M., Nimmo-Smith, M.I., and Eldridge, M.A. "The Consistency of Cursive Letter Formation as a Function of Position in the Word", Acta Psychologica, Vol 54, 1983, pp 197-204 Allograph (variant) selection based on preceeding context character [Witkin83] Witkin, A. P. "Scale-space filtering", Proceedings of the International Joint Conference on Artificial Intelligence, 1983, pp 1019-1022 Cited in Lipscomb91 Template matching of varying signals by filtering/smoothing at different scales to reduce noise, filtering is adaptive and variable [WolfeldJA81a] Wolfeld, Jeffrey A. "Real time control of a robot tactile sensor", Technical report MS-CIS-81-04 and Master's Thesis, Dept. of Computer and Information Science, U. Penn., August 1981 Early multi-touch sensor (cited in Lindemann): Pad sensor, 8x8 grid of pressure (force) sensitive sites, connected to strain gauges for parallel force. References for artificial force-sensing/touch-sensing robotic skin. [WuL82a] Wu, Li-De "On the Chain Code of a Line", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-4, No 3, May 1982 Chain codes of optically scanned / static character recognition OCR [YamamotoE81] Yamamoto, E., Fujii, N., Fujita, T., Ito, C., and Tanahashi, J. "Handwritten Kanji Character Recognition Using the Features Extracted from Multiple Standpoints", ?? Conference Proceedings, IEEE CH-15958/81/0000/0131, 1981 Using multiple features rather than single feature, and weighting results, gives better performance on character recognition. [YamamotoK83] Yamamoto, K. "Studies on the Recognition of Handprinted Characters by Structural Analysis Methods", Research Electrotechnical Laboratory, Report No 831 1-114, February 1973 (in Japanese) Outermost point recognition method for Romanji, numeral, Katakana [Yamasaki82] Yamasake, T., Inokuchi, S., and Sakurai, Y. "Training System for Handwritten Chinese Characters Using On-Line Character Recognition Techniques", Trans. of IECE of Japan, Vol E65 No 10, p 602, October 1982 (Abstract only) Science Citation Index Well-writing: teaching users how to write Chinese, not recognize Chinese handwriting [YasuharaM82a] Yasuhara, M. and Yasumoto, Y. "An Improved Adaptive Predictor in DPCM Based on Kalman Filter and Its Application to Handwriting Signal Encoding", Working paper submitted to IEEE Trans. on Comp., Univ. of Electro-Commuications, Japan, March 25, 1982 Encoding of handwriting motion/motor input based on model of handwriting motion [YasuharaM83a] Yasuhara, Makoto "Identification and Decomposition of Fast Handwriting System", Working paper, to appear in IEE Trans. on CAS, Vol 30 No 11, Novm 1983 Model of handwriting dynamics motion for fast handwriting (more dynamic) [Yedwab81] Yadweb, Laura; Herot, Christopher F.; Rosenberg, Ronni L.; and Gross, Carol "The Automated Desk", SigSmall Newsletter, Vol 7 No 2, October 1981, pp 102-108 Shneiderman83 cites for direct-manipulation desktop user interface. Desktop UI displayed on charactermode terminal. No stylus/digitizer input shown, uses joystick, arrow keys, keyboard. [Yhap81] Yhap, E.F. and Greanias, E.C. "An On-Line Chinese Character Recognition System", IBM Journal of Research and Development, Vol 25 No 3, May 1981, pp 187-195 National differences/variability in writing styles (Japanese, Chinese, Korean) Claimed 97.8% accuracy by excluding 5% of data as "poorly written" Stroke order, stroke connection variations in Chinese The 214 standard Chinese radicals "too many": some are rare Recognition diagram: signal filter, segment and direction, stroke Classification, alphabet/element recognition, composite ideograph output Contrast: features of primitive "stroke element" recognition vs chain code segments



[Yoshida82] Yoshida, K., and Sakoe, H. "Online Handwritten Character Recognition for a Personal Computer System", IEEE Trans. Consumer Electronics, Vol CE-28 No 3, pp 202-209, August 1982 System connects strokes of "normal" to make "running" forms (variability?), feature is angle sequence (chain codes?) Claims 99.5% accuracy on handwriting recognition [Yoshida83] Yoshida, K., and Sakoe, H. "Online Character Recognition by Stack DP Matching Method", PRL83-29, Sep. 27, 1983 (in Japanese, abstract in English) Character represented as branches in a reference pattern feature sequence. Detail "discrimination logics" for character which the regular method cannot handle. Kanji and Hiragana handwriting biblio80 (Prev page)

Main Page







Main Page


[ANSI84] ANSI "Draft Proposed American National Standard Graphical Kernel System", American National Standards Institute, Incorporated, X3.123-198x, 1984 GKS graphics API standard [Ahmed84] Ahmed, P. and Suen, C.Y. "Edge classification and extraction of shape features", Proceedings of the 7th International Conference on Pattern Recognition, 1984, pp 593-596 Cited in Li89 [Aleksander84] Aleksander, I., Stohnam, T.J., and Wilkie, B.A. "Recognition Apparatus", United States Patent 4,490,847, December 25, 1984, assigned to National Research Development Corporation, London, England. Video image recognition, based on learning (neural) networks [AllesHG85a] Alles, Harold G. and Kubik, Peter S. "Deformable Touch Sensitive Surface", United States Patent 4,542,375, September 17, 1985 Touchscreen digitizer using transparent deformable surface over a CRT display. Principle somewhat similar to light pen: where surface is pressed, it bends closer or makes contact with the front of the CRT display, causing light from the electron beam to be reflected by total internal reflection. One or more light sensors on the side detect the light, and from the (timed) position of the CRT spot, know the location. Should be able to detect multiple touches, since reflected light is detected all around the display. [Apple84] Johnson, Lynnea "Macintosh MacWrite", Apple Computer Inc, 1984, Product Number #M1502 (hardcopy book) User manual and introduction to MacWrite word processing application [ArcelliC85a] Arcelli, C. and Sanniti di Baja, G. "A Width-Independent Fast Thinning Algorithm", IEEE Trans. Pattern Analysis and Machine Intelligence", Vol PAMI-7 No 4, July 1985, pp 463-474 Skeletonization/line-thinning with a method to "recover" the original outline by building the skeleton back up Line thinning by topology-preserving removal operations [ArtwickB84a] Artwick, Bruce "Applied Concepts in Microcomputer Graphics", Prentice-Hall, 1984, ISBN 0-13580226-1 Describes viewports and Windows as projected coordinate calibration on a digitizing tablet. Early CAD



menus printed statically on the digitizer surface as a template. [Asija85] Asija, Satya P. "Automated Image Input, Storage and Output System", United States Patent 4,523,331, June 11, 1985 Automated image input, including handwriting recognition, and output, including generation Shows characters normalized to a 5x7 grid, and stored based on bit string of traversal of squares Recognition trained to each user: matches are by binary-value distance between two bitstrings Note: Compare with input technology of Casio PF-8000 calculator: Casio84a_PF8000 [Autodesk84] Autodesk "AutoCad 2 (tm) Drafting Package User Guide", #106-006, Autodesk Incorporated, 2320 Marinship Way, Sausalito CA 94965, 1984 [BaeckerR85a] Baecker, Ronald Michael "Curriculum Vitae, Ronald Michael Baecker", Ronald Maechel Baecker, Nov 29, 1985 CV and publication list as of 1985 for Ronald Baecker. Founder of HCR, Unix-based software start-up. [Baird84] Baird, H.S. "Model-based Image Matching Using Location", PhD Thesis, Department of Electrical Engineering and Computer Science, Princeton University, October 1984 Pattern matching by a template in feature-space, looking for minimum distances of feature points in a multi-dimensional vector. Mathematical pattern-matching thesis, without any examples of any specific patterns it was tried on! [BarkerPG84] Barker, P.G., Najah, M. and Roper, J.S. "User Experiences with a MICROPAD", Journal of Microcomputer Applications, Vol 7, 1984, pp 19-39 Cited in BarkerPG85 [BarkerPG85a] Barker, P.G. and Janah, M. "Implementing pictorial interfaces using a high-resolution digitizer", International Journal of Man-Machine Studies, Vol 23, 1985, pp 153-173 User interface picking menus from parts of pictures (graphical menus) -- happened to use a Micropad digitizer, not really handwriting recognition [BarkerPG85b] Barker, P.G. and Janah, M. "Pictorial interfaces to data bases", International Journal of ManMachine Studies, Vol 23, 1985, pp 423-442 User -interface using menus on picture parts, not lists, pick-boxes from a Micropad terminal [BeckerJ84] Becker, Joseph D. "Multilingual Word Processing", Scientific American, Vol 251 No 1, July 1984, pp 96-107 Character/word order, user-interface design for multiple language word processing (Arabic/Hebrew/English/Roman/Chinese) Korean is written in syllable clusters, Hebrew right-to-left, Arabic right-to-left with words diagonally, Mongolian vertically [BeckerJ85] Becker, Joseph D. "Typing Chinese, Japanese, and Korean", IEEE Computer, January 1985 Keyboard input for Chinese Kanji [Belaid84] Belaid, Abdelwaheb and Haton, Jean-Paul "A Syntactic Approach for Handwritten Mathematical Formula Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-6 No 1, January 1984, pp 105-111 Refers to ambiguity of nominally straight vs curved lines Mentions P/D, T/+ ambiguity Investigate highlevel context and ambiguity in mathematical formulas Refers to min/max extents, centroid of characters Says some irresolvable "confusion" ambiguity solved by user interaction User interface for 2-D mathematical symbol input: handwritten sketch input [Berkin84] Berkin, G.M., and Negin, Michael "Pattern Recognition System", United States Patent 4,441,205, April 3, 1984, assigned to Kulicke and Soffa Industries, Incorporated, Horsham, Pennsylvania. Alignment pattern recognition for video scanning of integrated circuits, small parts [BinaryFusion85a] Binary Fusion Software Archive "Borland Sidekick", www.tripod.com, Fetched October 2010 Downloadable install files (zip archive) for Borland Sidekick: files dated 1985. Mentions feature of onscreen dialer for telephone numbers.. [BirrellAD84a] Birrill, Andrew D. and Nelson, Bruce Jay "Implementing Remote Procedure Calls", ACM Trans. on Computer Systems, Vol 2 No 1, February 1984, pp. 38-59 Design review of RPC implementation on Cedar: cites to RFC707 1976. Practical problems of network failure, address/pointer parameters (to be marshalled), binding to target/callee, authentication and security: PKI/encryption for authentication etc. [Blesser84] Blesser, Barry "Comments on Variability due to Pen-Artifacts", internal report, Pencept, Incorporated September, 1984 Apparent variability of handwriting in on-line recognition caused in part by defects/performance of tablet digitizers: retrace, hooks, etc. [BlesserB85a] Blesser, Barry "Tablet Error Analysis", internal report, Pencept, Incorporated September, 1985 Mathematical measurements of tablet performance, focusing on defects and quirks of different types of digitizing tablets: describes effects of search algorithms used to locate stylus, local stretch, external voltages, electronic defects, etc.



[BlesserB85b] Blesser, Barry "Tablet Technologies", internal report, Pencept, Incorporated September, 1985 Review of tablet technologies: hundreds of types, multiple categories -- resistive=spatially continuous; grid electrostatic/electromagnetic=spacialy discrete, pulse excitation, sinewave continuous excitation, contact control (tip switch): cases study of Pencept tablet technology. [BoieRA84a] Boie, R. A. "Capacitive Impedance Readout Tactile Image Sensor", Proc. Int. Conf Robotics. Washington, D.C., 2004, pp 370-378 (abstract only) 8x8 / 32x32 capacitive tactile sensor touchpad for robotics, matrix of X and Y conductors measuring capacitance change at each crossing to form an image. Cited by Buxton for early multi-touch. [BoieRA85a] Boie, Robert A. and Miller, Gabriel L. "Conformable Tactile Sensor", United States Patent 4,526,043, July 2, 1985 capacitive tactile sensor touchpadfor robotics or touchscreen; center of areas of contact easily determined; elastomeric/elastic dialectric material between two layers of grid, matrix of X and Y conductors measuring capacitance change at each crossing to form an image. Multi-touch. Multiplexing is analog. [BoiesSJ85a] Boies, Stephen J., Gould, John D., Levy, Stephen, Richards, John T., Schoonard, Jim "The 1984 Olympic Message System -- a Case Study in System Design", IBM Research Division Report RC 11138 (#50065), 5/2/85, 53 pages Design study of early voice-mail/messaging system prototype for 1984 olympics. Emphasizes empirical testing and design iteration for usability -- compare with Agile methodology? [BoivieRH84a] Boivie, Richard H. "Directory look-up method and apparatus", United States Patent 4,453,217, June 5, 1984 Spelling correction by recursive division of the dictionary: skip one character at a time: interesting Dictionary searching method [BoivieRH85a] Boivie, Richard H. "Character Recognition Arrangement", United States Patent 4,525,860, June 25, 1985 Described curve tracer/follower for OCR of handwritten characters OCR using line thinning, then simulated chain codes generated by a curve tracer/follower: claims to be font-independent / handwriting recognition "standard spelling": coded sequence of chain codes Refers to two approaches for character recognition: theoretic (or discriminant) versus syntactic (or structural) pattern recognition approach: dislikes both Subpatterns (features) of a handwritten character can only be recognized in the (graphical) context of the entire character pattern Thinning handwritten characters and comparing with dictionary of chain codes for recognition? [BonoPR85a] Bono, Peter R. "A Survey of Graphics Standards and Their Role in Information Interchange", IEEE Computer, Vol 18 No 10, October 1985, pp 63-75 User-interface management system (UIMS) and graphics standards: GKS, PHIGS, etc. See also Bono presentation on handwriting character recognition in standards. [BonoPR85b] Bono, Peter R. "Software Standards: Which Ones Are Here to Stay?", S. Klein Computer Graphics Review, Inaugural Issue, 1985 Permanence/effect of standards depends on whether major manufacturers support them, base produces on them: examples GKS Graphics Kernel System and CGM Computer Graphics Metafile. Overview of standards-making process (GSK, PHIGS, etc.) [Borland84a] Borland International "Sidekick Version 1 Owner's Handbook", Borland International, 1984 PIM personal information manager for DOS: calculator, notepad, auto-dialer (finds telephone numbers by screen scraping). Character-mode windows. [Borland85a] Borland International "Sidekick Version 1.5 Owner's Handbook: The Desktop Organizer just a Keystroke Away", Borland International, March, 1985 PIM personal information manager for DOS: calculator, notepad, auto-dialer (finds telephone numbers by screen scraping, or from telephone directory file / contact database) using Hayes 1200/1200B modem. Character-mode windows, windows auto-rearrange. [BozinovicR84a] Bozinovic, R. and Srihari, S.N. "Knowledge-based Cursive Script Interpretation", Proceedings of the 7th International Conference on Pattern Recognition, Montreal, 1984, pp 774-776 Cites own poor performance if did not exclude "hill and dale" sloppy writing, crowded writing Assumes a premise that people use script, not printing, in normal writing [BraultJJ84a] Brault, J.J. and Plamondon, R. "Histogram Classifier for Characterization of Handwritten Signature Dynamic", Proceedings of the 7th International Conference on Pattern Recognition, Montreal, 1984, pp 619-622 Signature verification: small sample size: 243 signatures of 50 people: test protocol had no forgery attempts (!) [BrocklehurstE85a] Brocklehurst, E. "Computer methods of signature verification", Journal of Forensic Science, Vol 30, 1985, pp 445-457 Cited in Yoshimura91 [BrogardhT84a] Brogardh, Torgny "Device for Simultaneous Creation and Detection of Markings", United States



Patent 4,475,240, October 2, 1984, assigned to ASEA Aktiebolag, Vasteras, Sweden. Digitizer tablet using a light-sensitive optical sensor to detect markings as they are written/drawn [BrogardhT85a] Brogardh, Torgny "Optical Digitizer/Position Measuring Device", United States Patent 4,531,230, July 23, 1985, assigned to ASEA Aktiebolag, Vasteras, Sweden. Fiber-optic optical digitizer tablet using a grid of an optically-sensible pattern [Brooks85] Brooks, C.P. and Newell, A.F. "Computer Transcription of Handwritten Shorthand as an Aid for the Deaf -- a Feasibility Study", International Journal of Man-Machine Studies, Vol 23, 1985, pp 45-60 Character recognition for shorthand DCR for shorthand much more complex than was thought 1/4 to 1/2 of words in spoken conversation unintelligible out of context Hooks, loops, circles are "real" features in shorthand Refers to endpoint trimming on digitizer strokes "invariant" base form, and numerous real written forms for one writer Hard to write vertical curved strokes reliably [BrownG85] Brown, G.P., et al "Program Visualization: Graphical Support for Software Development", IEEE Computer, Vol 18 No 8, August 1985, pp 27-35 Interactive graphical input and display interface languages for programming: compare with layout editors? Digitizer using separately mounted Lexan sheet over physical display (with shift in position?) [BrownMR85a] Brown, Mark R.; Kolling, Karen N.; and Taft, Edward A. "The Alpine File System", ACM Trans. Comp.Systems, Vol3 No 4, November 1985, pp. 261-293 file system similar to Multics file system: part of Cedar system. Access control list uses sequence syntax of RNames for users and/or groups with type of access granted (limit to 20 RNames in syntax): RName groups can be nested heirarchy expression. [BrownRM85] Brown, Robert M. "Handprinted symbol recognition system", United States Patent 4,491,960, January 1, 1985, assigned to the United States of America as represented by the Secretary of the Navy. Cited in Sklarew88b: on OCR of handwriting? uses logic tree of robust features first, then lesser discriminants, then a quality-assurance module filtering/noise-removal: (1) pre-recognition ("trash" filter), (2) potential symbol identification, and (3) final quality assurance; for on-line handwriting recognition [BuxtonW85] Buxton, William "Notes on Chord Keyboards", Computer Systems Research Institute, Univ. of Toronto, Toronto Ontario CA M5S 1A4 mentions WriteHander chord keyboard from NewO Corporation [BuxtonW85a] Buxton, William, Hill, Ralph, and Rowley, Peter "Issues and Techniques in Touch-Sensitive Tablet Input", SIGGRAPH 85, San Francisco July 22-26, Vol 19 No 3, 1985, ACM 0-89791-166-0/85/007/0215, also Computer Graphics Vol 19 No 3. Also on billbuxton.com: fetched 2000 Describes virtual tablets, multiple virtual devices on a tablet. Use of pressure/force and multiple-touch input devices. Refers to inaccuracy ("jitter") of touch tablets / touchscreens at light pressure, such as at end of stroke/touch-lift, but does not mention inaccuracy of lifting motion. (Minimum distance for pressand-hold). Virtual switch device, compare with Apple slide/swipe-to-unlock gesture. Advantages of multiple virtual devices on a display, for which multi-touch input would be especially advantageous. Two levels of pressure/force (haptic?). Painting with pressure-sensitive tablet. Color mixing with multiple touch sliders. Strong advantages to using multi-touch input. Also: Electronic video file of demonstration. Cited in Wanderley00, but with wrong authors? [BuxtonW85b] Buxton, W., Sniderman, R., Reeves, W., Patel, S. and Baecker, R. "The evolution of the SSSP score-editing tools", In Foundations of Computer Music, C. Roads and J. Strawn, Eds, MIT Press, Cambridge, Mass 1985, ch 22, pp 387-22; Originally published in Computer Music Journal, Vol 3 No 4 Issue 12, December 1979; pp 14 ff. Gestures/marks for editing musical scores, handwriting music recognition. Recognition based on changes in angles only, not lengths of segments. Semantically uses multiple segments to indicate note duration, but does not treat as compound recognition. Shapes later copied by PLI group at IBM for music editing demo. Also available at http://www.billbuxton.com [BuxtonWB85d] Buxton, William "A Multi-Point Three-Dimensional Touch Sensitive Tablet", University of Toronto, 1985: available at billbuxton.com Video to accompany "A Multi-Touch Three Dimensional Touch-Sensitive Tablet". Lee's capacitive-matrix multi-touch tablet. [Byte84a] BYTE Magazine "Real World Interfacing", BYTE Magazine, Vol 9 No 4, April 1984 Pencept advertisement for handwriting recognition tablet for IBM PC: "Is the Penpad 320 the greatest graphics tablet in the world? Or, is it the greatest keyboard and mouse? [Byte85a] BYTE Magazine "Bargain Computing", BYTE Magazine, Vol 10 No 3, March 1985 Space Tablet three-dimensional/3D sonic digitizer pointer, Soniture Inc. KoalaPad digitizer tablet. [CIC84a] CIC "Handwriter (TM) Product Literature (in Japanese)", CIC Japan Incorporated, Suite 918, Shuwa Kioicho TBR Building, 5-7 Kojimachi, Chiyoda-Ku, Tokyo 102, Japan, 1984 Forms data entry handwriting recognition product from CIC Handwriter with tablet. States 99% correct recognition, 0.5..1.0 seconds per character input speed.



[CIC84b] CIC "The Handwriter: A Quiet and Natural Data Entry System", Communication Intelligence Corporation, Menlo Park, CA, 1984 (date approximate) Information in English on CIC Handwriter recognition tablet. [CIC85a] CIC "Handwriter (R) focus:ABC (TM) Accounting Package", Communication Intelligence Corporation, 800 El Camino Real, Menlo Park, CA 94025, 1985 (hardcopy file) Spreadsheet from CIC Handwriter product [CIC85b] CIC "Handwriter (R) Lotus 1-2-3 interface Kit", Communication Intelligence Corporation, 800 El Camino Real, Menlo Park, CA 94025, 1985 (hardcopy file) Word-processing from CIC Handwriter product [CIC85c] CIC "Handwriter (R) WordStar (R) interface Kit", Communication Intelligence Corporation, 800 El Camino Real, Menlo Park, CA 94025, 1985 (hardcopy file) The ASCII handwriting recognition product Hew Crane said did not exist [CIC85d] CIC "Handwriter (R) GrafText (TM) System Model GT-5000 / Host Communication Protocol Manual", Communication Intelligence Corporation, 800 El Camino Real, Menlo Park, CA 94025, June 1985 (hardcopy file) Similar product to Pencept PenPad. Describes recognition modes and templates for forms: graphic, text, remote-function, local-function areas. Calibration of tablet to paper form. Handwriting recognition returned multiple matches (up to 5). Electronic ink display in tablet driver. [CItoh85] CItoh Corp "CI-Handwriter Fit-In-1 product information", 1985. (OEM version of CIC Handwriter product). Japanese handwriting character recognition product [CalComp85a] CalComp "CalComp 2500 Series Digitizer Operator's Manual User's Guide", CalComp Incorporated 50293-1, 1985 [CalComp85b] CalComp "CalComp 2500 Series , 9500 Series Digitizer product information", CalComp Incorporated CalComp digitizer tablet: shown with stylus, four-button cursor/puck, 16-button cursor/puck with reticule [CardelliL85a] Cardelli, Luca and Pike, Rob "Squeak: a Language for Communicating with Mice", Proc. SIGGraph '85, San Francisco, pp. 199-204 Tutorial on Squeak, programming language for UI implementation using multiple input streams e.g. keyboard input stream and mouse event input stream. "Menu" event on press-and-hold, when mouse button depressed for a significant time (presumably without being moved). [CarnesWR84a] Carnes, W.R. "Signature Verification System", United States Patent 4,443,436, February 21, 1984 From patent list [Carpenter85] Carpenter, G.A. and Grossberg, S. "Neural Dynamics of Category Learning and Recognition: Attention, Memory Consolidation, and Amnesia", in "Brain Structure, Learning, and Memory", Davis, J., Newburgh, R., and Wegmen, E. editors, AAAS Symposium Series, 1985 Adaptive recognition: refers to "as recognition categories stabilize" in learning [CarrTH84] Carr, T.H., Brown, T.L., Brown, J.S., and McDonald, J.L. "Handwriting of connected discourse: Speedaccuracy tradeoffs in a skilled performance", Proceedings of Psychonomic Society, November, 1984 Cited in Rhyne86: : was in Tappert's group 1986 Rhyne86 cites this that mental set affects legibility of handwriting (variability) [Casey84a] Casey, R.G. and Nagy, G. "Decision Tree Design Using a Probabilistic Model", IEEE Trans. Information Theory, Vol IT-30 No 1, January 1984, pp 93-99 Applications: single-font OCR system had to replace five keypunch operators at prices of 1975 Parallelism vs serial: decision tree is best technique for speed/cost tradeoffs in character recognition Paradigm: iterative optimization of decision tree by designer vs automatic construction of decision tree (designer specifies features to use) Decision trees in OCR: for small trees, need very uniform styles and pixel data (is not that obvious?) Estimated versus actual performance: adding more OCR pixels does not help much since they are not significant features (size of character/resolution) Testing: reports real results on trading reject for substitution errors [Casio84a] Casio "Casio PF-8000 Calculator user manual (in Japanese)", Collection from g-mark.org, http://www.g-mark.org/library/supercollection/g40/1984/index-20.html, 2002 1984 Calculator with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad. Also two physical devices in collection. Not sure about relationship of PF8000 and IF8000 products unistroke / single-stroke characters (?) Dr. Theodore Kuklinski had one in his collection in 1990 Additional material in file: Casio/Casio Data Bank PF-8000: http://www.geocities.co.jp/Technopolis/6766/PF-8000.htm, full-frame picture Additional reference on Casio Data Bank PF-8000: http://www.steamtrain.de/floh/bilder_2/casio_data_bank.jpg, showing writing styles [Casio84b] Casio "Casio PF-8000 Calculator user manual (in English and Spanish)", Casio 123B M publication, date believed to be 1984 1984 Calculator with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad, separate from the display Shows a "gesture" character for editing (e.g.



backspace/delete), demonstrated in a name/address-book application included in the unit. Functional unit in personal collection of Jean Renard Ward, this is the user documentation. Unistroke / single-stroke characters. See also video files. Includes a reference sheet for writing single-stroke characters, similar to the "Graffiti" unit by U.S. Robotics in the 1990s [Casio84c] Casio "Operation Manual Casio PF-8000 Calculator (in English)", Casio, 1984 1984 Calculator with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad. [Casio84d] Casio "Casio Electric Notebook PF-8000", www.g-mark.org/library/supercollection/g40/1984/index2.html (fetched 2002) Description (in Japanese) of 1984 Calculator with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad [Casio84e] Casio "Casio PF-8000 Calculator", In personal collection of Jean Renard Ward -- cover missing, approximately 34 pages (physical device) 1984 Calculator with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad [Casio84f] Casio "Casio TC-500 touch screen calculator watch product information", Casio 1984 wristwatch with touchscreen input, handwriting recognition for digits and arithmetic operators. See also TC-550, TC-600. File contains user guide. [Casio84g] Casio "Casio AT-550 Touch Screen Calculator Watch: Video", Casio Video of 1984 wristwatch with touchscreen input, handwriting recognition for digits and arithmetic operators. Uploaded to YouTube.com by Bill Buxton, 2012. [Casio84h] Casio "Casio PF-8000 Data Bank PF-3000 Data Bank Screen-Write your data", Casio Photo brochure of Casio PF-8000 Data Bank touchscreen/handwriting calculator/PDA: shows details of touchscreen [CasioCalculatorWatch84a] Casio Calculator Watch "Vintage Casio TC-600 Touch Screen Calculator Retro LCD wristwatch", casiocalculatorwatch.net, fetched 2010 (linked index) Casio wristwatch with unistroke character input on touchscreen, circa 1984: indexed under 2010 (date fetched) [Castleberry85] Casteleberry, Donald E. and Becker, Charles A "Electroscopic information display and entry system with writing stylus", United States Patent 4,520,357 May 28, 1985, assigned to General Electric Company Sklarew cites as electroscopic/electrostatic tablet/display integrated device Capacitive-coupling digitizer (with a magnetic stylus) integrated with a matrix display [Chainer85a] Chainer, T.J., and Worthington, T.K. "Segmentation Algorithm for Signature Verification", United States Patent 4,553,258, November 12, 1985, assigned to International Business Machines, Armonk, New York. Finds pen lift/down stroke start/end via A.C. coupling to a force/pressure gauge [Chainer85b] Chainer, T.J., Gundersen, S.C., and Worthington, T.K. "Semi-Independent Shifting Technique for Signature Verification", United States Patent 4,553,259, November 12, 1985, assigned to International Business Machines, Armonk, New York. Force/pressure and acceleration information on signature verification correlating data by shifting it [Chainer85c] Chainer, T.J., Scranton, R.A. and Worthington, T.K. "Data Input Pen for Signature Verification", United States Patent 4,513,437, April 23, 1985, assigned to International Business Machines, Armonk, New York. Digitizer stylus pen with force/pressure and acceleration sensitivity for signature verification: Better force/pressure-sensitive (piezo-electric sensor) pen stylus than Herbst 4,142,175 [Chainer85d] Chainer, T.J. and Worthington, T.K. "Shifting Technique for Signature Verification", United States Patent 4,562,592, December 31, 1985, assigned to International Business Machines, Armonk, New York. Shifting (to reduce correlation computation) of segments of signatures for acceleration and force/pressure data for signature verification [Chalkboard84a] Chalkboard "Electronic Chalkboards: Powerpad Product Information", Chalkboard Inc, 1984 Educational software using low-resolution digitizer tablet for VIC-20, Commodore 64, IBM-PC (DOS), Apple-II and Atari computers. User interface is mylar overlays with pre-printed simplified keys for child use. Educational software include Mico Maestro for music, 'Lectric Paintbrush, Music Math. Additional material in paper file [Cheung85] Cheung, Y.S. and Leung, C.H. "Chain-code transform for Chinese character recognition", Proceedings of IEEE International Conference on Systems, Man, and Cybernetics", Tucson, Arizona, 1985, pp 42-45 Japanese/Kanji/Katakana product from Hew Crane and James Dao at CIC: Cited in Leung87 [ChunK85a] Chun, Kukjin and Wise, Kensall D. "A High-Performance Silicon Tactile Imager Based on a Capacitive Cell", IEEE Trans. on Electron Devices, Vol ED-32 No 7, July 1985, pp. 1196-1201 / Technical report 187, U. Michigan, May 1986 Tactile sensor for robotic touch: force/pressure capacitor of deformable pad/outer skin over silicon capacitive sensors in 8x8 array, may be attached to glass, may be combined to give 16x16 array.



capacitive matrix touchpad sensor, intended for robotics applications. Matrix of capacitive forse sensors on 2mm centers, produces X/Y image of touch pressure (hence the name). Compare with Rekimoto multitouch tablet. [Cobblestone85] Cobblestone Design "Pencept: Software Control at the Stroke of your Pen / Write into your computer", www.cobblestonedesign.com/Pages/frames/Pencept.html Photograph of trade-show booth from Comdex for Pencept, handwriting recognition. [ComerfordR84a] Comerford, Richard "Pointing-device innovations enhance user/machine interface", Electronics Design News, July 26, 1984, pp 54-66 (hardcopy book) Appears in Sherr 1988 "Input Devices". Review of digitizer/tablet/joystick/mouse/touchscreen/trackball vendors, 1984 [Commodore84a] Beekman, George "The Commodore 64 Home Companion", DataMost, 1984, illustrations by Martin Cannon Instruction and software catalog for Commodore 64 home computer, and various third-party software and devices. Describes KoalaPad touch digitizer, and also ChalkBoard Power Pad: Power Pad uses different construction and can sense and report more than one touch at a time: multi-touch. [ComputerMuseum84a] The Computer Musecum "The Computer Museum Report Winter: Grand Re-opening", The Computer Museum Report, Winter 84/85 Vol 11 TCMR-V11 Report on Computer Museum physical exhibits: includes original RAND tablet digitizer, GRAIL (GRAphical Input Language) programming, machine recognition of handwriting (hand-printing). [Computerwoche84] Computerwoche "Das Penpad-Terminal, zur handschriftlichen Dateneingabe.", Computerwoche, May 4, 1984, www.computerwoche.de Press announcement of PenPad 200 handwriting recognition computer terminal, using pre-defined forms on a digitizer tablet. "Auf das Tablet werden Formulare gelegt, deren Einteilung der Benutzer vorher nach seinen Bedürfnissen festgelegt hat." [ConvisDB85a] Convis, D.B., Grim, P.J., and Reed, M.A. "Retro-stroke Compression and Image Generation of Script and Graphic Data Employing an Information Processing System", United States Patent 4,550,438, October 29, 1985, assigned to IBM Corporation, Armonk, New York. Uses local-maxima chain codes for image compression of script [Cook84] Cook, Mike "Light Pen", The Micro User, Vol. 2 No. 1, March 1984 Light pen - software techniques (via rubber-banding) to get single-pixel resolution from a light pen that only has resolution to character cells in a character CRT display [Crane85a] Crane, H.D., Ostrem, J.S., and Edberg, P.K. "Method for Distinguishing between Complex Character Sets", United States Patent 4,531,231, July 23, 1985, assigned to Communication Intelligence Corporation, Menlo Park, California. Attempt to patent Pencept's "area" definition on Penpad 320: Kanji (Chinese) in one area, Kana/Roman in another (Katakana/Hiragana Japanese) Kanji/Kana by top half / full height of character box (similar to upper/lower case) [Crane85b] Crane, H.D., and Ostrem, J.S. "Complex Pattern Recognition Method and System", United States Patent 4,561,105, December 24, 1985, assigned to Communication Intelligence Corporation, Menlo Park, California. Chines/Kanji character recognition using stroke shape and curvature Distance metrics between strokes for handwriting recognition of Chinese/Kanji characters Allows for shift in position, rotation of strokes to allow for better recognition matches [DOD85a] United States Department of Defense "Department of Defense Standard: Trusted Computer System Evaluation Criteria, December 1985", DoD 5200.28-STD, December 26, 1985, supersedes CSC-STD-001-83, 15 Aug 83 Requirements document for secure computer systems: compare with documentation on Multics for implementation of a secure computer system [Davies85] Davies, K. "Continuous Speech Recognition", Probe Research Seminar: The Coming of Age of Voice Recognition, San Francisco California, 1985 Cited in Mangione86: : member of IBM Yorktown Hts. staff (with Tappert) Adaptive recognition dies trying to get complete set of samples [DosterW84a] Doster, W. and Oed, R. "Textbearbeitung auf Personal-Computern mit handschriftlicher Direkteingabe", presentation at Arbeitstagung der Fachgruppe Interaktive Systems der Gesellschaft für Informatik, March 1984, in Zurich, Switzerland. Reprinted in PC-Praxis. Author's address: AEG Aktiengesellschaft, Research Center Ulm, Sedanstrasse 10, D-7900 Ulm, West Germany. Points out their variability model cannot be statistically verified. Describes AEG's general DCR work-station, including driver design for tablet digitizer [DosterW84b] Doster, W. and Oed, R. "Word Processing with On-line Script Recognition", IEEE Micro, October 1984, pp 36-43



AEG's word-processing user interface for handwriting recognition input. Describes word processing (for handwriting user interface) as two phased: text creation, and text editing. Some use of gestures or recognition macros, separate gesture, pointing (virtual tablet) and text input areas on separate digitizer from display. Compare with PenCept PenPAD and recognition macros, virtual tablet definitions. [DosterW84c] Doster, W. "Das papierlose Büro: Lesemaschine für Handschriften", Funkschau (West Germany), Vol 25, 1984, pp 61-63 General article on AEG's handwriting recognition system for forms data entry and word processing in an electronic automated office [DosterW85a] Doster, W. "Pattern Recognition Techniques as a Stimulus for Advanced Text Processing", PROTEXT II: Proceedings of the 2nd International Conference on Text Processing Systems, October 23-35, Dublin, Ireland, pp 155-161 Says putting in multiple variants a deficiency of non-adaptive Says "user-definable" special gesture symbols (for WordStar word processor) better for command, but his examples are of modified standard characters Points out that non-adaptive systems are user-adaptive, and results are the same "perfect" mutual enrollment [DosterW85b] Doster, W. and Schuermann, J. "Bildanalyse von Textdokumenten und Handschriftliche Direkteingabe - Zwei Verfahren Auch für BTX-Anwendungen", Informatik Fachberichte 92, Springer-Verlag, New York, 1985, pp 247-265 (partial copy) Two research projects at AEG on the automated electronic office: automatic document analysis for O.C.R. images of text and pictures, and on-line handwriting recognition [Downton84] Downton, A.C. and Brooks, C.P. "Automated machine shorthand transcription in commercial applications", Proceedings of INTERACT '84, IFIP Working Group 6.3, London, United Kingdom, September 4-7, 1984, pp 151-156 Need whole conference proceedings: user interface with handwriting editing. [Dye84] Dye, R., Newell, A.F., and Arnott, J.L. "An adaptive editor for shorthand transcription systems", Proceedings of INTERACT '84, IFIP Working Group 6.3, London, United Kingdom, September 4-7, 1984, pp 157162 Need whole conference proceedings: user interface with handwriting editing. van Raamsdonk? [EdgarAD84a] Edgar, Albert D. "Apparatus and Method for Remote Displaying and Sensing of Information using Shadow Parallax", United States Patent 4,468,694, August 28, 1984 Optical shadow parallax pointing input combined with display (so no registration problems). Direct pointing to documents and objects on desk surface. By using multiple shadows or multiple cameras, can extract height information from tablet image, to determine touch and lift. Compare with Wellner? [Ejiri85] Ejiri, Koichi "Method of Assigning Direction Code to Boundary Picture Element in Character Recognition System", United States Patent 4.524,454 Line-thinning, extraction of plausible writing direction from edges of scanned OCR images of characters, then to encode as direction codes / chain codes (There is another paper on this sort of thing in here someplace ...) [ElGamalT85a] El Gamal, Taher "A Public Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms", IEEE Trans. on Information Theory, July 1985, vol. IT-31 no. 4 pp. 469..472 Public-key PKI signature algorithm, includes section on mathematical attacks. [Epson84a] (unknown) "Epson RC-20 wristwatch", homepage.mac.com, fetched 2010, posted 1998 (in Japanese, machine translation in file) Touchscreen wristwatch, compared to Casio VDB-1000 databank wristwatch with touchscreen. [Ewing85] Ewing, J., Mehrabanzad, S., Sheck, S., Ostroff, D. and Shneiderman, B. "An Experimental Comparison of a Mouse and Arrow-Jump Keys for an Interactive Encyclopedia", to appear in International Journal of ManMachine Studies, September, 1985. Author's address: Department of Computer Science, University of Maryland, College Park, Maryland 20742 Pre-publication version only User-interface on pointing methods: applies to mouse emulation on a tablet? [Flurry85] Flurry, G.A. "Electronic Handwriting Facility", IBM Technical Disclosure Bulletin, Vol 27 No 9, February 1985, pp 5364-5366 Description of IBM's handwriting work-station? [FoleyJD84] Foley, J.D., Wallace, V.L., and Chan, P. "The Human Factors of Computer Graphics Interaction Techniques", IEEE Computer Graphics and Applications, Vol 4 No 11, November 1984 The granddaddy of all user-interface papers for interactive graphics [FormsAutomation85] Forms magazine "One-step data entry system links forms with computers", forms automation product review, April, 1985, pp 34-41. Published by National Business Forms Association, 443 East Monroe Avenue, Alexandria VA 22301 Description of DataPad/ AnnoGraphics / Portable Computer handwriting recognition product: was marketed under several different name.



[Fox84] Fox, A.S., Kim, J. and Tappert, C.C. "Segmenter for Known Number of Characters", IBM Technical Disclosure Bulletin, Vol 27 No 7A, December 1984, pp 3691-3693 Segmentation when writing prototypes, because number of shapes is known [FushimotoH85a] Fushimoto, Hideo; Toyomura, Shigeru; Nishimuro, Yshiaki; Asakura, Osamu; and Rachi, Masahiro "Foldable Electronic Apparatus", United States Patent 4,517,660, May 14, 1985 Foldable calculator/PDA, side-by-side configuration, or close and use calculator keys on outside. Keyboard/keypad input, but full display. [GSS84] GSS "GSS-Drivers User's Guide", Second Edition, Graphic Software Systems, Incorporated, March 1984 [Gaines84] Gaines, B.R., McKellar, I.D., Dinger, W.P., and Fast, S.R. "Some Experience in the Real-time Processing of Handwriting", Proceedings of the 7th International Pattern Recognition Conference, Montreal, 1984, pp 630-632 Says natural user-interface is speech and handwriting together Says tablet digitizers with 1000x1000 resolution and 1% linearity good enough for handwriting (!) Say peak handwriting speed/velocity is 50 inches/second (?), normal is 2.5"/sec Says "skilled" user gets 97%-100%, "unskilled" only 70% recognition success [Genter85] Genter, R.E. "Guidelines for C Program Development", Internal report, Intermetrics, Incorporated, Cambridge, Massachusetts, September 1985 Programming/coding style [Gharachorloo85] Gharachorloo, N. "On-line Character Recognition Using Closed-Loop Detector", United States Patent 4,495,646, January 22, 1985 Tablet-free stylus using mechanical design to detect rough stylus direction of motion Claims handwriting recognition using mechanical sensing of chain codes [GlennWE84a] Glenn, W.E. "Apparatus for Determining Position and Writing Pressure", United States Patent 4,488,000, December 11, 1984, assigned to New York Institute of Technology, Old Westbury, New York. Force/pressure-sensitive digitizer tablet using acoustic sound waves in a sheet [GoldbergA84] Goldberg, Adele "Smalltalk-80: The Interactive Programming Environment", Addison-Wesley, Reading MA, 1984 Introduction to Smalltalk language and environment. (Electronic file.) [Goldwasser85] Goldwasser, Eric, and Goldwasser, Dorothy "Method of Creating Text Using a Computer", United States Patent 4,559,598, December 17 1985 Pen/stylus or other pointing means, select words from dynamic menus to input text. Intended as aid to the blind: compare with QwikWriting and other means for one-handed input of handwritten text with constrained recognition. [Gould84] Gould, J.D. and Alfaro, L. "Revising Documents with Text Editors, Handwriting-Recognition Systems, and Speech-Recognition Systems", Human Factors, Vol 26 No 4, August 1984, pp 391-406 Applications: text editing much faster with handwriting than with keyboard, even though same number of characters input User-interface: differences in external feedback greatly affect operational speed Userinterface: time spent inputting a command much less than think time (and typing adds to effective think time) User-interface: electronic ink would eliminate spatial displacement within written copy and screen copy Speech simulation of handwriting editing, but no standard proof-editing marks Speech: no visual feedback a minus on speech recognition test Speech: USERs make more errors in speech than with writing or typing Martin89 cites as saying most of time in producing a letter is think time, so user-interface of speech recognition or typing not that great a difference [Graham84a] Graham, Martin "Signature Verification Sensor", United States Patent 4,475,235, October 2, 1984, assigned to Rolm Corporation, Santa Clara, California. Capacitive sheet/plate digitizer tablet [GreenM85a] Green, Mark "The University of Alberta User Interface Management System / UIMS", Proc. SIGGRAPH '85, Vol 19, No 3, 1985, San Francisco July 22-26, 1985, pp. 205-213 UIMS based on Seeheim model: Presentation layer, Dialog Control (Interaction) layer, Application Interface Layer, with user interface / user input events and event handlers. [GreeneR85a] Greene, Richard "The Drawing Prism: A Versatile Graphic Input Device", Proc. SIGGRAPH '85, Vol 19, No 3, 1985, San Francisco July 22-26, 1985, pp. 103-110 Drawing prism: Touch surface/tablet system using total internal reflection and refraction. User marks with brush or any object on one side of a prism, video reads points of contact by viewing at an angle that only sees light reflection where surface it touched. Size up to 16" by 20" constructed in prototype from oil in plastic surfaces. Refers to multiple fingers in finger painting: multi-touch. Cites to Mueller patent using same principle. [HagenMD85a] Hagen, Mark D.; Mendel, Peter J.; Regehr, John L. "Rotatable Display Work Station", United States Patent 4,542,377, September 17, 1985 CRT display that can be rotated mechanically between landscape and portrait orientations. Algorithm for



rotate control logic to read characters out from memory for current orientation, with user scrolling for data off edge of display. [Hayamizu84] Hayamizu, S. and Oka, R. "Experimental Studies on the Connected-Words Recognition using Continuous dynamic Programming", Denshi Tsushin Gakkai Ronbunshi, Vol 67-D No 6, June 1984, pp 677-684 (in Japanese), translated in Systems, Computers and Controls, Vol 15 No 4, 1984, pp 80-88 Speech recognition: performance goes down with vocabulary size (everybody should show performance with increasing vocabulary size) Recognition errors are confusion (substitution), missing, and "ghost word" [HertzfeldA84a] Hertzfeld, Andy "Folklore - The Original Macintosh: Thunderscan scanner", folklore.org, June 1984: fetched 2015 Modification to early C. Itoh / Apple Macintosh printer to add optical sensing element, high-quality scanner. With large images, added inertial scrolling with coasting at variable speed, hysteresis. [Higgins84] Higgins, C.A. and Whitrow, R. "On-line Cursive Script Recognition", Proceedings of Interact '84, 1st IFIPS Conference on Human-Computer Interaction, September 1984, pp 140-144 Unlikely that "unconstrained" can be done, since humans find stuff illegible Segmentation at turning points, not Y-maxima/minima Features must have enough variety to be effective, but not be too sensitive Extra feature tests ("optional") for specific characters Context - "higher shape templates" for combined shapes Refers to including new shapes on the fly for mis-recognition Training is "manual: results in inaccurate feature values for templates" (?) From incorrect weighting, best word match does not always come out on top [Hitachi84] Hitachi, Limited: "Humanification product literature", (in Japanese), 1984 Hitachi Kanji/Chinese handwriting recognition product [HondaK85a] Honda, Kazuhiro and Kuramochi, Hiroshi "Pressure-sensitive and conductive rubber", United States Patent 4,533,604, August 6, 1985 Conductive rubber, pressure-sensitive: force on rubber causes carbon, metallic, or other particles to be pushed together, reduces resistance. Type of rubber (elastomeric silicone, etc.) with different density or specific gravity, particles tend to sediment [Honeywell85a] Honeywell Information Systems "MULTICS Project Administrator's Manual", Honeywell Information Systems AK51-02 February 1985 Multics sysadmin manual: as_who for sudo superuser operation, uses Multics ACLs access control listss [HouseVD84a] House, V. Dean "Touch controlled membrane for multi-axis voltage selection", United States Patent 4,444,998 two-sheet resistive film digitizer tablet, measure DC voltage. Also two resistive films on both sides of a conductive sheet. Compare with Krey digital pad? [Huang85] Huang, J.K. "The input and output of Chinese and Japanese characters", IEEE Computer, Vol 18, January 1985, pp 18-24 50,000 total Chinese characters: three-corner code method TCCM for encoding and indexing for keyboard input [Hulls85] Hulls, L.R. "Absolute Position Coordinate Determining Device employing a Single Phase Difference Measurement to Determine Cursor Position", United States Patent 4,552,991, assigned to Numonics Corporation, Lansdale Pennsylvania, November 12, 1985 Phase-shifting tablet digitizer (electromagnetic?), using not a simple grid, but in each ordinate a mixed set of loops at physically different phase. [HutchinsEL85a] Hutchins, Edwin L; Hollan, James D.; and Norman, Donald A. "Direct Manipulation Interfaces", Human-Computer Interaction, 1985, vol 1, pp. 311-338 Tutorial on direct-manipulation user interfaces circa 1985. Early examples of direct manipulation include Sutherland's Sketchpad 1963, ThingLab 1979, electronic spreadsheet programs early 1980's. [IBM85] Anonymous "Multi-Segment System for Recognizing Cursive Writing", IBM Technical Disclosure Bulletin, Vol 17 No 11, April 1985, pp 6735-6739 Segmentation algorithm for script (Tappert?) for run-on and script handwriting [IchikawaS84] Ichikawa, S. and Gyoba, J. "Methodological Problems on Pattern Psychophysics", (Japanese journal), Vol 27 No 2, 1984, pp 132-157 Not sure of English name of Japanese journal Similar to functional attributes, describes general problem of figuring out human recognition mechanisms for multiple features [IkedaE84a] Ikeda, Eiichi; Ikuo, Kayama; and Izaki, Masahiro "Pressure-sensitive conductive elastic sheet", United States Patent 4,448,837, May 15, 1984 Conductive rubber or elastomeric sheet used in touch digitizer: particles are ferromagnetic, coarse particles aligned by magnetic field perpendicular to sheet during fabrication, fine particles near surface. Gives better palm rejection (large area, light pressure does not cause conduction?) by making dispersal of particles more uniform (fewer shorts?) [Ikeuchi84] Ikeuchi, K. "A Model of Character Recognition by Humans", Proceedings of the 7th International



Pattern Recognition Conference, Montreal, 1974, pp 521-524 Three stages of human recognition: feature extractor, naming, decision Human features: straight line, turned line, end point, curved line Compare with Shillman [Jansen84] Jansen H. et al "Interpretation of freehand drawings for mechanical design processes", Computers and Graphics, Vol 8 No 4, pages 351-369, 1984: Pergamon Press Ltd, USA [Jelinek85] Jelinek, F. "The Development of an Experimental Discrete Dictation Recognizer", Proceedings of IEEE, Vol 73 No 11, November 1985, pp 1616-1624 Speech: gives results of (huge) number of samples needed to get statistically useful vocabulary coverage Speech: interface problem for word processing is not a trivial one Speech: editing (pointing) with pencilfundamentally better user interface than voice/speech recognition Speech: for editing, best interface will include digitizer and handwritten character recognition for editing marks (mode separation? ref. Buxton "tension") Speech: an adaptive system that does worse with non-native speakers of English (!) (Features are biased? Not really a general adaptive system?) Adaptive speech: cites user who did worse when analyzing own pronunciation (ref: problems of user understanding enrollment problems) [JohnstoneE85a] Johnstone, Eric "The Rolky: A Poly-touch Controller for Electronic Music", Proc. ICMC '85, pp. 291-295 Multi-touch digitizer using total internal reflection on glass: half-silvered mirror to CRT display to that hand does not obscure image. [KADesign85] KA Design group "business plan", March 1, 1985 Puck/pointer: proposed mouse-like tablet product using a small joystick. Contains patent references to Victor B. Kley, 4,435,616, Graphical Data Entry Apparatus, resistive sheet tablet digitizer. [Kahan85] Kahan, S. "Problems in recognizing hand-printed characters", M.S. Thesis, Department of Electrical Engineering and Computer Science, University of California at Berkeley, February 1985 Cited in Kahan87 [Kai-tungHuang85] Kai-tung Huang, J. "The Input and Output of Chinese and Japanese Characters", IEEE Computer, Vol 18 No 1, January, 1985, pp 18 - 24 Chinese and Japanese Kanji input via keyboard [Karat84] Karat, J., McDonald, J., and Anderson, M. "A comparison of selection techniques: touch panel, mouse, and keyboard", Technical report No TR-51.0166, IBM Corporation, Entry Systems Division, Austin, Texas, 1984 Mack89, user-interface on integrated tablet/display [Kasanuki85] Kasanuki et al "Sophisticated Man-Machine Systems (2)", 30th Conference of the Data Processing Society of Japan, pp 1831-1832, Japan Cited in Sklarew patents [KasdayLR84a] Kasday, Leonard, R. "Touch Position Sensitive Surface", United States Patent 4,484,179, November 20, 1984 Membrane-based touchscreen digitizer, top film presses on lower glass causing total internal reflection (compare with FTIR?) detected optically. Can sense multi-touch multiple touches, and sense force/pressure (not clear how). Light detectors are at edges of screen, can be transparent. One use is re--labelable keys (compare with Ken Knowlton, Kaplow) Describes two-finger gestures for rotation. States that device can detect more than one touch at a time by using timing information, and measure force by greater contact area causing stronger signal. [Kataoka84] Kataoka, Hiroyuki and Arai, Yoshio "Character Shaping Device", United States Patent 4,440,513 assigned to Fuji Xerox Co., Ltd. Tokyo Japan, April 3, 1984 Handwriting text editing: Normalizes size, scale, position, shape of handwritten characters prior to printing to be neatly spaced and lined up better, but does not talk about recognition per se [KernighanBW84a] Kernighan, Brian W. and Pike, Rob "The UNIX Programming Environment", Bell Telephone Laboratories, 1984 Original Unix system documentation from Bell Labs: basic shell, troff, sed, system calls for file system, processes, signals, interrupts. Explanation of standard I/O (stdout, stdin, etc.), UNIX-style file permissions (owner, group, world). [KimJ84] Kim, J. and Tappert, C.C. "Handwriting Recognition Accuracy versus Tablet Resolution and Sampling Rate", IEEE 7th International Conference on Pattern Recognition, 1984, Montreal Canada, pp 917-918 A "nothing" paper: recognition goes down with worse tablet resolution [Konneker84] Konneker, L.K. "A Graphic Interaction Technique Which Uses Gestures", IEEE Proceedings of First International Conference on Office Automation, page 51-55, 1984 Cited in Rhyne86, Buxton87 Rhyne86 cites this as an interesting handwriting input system: Also Goodisman91 Comments that "'C' code is not necessarily portable" [KrasnerG84] Krasner, Glenn, editor "Smalltalk-80: Bits of History, Words of Advice", Addison-Wesley, Reading MA, 1984 Background of Smalltalk language and environment. (Electronic file.)



[Krieger85] Krieger, Mark "Writing Portable UNIX Software", talk given at the 1985 UNIFORUM conference in Dallas, Texas by Mark Krieger, January, 1985. Speaker's address: UniPress Software Incorporated, Suite 312, 2025 Lincoln Highway, Edison NJ 08817 [KruegerMW85a] Krueger, Myron W.; Gionfriddo, Thomas; and Hinrichsen, Katrin "VIDEOPLACE - An Artitifical Reality", Proc. CHI '85, April 1985, pp. 35-40 Interactive environment sensing outlinse of user images in video sensing, video display. Shows two-hand (multi-touch) gestures for streatch, resize, move of interactive objects. Compare with interactive desk. Artificial reality / responsive environment. [KuklinskiT84a] Kuklinski, T. "Components of Hand-Print Style Variability", Proceedings of the IEEE 7th International Conference on Pattern Recognition, pp 924-926, 1984 Handwriting variability classifiably affected by schooling, country of origin (education), age (small children, elderly), profession (architects print neatly), haste/type of note taking, writing instrument and surface, etc. Early form of generative model for handwriting/handprinting variability: See Ward/Kulklinski [KuklinskiT85a] Kuklinski, T. "A Case for Digitizer Tablets", Computer Graphics World, May, 1985, pp 45-52 Overview of digitizer tablets: electromagnetic vs. resistive vs. electrostatic, pressure/force measurement. GKS standard supports locator (point), stroke (path), valuator, selector, string (handwriting recognition). Tablet vs. mouse: tablet does not require screen feedback, can do absolute *or* relative positioning/movement. Tablet templates / soft function areas for caommand input. Future screen inking /electronic ink, portable tablet clipboards [KuklinskiT85b] Kuklinski, Theodore) "Internal speaking notes, Development of Pencept Digitizing Tablet", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1985 History of digitizer performance and technology problems that lead to development of Pencept electromagnetic digitizing tablet with two-coil sensing stylus to correct for stylus tilt. Credits to Dr. Barry Blesser. Contains discussion of performance problems of digitizing tablets in general. [Kurta85a] Kurta Corporation "Kurta IS/PenMouse digitizing tablet, ca. 1985", Kurta Corporation (physical device) PenMouse is a digitizing tablet with a mouse-emulation mode. (Stylus missing). [Kushnir85] Kushnir, M., Abe, K., and Matsumoto, K. "Recognition of Handprinted Hebrew Characters using Features Selected in the Hough Transform Space", Pattern Recognition, Vol 18 No 2, 1985, pp 103-114 Features are stroke direction, concavities, endpoints, intersections, etc. Says high dimensionality of feature space means template matching takes up lots of memory and compute time Handwriting recognition requires new strategies beyond printed recognition Accuracy test uses larger training set than test set Statistics fail by grouping similar characters, which must be solved by heuristics (!) Issues of test samples versus training samples Source of errors is similar characters: boundary? [Kuzunki85] Kuzunki, S. et al "Handwritten Characters and Graphics Recognition Technology for Workstations", Hitachi Hyoron, Vol 67 No 3, pp 243-246, 1985 Cited in Sklarew patents [Lamarche84] Lamarche, F. and Plamondon, R. "Segmentation and Feature Extraction of Handwritten Signature Patterns", Proceedings of the 7th International Conference on Pattern Recognition, 1984, pp 756-759 Refers to time-axis distortion (variability) in signatures [Lamb84] Lamb, M.R. and Buckley, V. "New techniques for gesture-based dialogue", Proceedings of INTERACT '84, IFIP Working Group 6.3, London, United Kingdom, September 4-7, 1984, pp 135-138 Need whole conference proceedings: user interface with handwriting editing. [LamportL84a] Lamport, Leslie "Recursive Compiling and Programming Environments: Summary", Self-published manuscript July 1984 https://research.microsoft.com/en-us/um/people/lamport/pubs/pubs.html#recursive-compiling Selfmodifying compile construct added to Pascal: can compile and then execute source code, a la Lisp: build environments / development environment. Mentions that the original version of EMACS was written in TECO, not in Lisp -- original Teco editor from Digital Equipment Corporation? [LeeSK84a] Lee, Seonkyoo "A Fast Multiple-Touch-sensitive Input Device", Master's Thesis, Dept. of EE, U. of Toronto, October 1984: 115 pages (partial copy) Capacitive digitizer touchscreen: multi-touch sensor resolution is 64 by 32, increased by interpolation. [LeeSK85] Lee, S.K., Buxton, W. and Smith, K.C. "A Multi-Touch Three Dimensional Touch-sensitive Tablet", CHI'85 Proceedings, pp 21-ff, ACM 0-89791-149-0/85/004/0021 Capacitive digitizer technology, used for sensing multiple touch points of user's finger on surface: uses recursive division to determine position: used to emulate a virtual piano keyboard. Points to problems of delay between gesture and response to gesture. Also: Electronic video file of demonstration at CHI '85. [Leedham84] Leedham, C.G., Downton, A.C., Brooks, C.P. and Newell, A.F. "On-line acquisition of Pitman's handwritten shorthand as a means of rapid data entry", Proceedings of INTERACT '84, IFIP Working Group 6.3, London, United Kingdom, September 4-7, 1984, pp 145-150 Need whole conference proceedings: user interface with handwriting editing.



[LelerW85a] Leler, Wm and Soiffer, Neil "An Interactive Graphical Interface for Reduce", Proc. AAECC-3 Int'l. Conf. on Applied Algebra, Algebraic Algorithms, Symbolic Computation, and Error Correcting Codes, Grenoble France, July 15..18, 1985 (extended version of paper) Editing of 2D mathematical expressions using mouse or tablet (symbolic algebra program e.g. Macsyma). Circle gesture subject to problems of selecting (stray) parts of different sub-expressions. [LevyHM84a] Levy, Henry M. "Capability-Based Computer Systems", Digital Press (Digital Equipment Corporation), 1984 General reference/textbook on capability-based computer systems: single address format for all objects: hardware/software, primary/secondary memory. Capability address is a token specifying both ID for object, and access rights. Burroughs B5000/B5500 as early system, supervisor/kernel systems, support for capabilities in object-based design. Multics segmentation. [LewisPH85] Lerner "Peripherals: Astronomy Software", New York Times, May 21, 1985 Mentions PenPad 320 handwriting terminal with handwriting recognition: uses Motorola 68000 microchip [Linus84a] Linus Technologies "Business Plan for TechBook Limited Partnership", 5265 Port Royal Road, PO Box 1322, Springfield, Virginia, 22151, February 1984 Original business plan for Linus portable handwriting product: also describes gesture-based user interface [Loken-Dim85] Loken-Kim, K.H. "Artificial Intelligence Techniques and their Application to the Correction of Automatic Speech Recognition Input", Proceedings of Voice I/O Systems Applications Conference, September 1012, 1985, San Francisco, pp 425-441 Speech: context correction requires use of linguistic and extra-linguistic knowledge Speech: high recognition rate is not enough: user interface is important Speech: substitution (modal) recognition errors more severe than rejection in performance perception Speech: sequential restrictions in language are called syntax [Lukis85] Lukis, L.J. and Duhig, G.P. "Character Recognition Device", United States Patent 4,493,104, January 8, 1985, assigned to Moore Business Forms, Incorporated, Grand Island, New York. Pressure-sensitive resistive-sheet digitizer. Moore Business Forms DCR patent. [Lyon85] Lyon, Richard F. "Cursor Control Device", United States Patent 4,521,772 June 4, 1985, assigned to Xerox Corporation, Stamford, Connecticut Optical mouse: optical digitizer using passive printed pattern: but recognizes relative motion only, not absolute position. (See other papers in file for additional paper and Web references) [Lyon85a] Lyon, Richard F. "Imaging Array", United States Patent 4,521,773 June 4, 1985, assigned to Xerox Corporation, Stamford, Connecticut Optical mouse sensor, with auto-correlation in analog (?) circuitry. (See other papers in file for additional paper and Web references) [MabusthS85a] Mabusth, Scott "Touch Sensitive Control Device", United States Patent 4,550,221, October 29, 1985 Capacitive grid touchpad: each row electrode and each column electrode measures for capacitance to ground, positions is (interpolated) X for the columns and (interpolated) Y for the row (not capacitance at a crossing point). Row traces and column traces each have extra length where they do not cross over: a capacitance change on a column and not on any row indicates user finger is over/on this extra part, used for additional function buttons. [Mandler85] Mandler, Egerhard, Oed, Richard, and Doster, Wolfgang "Experiments in On-Line Script Recognition", Proceedings of 4th Scandinavian Conference on Image Analysis, Norwegian Institute of Technology, Trondheim, Norway, June 17-20, 1985 Gives many European writing variation styles for "A": features are pathlength and arc; segmentation into single characters without context, not limited to alphanumeric characters, [McCombG85a] McComb, Gordon "Move over keyboard: now it's pens, pads, mice ... and more", Popular Science, May 1985, pp. 92-ff. mentions touchscreens (HP-150 computer terminal using infrared light sensors for finger: low resolution; MicroTouch, but does not work with finger), voice recognition, handwriting recognition/pen-computer from Pencept, 1985. Chalkboard Powerpad -- multitouch? [Meads85a] Meads, Jon A. "Report on the SIGCHI Workshop on Planning for User Interface Standards", Jon Meads and Associates, 2516 NE 19, Portland, OR 87212, 1985 Discussion of possible UI standards/guidelines. Background includes discussion of multiple device independent layers in conjunction with UIMS: consider virtual devices. [Meads85b] Meads, Jon A. "Friendly or Frivolous?", Datamation, April 1, 1985, pp 96-100 User-interface user-friendly: users do NOT plan ahead: Fitt's Law says a mouse may be user-friendly, but not a mouse and a keyboard [Meisel85] Meisel, W.S. "Empirical Artificial Intelligence in Speech Recognition", Proceedings of the 1st Annual Artificial Intelligence and Advanced Computer Technology Conference, Long Beach, California, April 30 - May 2,



1985, pp 182-187 Empirical artificial intelligence: cross between statistical and knowledge-based expert systems analysis Qualitative vs quantitative data/information AI research community has ignored empirical knowledge in expert systems AI: extracting intelligence from empirical evidence vs from human experts AI expert systems: possible flawed assumption is that humans can accurately tell you what they know Curse of dimensionality: your data may just not happen to show a particular feature or interaction among features Sample size vs number of features can be treated formally EAI: using expert knowledge to uncover structures, and using data to verify structures (is not this just normal scientific method?) Reducing "intrinsic dimensionality" of data and features Need off-line (expert?) development of recognition code/tables/features, because too complex to be done in real-time or on-line Human opinion/context has a large role in recognizing/labeling a speech sound Collect speech data for testing from real-world business correspondence and memos linguistic context rules applied to speech recognition [MinskyMR84a] Minsky, Margaret R. "Manipulating Simulated Objects with Real-world Gestures using a Force and Position Sensitive Screen", Computer Graphics Vol 18 No 3, July 1984, pp 195-203 Touch/pressure-sensitive digitizer tablet: uses four strain gauges / load cells at corners supporting a tablet to sense both position, and strength and direction of force (lateral force), and potential for vision (eye gaze?) gestures. Gestures include tapping, not just shape recognition: note that Penpad also had a "click" gesture of sorts for mouse/character-recognition mode. Includes flicking gesture, using side-force as well as motion. Refers to screen/stylus (finger) parallax problem. Cites to Put-That-There project. Objects can be pushed through virtual "rooms" via doorways at sides of displayed image, display changes to new room. Filter wild data points from touchscreen. Mentions multiple finger touch / multi-touch touchscreen version of hardware. Describes touch screen system using buttons that recognize a "selection" (tap/"hitting") gesture and a "move" (tap and hold/press-and-hold) gesture; the buttons control movement of turtle on the display; the move gesture allows the user to arrange the buttons on the display. I.e. gesture buttons. Has additional gestures, in that one button can be programmed (in various ways, by example) to click on other buttons in a particular order, e.g. to make a "paint something" button. [Mizzi85] Mizzi, Pierre M.A. "Hand-Held Computer", United States Patent 4,545,023, assigned to Engineering Project Development Limited, London, England, October 1, 1985 Hand-held "slate" style computer with no keyboard, provision of phoneme and/or handwriting recognition, and for TV/Radio reception via plug-in cards [MoriS84] Mori, S., Yamamoto, K., and Yasuda, M. "Research on Machine Recognition of Hand-printed Characters", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-6 No 4, July 1984, pp 386-405 It's important to know how well DCR system works on low-quality data Recognition of hiragana (small character set) about as difficult as Kanji (large character set) General review of Kanji character recognition Gives figures for total Kanji characters in use in Japan, and in China Mentions earliest commercial Kanji recognition products, ten current Cites a very large data-base collection, which "scope is not sufficient" Cites Blesser Mentions fundamental differences of human vs machine recognition (p 403) Variations unlimited: must be limited for practicality and academic work [MoriS85] Mori, Shunji; Yamada, Hiromitsu; Takeuchi, Kunihiko,; and Do, Michihisa "Feature Extraction System for Digitized Character Information", United States Patent 4,468,808, August 28, 1984 Feature extraction for optical character recognition: compare to chain codes, sub-segments with concavity or convexity, loops: states that convexity/concavity noise is eliminated automatically by processing [Morita85] Morita, M. "Japanese Text Input System", IEEE Computer, Vol 18 No 5, 1985, pp 29-35 Cited in Kankaanpaa87 special keyboard layout designs for keyboard input for Kanji/Chinese [Murase85] Murase, H., Wakahara, T., and Umeda, M. "Online Writing-Box Free Character String Recognition by Candidate Character Lattice Method", Trans. of the Institute of Electronics Communications Engineers of Japan, Vol J68-D, pp 765-772, April 1985. (in Japanese). Segmenting strokes into characters using varying spacing: box-less input [MurphyAS84] Murphy, A.S. and Wiseman, J.A. "Optical Digitizing Stylus", IBM Technical Disclosure Bulletin, Vol. 27 No. 5 October 1984, Pages 2806-2807 Cited in BennettWE91 Optical stylus digitizer using table with X and Y lines in different colors: appears to work like relative-position optical mouse, not absolute position [Murtagh84] Murtagh, F. and Raftery, A.E. "Fitting Straight Lines to Point Patterns", Pattern Recognition, Vol 17 NO 5, May 1984, pp 479-483 (abstract only) Feature: finding straight lines in OCR patterns (skeletons?) [MyersBA84a] Myers, Brad A. "The User Interface for Sapphire", IEEE CGA, December 1984, pp. 13-23 Windows management in user interfaces: review of tiled windows, partitioned, overlapping/covered. Active window (accepting input) need not be on top. Pointing devices all treated as relative, not absolute: mouse, electromagnetic tablet, four-button puck on digitizer. Cites to PERQ, Xerox Star, MacIntosh/Lisa. Tracking symbol picture (cursor) shows operation that will be performed on mouse/stylus-up.



[NEC84] NEC: "CR-100 product literature", (in Japanese), 1984 Nippon Electric Company 1984 Japanese/Kanji/Chinese on-line handwriting product [Nagayama85] Nagayama, T., Shibuya, J., and Kawakita, T. "Pen-Touch-Type Electro-Magnetic Transparent Touch Panel", Society for Information Display 1985 Digest of Technical Papers, May 1985,pp 32-35 Transparent digitizer table over LCD with DCR [Nagy84] Nagy, G. and Seth, S. "Hierarchical Image Representation with Application to Optically Scanned Documents", Proceedings of the 7th International Conference on Pattern Recognition, Montreal, 1984 Tiling/windows applied for format of written/printed documents [Nagy85] Nagy, G. "Image Database", Image and Vision Computing, Vol 3 No 3, August 1985, pp 111-117 User-interface aspects of image data bases not given much attention [Naito84] Naito, S. and Masuda, T. "Chinese Character Recognition Based on Personal Handwriting Characteristics", Trans. of IECE of Japan, Vol J67D No 4, April 1984, pp 480-487 (Abstract only). Science Citation Index Difference in handwriting variations/variability between writers 50% greater than for one writer (?) Claims 96.9% successful recognition when trained to writer, 67.7% for general use without training, for 3256 Kanji categories. [Nandhakumar85] Nandhakumar, N. and Aggarwal, J.K. "The Artificial Intelligence Approach to Pattern Recognition -- A Perspective and an Overview", Pattern Recognition, Vol 18 No 6, pp 383-389, 1985 Overview of DCR in the context of artificial intelligence [NationalOA84] National OA: "Panaword RL-W450 / FW-500 product literature", (in Japanese), 1984 National OA 1984 Kanji/Chinese product [Nestor85a] Nestor Inc. "Nestor Inc. Annual Report to Shareholders 1985", Nestor Inc., 122 East 42nd Street, NY NY 01068, 1985 Nestor handwriting recognition company: audit says $500K royalties could not be verified [Nouh84] Nouh, Adnan, Sultan, Abobakr, and Tolba, Roshdi "On Feature Extraction and Selection for Arabic Character Recognition", Arab Gulf Journal of Scientific Research, Vol 2 No 1, 1984, pp 329-347 Density features for Arabic handwriting recognition: position-by-position matching between characters: Freeman codes as divergence criteria Tilt: 0.01" to 20 degrees, 0.08" between 20 and 40 degrees [Numonics85] Numonics "Numonics #2200 Pad with Low Profile and High Resolution", Numonics Incorporated, 418 Pierce Street, Lansdale PA 19446, 1985 large-format (20" x 20") electromagnetic tablet with stylus or puck. [OKI84] OKI Corporation "SR 220 / SR 230 Online product information", OKI Corporation, 1984 Japanese handwriting character recognition product [OedR84] Oed, R. and Doster, W. "The Intelligent Graphics Tablet - More Than a Mouse", Proceedings of Advances in Microprocessor Peripherals, Journees d'Electronique et de Microtechnique, Lausanne, Switzerland, October 8-11, 1984, pp 183-190 Shows handwriting user-interface with control/editing area much like Penpad or C.I.C. Handwriter Shows diagram for stroke/character segmentation/parsing using overlapping rectangular extents User interface for handwriting into existing applications using keyboard buffer [OedR85] Oed, R. and Doster, W. "On-line Script Recognition - A User-friendly Man-Machine Interface", Proceedings of the COMPINT 85 Conference on Computer-Aided Technologies, IEEE Catalog No 85CH2136-0, pp 741-743 Shows diagram for stroke/character segmentation/parsing using surrounding rectangles. Separate writing areas on digitizer tablet for text input, gestures, virtual tablet/mouse input: compare with virtual tablets and recognition macros in Pencept PenPad. Makes statement that user-definable trainable/adaptable gesture-symbol recognition simplifies user interface, but this is questionable: semantics of symbol must still be defined, and mnemonicity and non-ambiguity (to recognizer) of a gesture is an open question. [OlsenDR84] Olsen, Dan. R. Jr., Buxton, William, Ehrich, Roger, Kasik, David J., Rhyne, James R., and Siberg, John "A Context for User Interface Management", IEEE Computer Graphics and Applications, December 1984, pp 33..41 Early article on features needed in UIMS User interface management systems: uniformity across devices and across application. [Omron84] Omron Corporation "Handwriting product information", Omron Corporation, 1984 Japanese handwriting character recognition product [Panati84] Panati, Charles "Panati's Browser's Book Of Beginnings", Houghton Mifflin, Publishers, 1984. ISBN 0395-56238-4 (hardcopy book) Chapter 6: "Scribble to Shorthand". Invention of Pens and Papyrus, historical reference to Stylus. Tieronian unistroke alphabet in the history of shorthand, 63 B.C., pages 81..82. Pitman Shorthand 1937, Taylor shorthand, Gregg Shorthand. Braille. [Parks85] "Apparatus for Signature Verification", Parks, John R; Carr, David R; Fox, Peter F. United States Patent 4,495,644, January 22, 1985, assigned to Quest Automation Public Limited Company, Great Britain.



Quest Automation Handwriting product: Micropad/Datapad using resistive film, pressure-sensitive digitizer tablet with hand rest Signature verification considering multiple scores of deviations, signature rejected either by a high combined score, or a very high score on one parameter [Parnell84a] Parnell "Digitizing System", United States Patent 4,473,717 Coarse search on grid tablet (electromagnetic/electrostatic) to find strongest conductor line and two neighbors, then fine determination by three-phase interpolation method on those three lines. Compare with Blesser interpolation. [Pavlidis85] Pavlidis, J., and Van Wyk, C.J. "An Automatic Beautifier for Drawings and Illustrations", A.C.M. Computer Graphics, Vol 19 No 3, July 1985, pp 225-234 Gesture/graphics input heuristics: cite for Pavlidis. Relevance to sketch recognition? [Pencept84a] Pencept "Penpack (TM) Multiplan (TM) User's Guide", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 Another word processor for the Penpad from Pencept [Pencept84b] Pencept "Penpack (TM) IBM (R) Personal Editor User's Guide", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 Forms data entry for the Penpad from Pencept [Pencept84c] Pencept "Penware (TM) Penform (TM) User's Guide", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 Pencept character set: forms data entry software for handwriting recognition using digitizer. [Pencept84d] Pencept "User Manual for Penpad 320", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 User manual for Penpad 320 handwriting-recognition tablet for MS-DOS. [Pencept84e] Pencept "Software Tool Kit for the Penpad 320", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 (hardcopy book) Handwriting recognition product of MS-DOS computers: includes gestures ("recognition macros"), definable virtual tablets in digitizer firmware. [Pencept84f] Pencept "Penpad (R) 320 Technical Data", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1984 PenCad, gesture / recognition macros for AutoCad and other MS-DOS CAD applications. [Pencept84g] Pencept "IBM PC peripheral accepts hand-printed or drawn input", IEEE CG&A, May 1984, "New Products", page 94 Product announcement for Pencept PenPad 320, hand-writing-recognition terminal for IBM-PCs. Essentially a keyboard replacement with handwriting recognition, combined with a general purpose digitizing tablet. "Application Packs" for recognition macros / gestures using simulated keyboard/mouse input for unmodified Multiplan, VisiCalc, Word Start, Lotus 1-2-3 spreadsheet. Included a simulated "mouse" mode, using either mouse input through the driver, or sending keyboard arrow-key input. [Pencept84h] Pencept "Penware Software for the PenPad / PenPad 320 and Application Software", Pencept Inc., 1984 Handwriting recognition tablet: Brochure on Pencept Software Took Kit with Command Template Editor: Pendraw drawing program, PenCad interface to AutoCAD (Zoom/Magnify gesture/recognition macro), etc., Penform business forms data entry. [Pencept84i] Pencept "Typical Character Shapes which are Recognized by the Pencept System", Pencept Inc., 1984 Handwriting recognition tablet: examples of recognizable distorted variability styles. Compare with CIC. [Pencept85a] Pencept "PenCad (TM) Penpad (R) Interface for Freelance (R)", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1985 Pencept gesture recognition additions / Recognition macros for Freelance graphical drawing program [Pencept85b] Pencept "PenCad Penpad Interface for CAD and Graphics, Version 3.4 (R)", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1985 pen-computing CAD interface using recognition macros / gestures, virtual tablets, stylus instead of a puck [Pencept85c] PenCept, Incorporated "Software Control at the Stroke of a Pen", SIGGRAPH Video Review, ACM Press: CHI '86 Videotape of live demonstration during CHI '85 conference showing gesture recognition / recognition macros, handwriting recognition, interfaces to AutoCad CAD and other applications. [Pencept85d] Pencept "Pencept Marketing Materials", Pencept, Incorporated, 39 Green Street, Waltham, Massachusetts 02154, 1985 Marketing and technical materials from PenCept, Inc. showing handwriting recognition and gesture recognition [Pencept85f] PenCept, Incorporated "Software Control at the Stroke of a Pen", SIGGRAPH Video Review, ACM Press: CHI '86. Available (2009) at http://www.open-video.org/details.php?videoid=8023



Videotape of live demonstration during CHI '85 conference showing gesture recognition / recognition macros, handwriting recognition, interfaces to AutoCad CAD and other applications. [PennycookBW85a] Pennycook, Bruce. W. "Computer-Music Interfaces: a Survey", Computing Surveys, Vol 17, No 2, June 1985, pp 267-ff Mentions Groove and NRC music systems using light-pen, three-D wand digitizer musical score-editing systems mentions in bibliography: some use pen/tablet input [PepperW84a] Pepper, William Jr. "Hand-held musical instrument and systems including a man-machine interface apparatus", United States Patent 4,430,917, February 14, 1984 One-dimensional touch panel element: one dimensional variation on touch pad or digitizer for a musical instrument [PetreP85a] Petre, Peter "Speak, Master: Typewriters That Take Dictation", Fortune magazine, January 7, 1985, pp. 74-78 Review of Speech-to-text speech recognition product and research systems (talkwriters) to replace keyboard terminals: IBM Frederick Jelinek large statistical database, Raymond Kurzweil, mentions dictionary compilation and dictionary size for template-based systems, NEC Yasuo Kato, short words hard to differentiate by sound alone, Victor Zue trained himself to read spectrum diagrams of spoken words for feature-based recognition [Photron85] Photron Limited "OSCON Digitizer technical information: SQ-3100 and SQ-4000MKII Specifications and Technical Guide", Photron Limited, 1985 Electromagnetic tablet, transmitter coils in stylus or cursor (puck). [PingryJ84a] Pingry, Julie "Inputting Graphics By Digitizer", Digital Design, april 1984, pp. 102-108 Overview of digitizer tablets, paper scanners, and video digitizers for simple graphics input. Refers to mouse-style cursors (puck). GTCO stylus pressure-sensitive, measures tilt, used for graphics drawing changes, joystick. Electromechanical 3D digitizers. On-tablet menus. Mentions Pencept dynamic (on-line) character recognition as keyboard alternative. [Plamondon85] Plamondon, R. and Lamarche, F. "Modelization of Handwriting: A System Approach", in "Graphonomics: Contemporary Research in Handwriting", Proceedings of the 1986 Graphonomics Conference, Hong Kong, July 8-12, 1985, Henry S.R. Kao, Gerard P. van Galen and Rumjahn Hoosain, editors, North-Holland, publishers, 1986 Analog filter on digitizer output gives "improved accuracy" (!!??!!). Tablet digitizer jitter noise is +- 0.002 inch (?) [Polhemus84] Polhemus "3-Space Digitizer Technical Bulletin", 1984, Polhemus Navigation Sciences Division of McDonnel-Douglass Electronics Co., Colchester, Vermont. [Poole84] Poole, Lon "A Tour of the Mac Desktop", MacWorld, Vol 1 No 1, 1984, pp 16-21 Cited in JohnsonJ89 MacIntosh desktop: icons, pointers, mouse, menus (Apple/system menu), windows. Windows zoom out from icons. Dialog and alert boxes. Drag gesture. [PortableComputer85] Portable Computer Products Company "Data Pad product announcement", April 1985. 10801 West Main Street, Suite 600, Fairfax VA 22030, 703-352-8116 (see also Annographics). Mark Spikell, founder. [PorterT84a] Porter, Thomas and Duff, Tom "Compositing Digital Images", ACM Computer Graphics, vol 18 No 3, July 1974 Early tutorial paper on alpha channel mixing factor for putting graphical object or image over a background. Alpha channel value indicates transparency, should be similar resolution to RGB color values. Suggests matte channels. Operators (BitBLT counterparts?) include clear, src, dst, src-over, dst-over, srcin, dst-in, src-out, dst-out, src-atop, dst-atop, xor. [Prentice85a] Prentice, Thomas C. "Methods for testing digitizer performance", Internal Report, Pencept, Incorporated, 1985 Tablet digitizer performance [Prentice85b] Prentice, Thomas C. "Understanding Digitizer Specifications", Internal Report, Pencept, Incorporated, 1985 Tablet digitizer performance [QuayleSF85a] Quayle, Stanley F. and Schlosser, Philip A. "Electrographic System", United States Patent 4,523,654, June 18, 1985 Electrostatic tablet digitizer using ratiometric measurements of induced AC voltage in a resistive sheet, active stylus/puck. Transparent conductor/semi-conductor of indium tin oxide (ITO) or other metal oxide. In one figure: Sheet is shown between two glass layers. Assigned to Scriptel. [Quest84a] Quest Automation "Micropad: The Point-of-Sale System Using Handwritten Data Entry", Quest Automatction / Micropad Inc. quest Micropad, La Grane Il. 1984 pressure-sensitive (air-gap resistive sheets touchpad) handprint recognition terminal, 40-character dispaly. Applications shown are pre-printed point of sale forms /receipts, and pre-printed exploded view of parts



assembly. "Micropad gets write to the point." Also signature capture, signature verification (!). [RaederG85a] Raeder, Georg "A Survey of Current Graphical Programming Techniques", IEEE Computer, August 1985, pp 11-25 Overview of graphical programming languages, pre-LabView. Nassi-Shneiderman diagrams for overview of program control; data-flow graphs vs. control-flow graphs. Also discusses graphical editors: menus making syntax/semantics more discoverable, etc. [RebmanJ85a] Rebman, Jack "Tactile Sensor for an Industrial Robot or the Like", United States Patent 4,521,685, Mune 4, 1985 Tactile X/Y/Z touch sensor using Hall effect: In addition to force/pressure, can also detect shear/transverse forces. Compare with multitouch arrays. [Richards85] Richards, Joh, T, Boies, Stephen J., Gould, John D. "Rapid Prototyping and System Development: Examination of an Interface Toolkit for Voice and Telephony Applications", IBM Research Division Report RC 11433 (#51395), 10/8/85, 9 pages. Also Proc. CHI'86, April 1986, pp. 216-220 rapid-prototyping application generator for voice recognition user interfaces: uses tables of utterances and action to take. [RoadsC85] Roads, Curtis "Research in Music and Artificial Intelligence", Computing Surveys, Vol 17, No 2, June 1985, pp 163-ff Overview of artificial intelligence applied to music composition/understanding. Mentions several graphical/pointing device music-editing projects e.g. Buxton (with gestures), especially concerning internal representation of music, user interface support for differeing composition strategies (by user) (composer's assistant). Cites to Countess Lovelace for computer composition of music. [RobinsonD84A] Robinson, Dan "SIDEKICK: Whoever wrote this gem understands Murphy's law", InfoWord, 26. Nov 1984, p 61 Sidekick TSR utility for DOS, with Screen dialer from screen of PC: select telephone number in text on screen, it dials [RoseC85a] Rose, Caroline (ed.) "Inside MacIntosh Vol I-II-III", Addison Wesley, 1985 Book version of Inside Macintosh, guide to using and programming MacIntosh. QuickDraw: bitblt routine copyBits, includes SrcOr, SrcXor, and notSrcOr modes. Masking is by regions equivalent to bitmap masks: region is any number of closed polygons in drawing coordinates (need not be same as bitmap coordinates). (copyMask: see 1994 edition) [RoseC85b] Rose, Caroline (ed.) "Inside MacIntosh Vol I-II-III, Promotional Edition", Apple Computer, March 31, 1985 "Telephone book" (informal) version of Inside Macintosh, guide to using and programming MacIntosh. QuickDraw: bitblt routine copyBits, includes SrcOr, SrcXor, and notSrcOr modes. Masking is by regions equivalent to bitmap masks: region is any number of closed polygons in drawing coordinates (need not be same as bitmap coordinates). (copyMask: see 1994 edition) [Rosson84] Rosson, M.B. "Characterizing Freeform Editing Behavior", IBM Research Report RC 10550 (Log #47200) 5/23/84, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598 Touch/gesture interfaces and direct manipulation: Boies said 75% of interactive user time is commands, not input: Rosson says 25%. Common mistake in keyboard-based editing mis-typing command names. [Rouse85] Rouse, Nancy E. "Manual Digitizers for CAD", Machine Design, June 20, 1985 lists all major digitizer tablet companies of the day, and their technologies (sonic, magnetic, magnetostrictive, etc.) 2-D: Altek, CalComp, Elographics, CTCO, Hitachi, Houston Instrument, Kurta, Numonics, Science Accessories Corp (SAC), Scriptel, Summagraphics; 3-D: Micro Control Systems, Polhemus Navigation Sciences, Science Accessories Corp [RubinRV85a] Rubin, R.V., Golin, E.J., and Reiss, S.P. "ThinkPad: a Graphical System for Programming by Demonstration", IEEE Software, Vol 2 No 2, pp 73-79 Interactive graphical interface language for programming: defining nested/heirarchicahl data structures by creating boxes inside boxes in graphical user interface. [SAC84] SAC "GP-8-3D Grafbar Technical Bulletin", Science Accessories Corporation, 1984, Southport, Connecticut. Acoustic 2-D/3-D digitizer tablet, large active area, refers to "standard digitizer output formats" [SAC85] SAC "GP-10 Two-dimensional Sonic Digitizer", Science Accessories Corporation, 1985(?), Southport, Connecticut. Acoustic 2-D digitizer tablet, large active area, refers to "standard digitizer output formats": Emulation of vendors Kura, Numonics, Summagraphics, Wacom, Hitachi, Houston Instruments, GTCO, Calcomp and SAC GP-7, GP-8, CP-9, GP-9XL. Formats and behavior customizable via the controller, built-in menu in controller [Sakai84] Sakai, T., Odaka, K., and Toida, T. "Several Approaches to Development of On-line Handwritten Character Input Equipment", Proceedings of the 7th International Conference on Pattern Recognition, 1984, pp



1052-1054 Boxed input: command-mode characters straddle boxes New text-editing method for keyboard emulation: uses multiple lines "category 1" characters: complete ambiguous due to sampling resolution [Sakoe84] Sakoe, Hiroaki "Pattern Matching Device Operable with Signals of a Compressed Dynamic Range", United States Patent 4,479,236, October 23, 1984, assigned to Nippon Electric Company, Tokyo, Japan. Iterative approximation to distance function between two patterns [Satoh85] Satoh, K., Ooka, A., and Wada, Y. "Character Recognition Apparatus", United States Patent 4,542,526, September 17, 1985, assigned to Sumitomo Electric Industries, Osaka, Japan. Handwriting character recognition using length and slope of line segments, classification to standard strokes Uses 8-direction chain codes for directions of entire stroke segments. Uses distance between strokes to segment strokes into characters [Schaeken84] Schaeken, B., Verschueren, W., Rene de Cotret, Y. and Hermanne, A. "A Hierarchical System for Handwritten Numeral Recognition", Proceedings of the 7th International Conference on Pattern Recognition, 1984, Montreal Canada, pp 1004-1007 Says recognition success % available, but does not give it (!) Handwriting recognition product [SchlosserPA84a] Schlosser, Philip A. and Quayle, Stanley F. "Electrographic System and Method", United States Patent 4,456,787, June 26, 1984 Resistive sheet electrostatic/capacitive digitizer with proximity sensing: Scriptel. See notes from Wang project. [Schuermann84a] Schürmann, Jürgen and Doster, Wolfgang "Umschreibende Rechtecke separieren Zeichen", Computerwoche Nr 46, 9 Nov 1984 Article on the various technologies used to move to a "paperless" office: optical scanning to convert paper documents to electronic form, handwriting/digitizers to capture character writing and sketches without paper [ScriptWriter85] Data Entry Systems "ScriptWriter Technical Information", Data Entry Systems, 6767 Madison Pike, Suite 195, Huntsville, Alabama 35806, 1985 Portable handwriting recognition tablet, two-line 40 char each display. (equals 80 char single line). Tablet technology: point-pressure sensing (elographics?) [Scriptel84] Scriptel "New Products: CAD Graphics Tablet", IEEE Communications Magazine, Vol 22 No 4, April 1984, pp 50..51 Scriptel electrostatic tablet product advertisement: photograph shows transparent front-surface digitizer, suitable for digitizing drawing under the tablet, or frosted surface for rear-projection. Not subject to magnetic biasing (apparent reference to Summagraphics) [Scriptel85a] Scriptel "Transparent Glass digitizer", Scriptel Corporation, product literature Product information: Scriptel SPD-Series electrostatic tablet: transparent glass digitizer -- shown with puck, not with stylus, resistive decoding technology [Scriptel85b] Scriptel "Scriptel Transparent Glass digitizer", archive.computerhistory.org, fetched 2007 Scriptel glass/transparent digitizer tablet, shown with puck: X506.84 [SearbyAD85a] Searby, Anthony D. and Walker, Ian C. "Computerized graphics system and method using an electronically synthesized palette", United States Patent 4,524,421, June 18, 1985 User interface for mixing colors for drawing palette on stylus digitizer. Amount of color picked up from palette can be based on pressure/force (spring potentiometer in stylus), or dwell time. Dwell/press-andhold. [ShackelfordRL85a] Shackelford, Russel L. "Gesture-based Communication with Computers: Feasibility Study", National Science Foundation Report NSF/IST-85002, July 1985. 54 pages. multipoint (multi-touch) gesture recognition system using Chalkboard Inc. Hand/palm recognition. Gestures based on touches of fingertips: both up/down (contact) and in/out (from palm) for one-handed multitouch typing. "Power Pad" touchpad/tablet resolution of 100 points / square inch (10 points/inch). Contains analysis of performance and engineering problems with pressure-sensitive Power Pad. [Shakunaga84] Shakunaga, T, Kaneko, H., and Yodogawa, E. "Automatic Writer Recognition by Line-based Spectral Resolution of 2nd Order Statistics", Trans. of IECE of Japan, Vol J67D No 7, July 1984, pp 776-783 (in Japanese: English abstract only) Science Citation Index Like signature verification, but regular text handwriting: identification vs. verification, with 5.2% reject, 0.0034% false acceptance [Sharp84] Sharp Corporation "Sharp OA Handwriting product information (in Japanese)", Sharp Corporation, 1984 Japanese office/desktop handwriting character recognition product/system with CRT monitor, printer. [Shneiderman85] Shneiderman, B. "Human-Computer Interaction Research at the University of Maryland", unpublished manuscript, October, 1985. Author's address: Department of Computer Science, Human-Computer Interaction Laboratory, University of Maryland, College Park, Maryland 20742 Later published in SIGCHI Bulletin, January 1986, Vol 17 no 3, pp. 27-32. User-interface studies on menu



structure: globally visible menu helps; arrow keys for pointing vs mouse; manuals are better than on-line help (why?); user preference for touch-screen pointing tablet/display. Synchronized scrolling of two lists of dissimilar but (semantically) related data objects. Direct-manipulation touchscreen/tablet interface for DOS: compare with Momenta? [Shridhar84a] Shridhar, M. and Badreldin, A. "A tree classification algorithm for handwritten character recognition", Proceedings of the 7th International Joint Conference on Pattern Recognition, 1984, pp 615-618 Cited in Kerrick88 Kerrick88 cites as reference on sequential, heirarchical, and tree-based methods for character recognition [Shridhar84b] Shridhar, M. and Badreldin, A. "High accuracy character recognition algorithm using Fourier and topological descriptors", Pattern Recognition, Vol 17, 1984, pp 515-524 98% accuracy on handwriting recognition of numerics: Fourier on outlines confuses 2 and 5, 6 and 9, sometimes 4 and 7 due to rotational invariance, so topological analysis must follow. [Sklarew85] Sklarew, Ralph "Linus Technologies Inc. Historical notes", Historical notes at www.blinkenlights.com Write-Top introduced by Linus Technologies in 1987, mentions earlier work (ca 1984) with Robert Nadeau at George Mason University [SlikowskiJ84a] Slikowski, Joseph "Interactive Computer-Aded Design System", United States Patent 4,451,895, May 29, 1984 User interface for a CAD/CAM design system using two screens: one an unobstructed graphics display screen, the other a light-pen or touch-screen menu screen with dynamic menus. Because of targeting problems, operation is on lift of the stylus or lightpen, not on touch-down. [Swan85] Swan, K. "Pragmatics of Programming Voice Recognition Technologies", Proceedings of Voice I/O Systems Conference, San Francisco California, 1985 Cited in Mangione86 Adaptive recognition: "trying to retraining an already overtrained behavior, such as speech, is not recommended" Training/adaptive: users are unlikely to become expert users if they become discouraged naive users Speech: human factors gap: most system designers and users weak in human factors/user interface issues Speech: system must be readily interfacable to external host applications Speech: user do not want any "control knobs" on their user interface Speech: features for good interface design never all on one product Speech: user interface system designer cannot differentiate between user and machine error Speech: products ignore transducer (microphone) problems, even though critical Speech: writing custom software for applications kills the product Speech: software must be transportable between programmers, not computers Speech: buffering problems cause system to lose second word even though product works fine Speech: speech equivalent of "graphical doodling" [SweetRE85a] Sweet, Richard E. "The Mesa Programming Environment", ACM SIGPLAN Notices, Vol 20 No 7, July 1985, pp. 216-229 Evolution of Mesa/BCPL/Alto debugger and IDE environment into multiple window environment. Threebutton mouse, multi-click gestures to select more, right mouse button adjusts selection (middle button is pop-up context menu). Also mentions lasso gesture (drawing a box around to select) and underlining gesture to select. [SziklaiGC84a] Sziklai, G.C. "Methods and Apparatus for the Automatic Classification of Patterns", United States Patent 4,454,610, June 12, 1984, assigned to Transaction Sciences Corporation, New Orleans, Louisiana. How to find the signature and courtesy amount on a check using a grey-scale image Signature verification via OCR scanning [TappertCC84a] Tappert, C.C. "Dehooking Procedure for Handwriting on a Tablet", IBM Technical Disclosure Bulletin, Vol 27 No 5, October 1984, pp 2995-2998 Hook removal: hooks are a problem of the tablet [TappertCC84b] Tappert, C.C. "Delayed Stroke Processor for Handwriting Recognition", IBM Technical Disclosure Bulletin, Vol 26 No 12, May 1984, pp 6616-6619 Delayed strokes: aligns dots only with cusps, crosses on down side of cusp Refers to character cut points in cursive character segmentation [TappertCC84c] Tappert, C.C. "Adaptive On-line Handwriting Recognition", Proceedings of the 7th International Conference on Pattern Recognition, 1984, Montreal Canada, pp 1004-1007 Refers to context by "alphabet selection" Mentions U/V ambiguity for single writer (see also Jans75) Mentions A/H, h/n, a/u ambiguity as problem of dynamic matching algorithm (!) Ambiguity of 1,l,I, and upper/lower C,K,O,P,S,V,W,X,Y,Z Mentions "new variations" even for single user (!) Mentions sloppy and ambiguous writing as separate cases Mentions "alphabet selection" like our shift boxes Discrete vs continuous writing, but testing was different (biased) Mentions problems with his hook removal algorithm Claims 94% recognition success, 98% if restricted subset, with three familiar writers Says dictionary lookup would give 100% recognition (!) Mentions segmentation for case where text is known (but when is that real?) Testing found several omitted variants from a "good" system [TappertCC85a] Tappert, C.C. "An Adaptive System for Handwriting Recognition", IBM Research Report RC 11175



(Log #50249) 5/21/85, 1985, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598 (to appear in Acta Psychologica). Implies need for big data base: "vast variation, need larger quantities" Refers to intra-writer variability Claims at most four strokes in an English (Latin) letter Mentions characters (C,c, 1,I,J, etc.) only distinguishable by context Mentions need to accommodate new variations: "small sample of data" Baseline drift, more writing time means more prototypes [TappertCC85b] Tappert, C.C. et al "Handwriting Recognition on Transparent Tablet over Flat Display", IBM Research Report RC 11856 (Log #52695) 3/3/86, 1986, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598. (To appear in 1986 Society for Information Display International Symposium Digest of Technical Papers). Essential to success of character recognition is a natural human interface Character recognition has biggest performance demands on digitizer tablets Refers to commercially available handwriting recognizers, but not by name Mentions some performance problems of commercially available tablets Refers to minimum tablet tick resolution for character recognition Mentions many tablet performance problems, "natural" writing speed [Tarnopolsky84a] Tarnopolsky "Graphic Answer Input Device for a Teaching Machine", United States Patent 3,793,469 Lukis87 on digitizer tablets [TeitelmanW84a] Teitelman, Warren "The Cedar Programming Environment: A Midterm Report and Examination, Report CSL-83-11", Xerox PARC Palo Alto Research Center, Report CSL-83-11, June 1984 Cedar language, OS, and programming system: successor to Smalltalk. Combines several smaller papers. Dorado bitmapped display, on-line documentation. [TeitelmanW84b] Teitelman, Warren "A Tour Through Cedar", Proc. ICSE '84, 7th Intl. Conf. on Software Engineering, pp 181-195 Cedar language, OS, and programming system: successor to Smalltalk. Shows GUI, Documentation Browser for on-line documentation. Cedar interpreter part of environment: basically saying having a source-level debugger is helpful. Tiled window management. Window==Viewer/View. [ThomasJC85] Thomas, John C. "Human Factors in IBM", IBM Thomas J. Watson Research Center, Report RC 11267 (#50753), July 15, 1985 organizational note on human factors group at IBM: mostly in service to product offerings. [Tyler84a] Tyler, Mark "Touch Screens: Big Deal or No Deal?", Datamation, Jan 1984, pp 146-154 (partial copy) Review of Easel system at Chemical Bank: mentions limitations of touchscreens: gorilla-arm/arm-fatigue, low resolution. Shows highlighting of buttons in screen, simulated virtual keyboard (alphabetical) for stock trading. Cited in touch-screen references such as C. Marlin Brown. [UmphressD85a] Umphress, David and Williams, Glen "Identify verification through keyboard characteristics", Int'l. Jnl. of Man-Machine Studies, Vol 23, 1985, pp. 263-273 Biometric authentication of users based on typing patterns: keystroke latency / time between strokes, per key. Can be applied continuously while user is at keyboard. See also Saltzer 1975. [VanDamm84] Van Damm, A. "Scientific American", Introduction to special issue on computer technology, September 1984 Apple MACINTOSH user-interface is really neat [VanRaamsdonk84] Van Raamsdonk, Cornelis "Apparatus for Text Editing and Processing", United States Patent 4,475,239, October 2, 1984, assigned to Olympia Werke AG, Wilhelmshaven, West Germany. User-interface for doing word processing with handwriting recognition, handwriting annotation input: prior art to just about every ink-editing system you can imagine. Editing of electronic ink. [Verschueren84] Verschueren, W., Schaeken, B., Rene de Cotret, Y., and Hermanne, A. "Structural Recognition of Handwritten Numerals", Proceedings of IEEE 7th International Conference on Pattern Recognition, Montreal, Canada, July 30-August 2, 1984, pp 760-762 Cites 6-0 ambiguity for single writer Says 79% accuracy, "then improved", but does not say how much Kerrick88 cites as example system using both locally and globally computed features [VideoPlace85a] Krueger, M.W.; Gionfriddo, T.; and Hinrichsen, K. "VideoPlace Sampler: Experiments in Man Machine Interaction", Artificial Reality Lab, Univ. of Conn., 1985. fetched 2012 from YouTube.com One of several videos of Myron Krueger work on VideoPlace, showing multi-hand/multi-touch gestures such as pinch and stretch: e.g. at running time 2:28 and 4:17. Two finger typing. [Wacom85a] WACOM Corporation "Wacom CU-510 Power Supply for Wacom digitizer tablet, ca. 1985", WACOM Corporation (physical device) Just the power supply, check for tablet in attic [Wakahara84a] Wakahara, T. and Umeda, M. "On-line Cursive Script Recognition using Stroke Linkage Rules", Proceedings of IEEE 7th International Conference on Pattern Recognition, Montreal, Canada, July 30-August 2, 1984, pp 1065-1068



Not really cursive handwriting recognition: connected strokes in Japanese Kanji writing. Appears to presume stroke order is highly predictable for Japanese/Chinese writing. [Wakahara84b] Wakahara, T., Odaka, K., and Umeda, M. "Electro-Luminescent Display Superposed by Transparent Electromagnetic Coupling Tablet and Its Application to Script Input Japanese Word Processor", (source?) (in Japanese), 1984 Integrated DCR tablet and display, electronic ink Display parallax corrected by user-selectable offset in electronic ink [Waltz85] Waltz, D.L. and Pollack, J.B. "Massively parallel parsing", Cognitive Science, Vol 9, 1985, pp 51-74 Cited in Feldman88 [WangPSP85] Wang, P.S.P. "Dual Level Pattern Recognition System", United States Patent 4,521,909, June 4, 1985, assigned to Wang Laboratories, Inc., Lowell, Massachusetts. Two-level character recognition: a first cut may or may not work, but a second pass sorts out characters left ambiguous by the first pass Template matching with learning for new patterns Two-level template matching using a coarse digitizing grid, then a fine grid for what's left: only advantage is shorter comparison time [Ward85a] Ward, Jean Renard "Software Control at the Stroke of a Pen", SIGGRAPH Video Review, ACM Press: CHI '86 Videotape of live demonstration during CHI '85 conference showing gesture recognition / recognition macros, handwriting recognition, interfaces to AutoCad CAD and other applications. [Ward85b] Ward, J. and Blesser, B. "Implications of Using Interactive Hand-print Character Recognition for Computer Input", Proceedings of the 1985 COMPINT Trends and Applications in Computer Graphics Conference, May 1985, IEEE Catalog No 85CH2148-5 Pre-cursor to Ward85 IEEE article: only cite Ward85, not this one [Ward85c] Ward, J. "UNIX as a development tool for a non-UNIX microprocessor", CommUNIXations, August/September 1985, Vol V No 5, pp 26-30; ACM SIGSOFT Software Engineering Notes Vol 10 No 3, July 1985, pp. 95-108 Cross-platform development for Pencept handwriting recognition tablet processor from Unix to embedded processor, using Unix-compatible kernel. Mostly a detailed report on requirements for embedded firmware development in "C" and Unix. [Ward85d] Ward, J. "Method and Apparatus for Removing Noise at the Ends of a Stroke", United States Patent 4,534,060, August 6, 1985, assigned to Pencept, Incorporated, Waltham, Massachusetts. Eliminate wobbles at ends of strokes for pre-processing handwritten characters [Ward85e] Ward, J. and Nilssen, A. "Apparatus and Method for Emulating Computer Keyboard Input with a Handprint Terminal", United States Patent 4,562,304, assigned to Pencept, Incorporated, Waltham, Massachusetts, December 31, 1985 Keyboard emulation patent intended to blockanybody from doing on-line character recognition with standard keyboard input [Ward85f] Ward, Jean Renard "Pencept's Challenge", /usr/group CommUNIXations, August/September 1985 engineering article on hardware/firmware development effort for the PenPad 300/320 tablet/handwriting products [Ward85g] Ward, J. and Blesser, B. "Interactive Recognition of Hand-printed Characters for Computer Input", IEEE Computer Graphics and Applications, September 1985, pp 24-37 Survey/review article on problems and techniques associated with Gesture/Handwriting recognition. Mentions digitizer performance and accuracy, ambiguity of training-based recognition (train-the-user / train the recognizer), boxed input, parsing of strokes to characters in text. PenPad handwriting recognition / gesture product from Pencept circa 1985: has a press-and-hold/long-hold operation, using the barrel button. Long-hold button gesture used to change modes. [Watanabe85] Watanabe, Y., Gyoba, J., Hirata, T., and Maruyama, K. "A Psychological Approach to the Human Recognition of Ambiguous Characters", Journal of the Institute of TV Engineers of Japan, Vol 39 No 6, pp 509515, 1985 (in Japanese). Mentions different "correct" labels for readers and writers of characters Doing same stuff as early Shillman/Blesser work, but for Japanese Refers to "confoundable Katakana" characters "Hayashi's Quantification, model II" [Weibel84] Weibel, Peter "Zur Geschichte und Aesthetik der digitalen Kunst", Supplement to catalog of Ars Electronica 84, Linz Austria, September 1984 History of computer graphics as artistic medium (mostly animation), first section a discussion of the technical implementation. States that for graphical artistic input, a data rate of 24 points/second on a digitizing tablet with a stylus is usually enough. [Werth85a] Werth, L.J. and Paulson, L.G. "Pattern Processing System", United States Patent 4,541,115, September 10, 1985, assigned to Pattern Processing Technologies, Minneapolis, Minnesota.



Visual image inspection pattern matching using address loop correlation between images: recognition part of a trainable system Visual pattern image recognition: responds to time-varying images (patterns over time: claims application to speech recognition as well) [Werth85b] Werth, L.J. and Paulson, L.G. "Training Controller for Pattern Processing System", United States Patent 4,504,970, March 12, 1985, assigned to Pattern Processing Technologies, Minneapolis, Minnesota. Visual image inspection pattern matching using address loop correlation between images: trainable system [Weyer85] Weyer, S.A. and Borning, A.H. "A prototype electronic encyclopedia", ACM Trans. Office Information Systems, Vol 3 No 1, 1985, pp 63-88 Prototype of interactive electronic encyclopedia, beyond browsing information hyperlinked based on semantic content, multimedia pages. Physical user interface of prototype is mouse/GUI. See also ThingLab. [WhetstoneA84a] Whetstone, Albert L. "Position Determination Device", United States Patent 4,482,784, November 13, 1984 Magnetostrictive digitizer tablet using L/C delay line tuned to match propagation speed of magnetostrictive pulses. [Whetton85] Whetton, Cris, quoted in "A Glitch in Spline Shaved Line", ACM Software Engineering Notes, Vol 10 No 5 , pp 9-10 regarding an incident where published curves for a control system were transcribed from photocopies, October, 1985. Xerox-copy error, for tablets, optical scanners: "Photocopies and scanners cannot lie: but they sure can distort!" [WolfeAL84a] Wolfe, Andrew Lawrence and Barrett, Gary Lloyd "Multimodal Touch Sensitive Peripheral Device", United States Patent 6,037,930, Marcy 14, 2000 Priority date is 1984. Touchpad used to simulate mouse mode, absolute mode, joystick mode, and numeric keypad mode in controller. Compare with Buxton virtual devices, or Pencept virtual tablets and virtual devices? [Wong85] Wong, K.H. and Fallsize, F. "Dynamic Programming in the Recognition of Connected Handwritten Script", Proceedings of IEEE Conference on Artificial Intelligence Applications, 1985, pp 666-670 Tests limited to only one writer Script recognition: complains about segmentation mismatches Context: says storing templates for all possible words not practical [WuWT84] Wu, W.T. "Heuristic Approach to Chinese-Character Search", Kybernetics (Great Britain), Vol 13 No 1, 1984, pp 39-42 (abstract only). [Yankelovich85a] Yankelovich, H., Meyrowitz, N., and van Dam, A. "Reading and writing the electronic book", IEEE Computer, Vol 18 No 10, pp 15-30, 1985 Electronic book / e-book: mostly about hypertext linking and (multiple) reader annotations, with multimedia -- not portability. Cited in Barker,PG89 [Ye84] Ye, P.J., Hugli, H., and Pellandini, F. "Techniques for On-line Chinese Character Recognition with Reduced Writing Constraints", Proceedings of the 7th International Conference on Pattern Recognition, Montreal, 1984, pp 1043-1045 On Chinese, 99% accuracy if use stroke order/sequence, 92% if do not Test protocol: says extrapolation from 500 to 2500 Chinese characters straightforward [YouZ84] You, Z. and Jain, A.K. "Performance evaluation of shape matching via chord length distribution", Computer Vision and Graphical Information Processing, Vol 29, 1984, pp 185-198 Cited in Taxt90 [YoungG84a] Young, Gales "Hugh Le Caine's 1948 Sackbut Synthesizer Performance Mode of Electronic Instruments", Proc. ICMC Int'l. Computer Music Conf. 1984, pp. 203..212 Biography and technical details of Hugh Le Caine touch-sensitive electronic musical instruments, Electronic Sackbut 1948, Elmus Labs. Goal was to allow performer to control timbre and other qualities of tone on electronic keyboard instruments. Piano-style keyboard, keys could be deflected left or right to "lip" the pitch. [Zhang84] Zhang, T.Y and Suen, C.Y. "A Fast Parallel Algorithm for Thinning Digital Patterns", CACM, Vol 27 No 3, March 1984, pp 236-239 Skeletonization/thinning based on opposing corners and boundaries along diagonals, preserving endpoints and pixel connectivity [Zue85] Zue, V.W. "The Use of Speech Knowledge in Automatic Speech Recognition", Proceedings of IEEE, Vol 73 No 11, November 1985, pp 1602-1614 General-purpose pattern recognition not extensible to read world of multiple speakers, large vocabulary, continuous speech. Phonetically-based approaches abandoned for adaptive recognition because no one could deal with phoneme detection well. General-purpose pattern recognition appeals because it is simple, but it does not scale up to real problems. Variability in speech due to co-articulation of phonemes. Human reading of spectrograms: speech segmentationbetter on nonsense words than on regular text (!). Knowledge of allophonic variations helpful for phonetic decoding. Use known properties of human



auditory system to reduce irrelevant acoustic variabilities. Much recognition to be learned from studying human decoding of speech signal. Synthesis (artificial generative variability) methods no help on analysis/recognition. Stressed syllables are "islands of reliability" in speech segmentation. Recognition dichotomy of template matching vs feature extraction not absolute. Template matching works well on small, distinct set, but not extensible to real world. Speech signal is a special kind of signal. Ignorance: correctly defining what you do not know is critical: tells you what to work on to make the best progress. Ignorance: admit what you do not know, then do something about it. biblio83 (Prev page)

Main Page







Main Page


[AEGTelefunken86] Richard Oed, Eberhard Mandler, Wolfgang Doster and Juergen Schuerman "Personal correspondence", September 1986 Independent group working on prototype on-line recognition product [Abe86] Abe, K., Azumatani, Y., Mukouda, M., and Suzuki, S. "Discrimination of Symbols, Lines, and Characters in Flowchart Recognition", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1071-1074 Fundamental problem is separating/segmentation (flowchart) symbols drawn small size from characters Problem of broken lines and curves messes up flowchart/character recognition Sketch recognition [AhmedP86a] Ahmed, P. "Computer recognition of totally unconstrained handwritten ZIP codes", Ph.D. Thesis, Department of Computer Science, Concordia University, Montreal, Canada, 1986 OCR static recognition of zip and postal codes (digits only). Hybrid recognition (two layers). Recognition rate of 89%, but substitution rate 10%, rejection rate 5% -- doesn't add up? Secondare back-end recognition for rejected characters, 99% success, but states 7% rejection rate -- does not add up? No statement of segmentation errors, binarization problems. [AminA86] Amin, A. and Masini, G. "Machine Recognition of Multi-font Printed Arabic Texts", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 392-395 Arabic recognition: 95% segmentation, 85% recognition on 100 Arabic words. Commercial product by Doctor Mark Spikell. [AnnoGraphics86a] Spikell, Mark "Personal correspondence", AnnoGraphics / DataPad, 1986 Correspondence concerning presentation to PHIGS/GKS standards on handwriting recognition, presentation at George Mason University. Marketing of DataPad terminal? [AnnoGraphics86b] AnnoGraphics "Datapad (TM) Portable Hand-print Character Recognition Data Entry System", AnnoGraphics, Incorporated, 10720 Kelley Driver, Fairfax, VA 22030, 1986, Mark Spikell, founder. Handwriting recognition / data-entry tablet terminal, single line display, paper forms input. [Aoki86] Aoki, K. and Yamaya, Y. "Recognizer with Learning Mechanism for Hand-written Script English Words", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 690-692



Recognition rate stated as 85.4% with three writers (but says "real" performance would be much better). [Applicon86] Applicon "Bravo3 User's Guide", Applicon Incorporated, Ann Arbor, Michigan 1986 cited in RubinSM84: Ledeen recognizer? [Averbuch86] Averbuch, A., Bahl, L., Bakis, R., Brown, P., Cole, A., Dagget, G., Das, S., Davies, K., DeGennaro, S., de Sourza, P., Epstein, E., Fraleigh, D., Jelinek, F., Katz, S., Lewis, B., Mercer, R., Nada, A., Nahamoo, D., Picheny, M., Shichman, G., and Spinelli, P. "An IBM-PC Based Large-Vocabulary Isolated-Utterance Speech Recognizer", IBM Research Report RC 11663 (Log #53271) 1/28/86, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598 Trainable Speech recognition: a gigantic natural data-base of 27 million words in correspondence was still not enough for coverage of actual vocabulary to get adequate performance from use of a trained dictionary. Enrollment: in 27 million words of correspondence, several days of week and months of year missing from 5000 most frequent and had to be added by hand. [Babaguchi86] Babaguchi, N., Tsukamoto, M., and Aibara, T. "Knowledge Aided Character Segmentation from Handwritten Document Image", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 573-575 Complete character segmentation (in OCR) not possible only with general knowledge. Specific knowledge of characters will help segmentation (i.e. segmentation not general-purpose-method solvable as humans do it?) [BachMJ86a] Bach, Maurice J. "The Design of the Unix Operating System", Bell Telephone Laboratories, 1986 Unix operating system design and programming reference: overview of login implementation, kernel etc. Includes API definitions: chmod for file permissions [Bahl86] Bahl, L.R. "Speech Recognition - An IBM Research Project", unpublished manuscript, December 1986. Author's address: Continuous Speech Recognition Group, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 Speech recognition: enrollment of 1000 sentences, testing on only 50 sentences [BairdHS86a] Baird, H.S. "Accurate Skew Estimation and the Top-Down Analysis of Document Images", unpublished manuscript, December 12, 1986, submitted to 1987 IEEE/CS 1st International Conference on Computer Vision, London. Author's address: AT&T Bell Laboratories, 2C-557, 600 Mountain Avenue, Murray Hill, NJ 07974 Published version? Automatic rotation/scaling/skew-angle correction using "minimum energy" in OCR preprocessing. [BairdHS86b] Baird, H.S., Kahan, S., and Pavlidis, Theo "Components of an Omnifont Page Reader", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 344-348 Segmentation: Accidental merging/splitting of characters known to be a severe problem in optical character recognition. Claims that production of multi-font OCR recognition an engineering technology problem, not science [Bao-chang86] Bao-chang, P. "Floating Mask Method for Extracting Hand-printed Character Features", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 324-326 Chinese OCR: 99.93% accuracy, 200,000 samples of 51 characters by 90 persons Testing: uses very small subset of Chinese, thereby avoiding the problem of large character set. [BarkerPG86] Barker, P.G., Najah, M., and Manji, K.A. "Pictorial communication with Computers", manuscript in preparation, October, 1986. Interactive Systems Research Group, Department of Computer Science, Teeside Polytechnic, County Cleveland, United Kingdom. Pictorial user interfaces (icons): user interface varies dynamically -- changes in response to user actions appropriately. Example shows system with digitiser/tablet, static paper overlays (templates), pecking/pointing to printed icons/pictures. Character stroking/handprint terminal with handwriting recognition. [BierEA86a] Bier, Eric Allen and Stone, Maureen C. "Snap-Dragging", Proc. SIGGRAPH '86, Dallas, Texas, Aug 1986, SIGGRAPH Vol 20 No 4, 1986, pp. 233-240 Snap to grid vs. heuristic constraints similar to compass and ruler for pointing and positioning, e.g. in Gargoyle technical illustration system. Gesture of touching two points of a polygon established constraint of horizontal or vertical. [Bin86] Bin, L. and Shuxiang, Z. "A New Approach to Recognition of Both Handwritten and Multi-font Printed Chinese Characters", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 641-643 Model for recognition taken from Chinese education from psychologists (cognitive?), educators, linguists. The better a method fits a (cognitive) model, the better the recognition algorithm is [BlesserB86a] Blesser, B. "Digitizing Tablet System Having Stylus Tilt Correction", United States Patent 4,577,057, assigned to Pencept, Incorporated, Waltham, Massachusetts, March 18, 1986 Two-coil digitizer tablet stylus/pen for tilt measurement and/or error correction



[BlesserB86b] Blesser, B. "Digitizing Tablet System Including a Tablet Having a Grid Structure Made of Two Orthogonal Sets of Parallel Uniformly Sized and Spaced U Shaped Loops of Conductive Material", United States Patent 4,582,955, assigned to Pencept, Incorporated, Waltham, Massachusetts, April 15, 1986 Digitizer tablet geometry to minimize stray signal pick-up [BlesserB86c] Blesser, B., and Ward, J. "Human Factors Affecting the Problem of Machine Recognition of Handprinted Text", Vol III Proceedings of the 7th Annual Computer Graphics Conference, National Computer Graphics Association, May 11-15 1986, pp 498-514 Paper shows all the variations of "G"s: actual presentation did "A"s. Gives many examples of combinatorial explosion of handwritten character variability from base forms. [BonoPR86] Bono, Peter R. "Support for Character Recognition Devices by the Graphics Standards", presentation to PHIGS graphics-standards committee, May 23, 1986 Presentation on behalf of several on-line pen computing companies to the ANSI graphics standards committees: organized by Pencept, includes CIC, IBM others. Request concerned defining handwriting input for user interfaces (characters with position: like gestures) in some fashion, rather than having separate locator and text input. [Borning86] Borning, Alan "Graphically Defining New Building Blocks in ThingLab", Human-Computer Interaction, 1986, Vol 2 No 4 pp 269-295 (abstract only) Animation editor in ThingLab, with object constraints [BraultJJ86] Brault, J.J. and Plamondon, R. "Coupling Visual and Dynamic Features to Study Handwritten Signatures", Proceedings of Graphics Interface/Vision Interface 86, Vancouver, B.C. 26-30 May 1986 Shows defective tablet line, but thinks it's just digitizer noise. [Burr86] Burr, D.J. "A Neural Network Digit Recognizer", Proceedings of the 1986 International conference on Systems, Man, and Cybernetics, Atlanta, Georgia, 1986, pp 1621-1625 Martin,GL90 [Buxton86] Buxton, W. "Chunking and Phrasing and the Design of Human-Computer Dialogues", Proceedings of IFIP World Computer Congress, Dublin Ireland, September 1-5, 1986; also Baecker95a Towards the Year 2000 Gesture input / sketch recognition in graphical user interfaces [Buxton86a] Buxton, William, and Myers, Brad A. "A Study in Two-Handed Input", to appear in 1986 Conference on Computers and Human Interaction, CHI'86, Boston April 12-17, 1986 digitizer tablet plus other controls, for selection of text: multi-touch/multi-hand. [Buxton86b] Buxton, William "There's More to Interaction than Meets the Eye", in User Centered System Design: New Perspectives on Human-Computer Interaction, Norman, Donald A. and Draper, Stephen W, Eds, Lawrence Erlbaum Associates, Hillsdale NJ and London, 1986 Comparison/study of locator/pointing devices and user interfaces: isometric, and acceleration joysticks (and multiple isometric joysticks: different size and force), trackballs, nulling problem for multiple sliders. Tablet best, in part because it can emulate all other devices, as general purpose tools can be used for more limited applications. Notes that for two-hand (multi-touch) inputs, input grammars don't do well. [BuxtonWB86c] Buxton, William "A Study in Two-Handed Input", University of Toronto, 1986: available at billbuxton.com Video presentation showing user interface using two hands: mixer panel for left hand, tablet with puck for right hand. Companion video to "A study in Two Handed input". [BuxtonWB86d] Buxton, William "Buxton/Ward personal correspondence", Jean Renard Ward, 1986 Personal correspondence with Bill Buxton, 1986/1987. Conference panel, two-handed (multitouch) tablet, etc. [CIC86a] Communications Intelligence Corporation "Product literature CIC Handwriter for IBM 5550/5540 terminal, IBM JX terminal, Handwriter for Japanese PC", Communications Intelligence Corporation, 1986 (in Japanese) Line of handwriting recognition tablets using coprocessor board for Japanese Katakana/Hiragana handprinted characters. Compare with same company Proficient and other applications in U.S. [CIC86b] Communications Intelligence Corporation "Correspondence and press file, Communications Intelligence Corporation", Communications Intelligence Corporation, 1986-1990 Correspondence file with CIC. 1986 entry into US market after Japan. Technial information on Handwriter handwriting recognition forms data entry combining data collection (paper forms) and data entry (PC), "legibility exercises" to train users on ambiguous character e.g. 5-S, WRITE-ON keyboard replacement users have difficulty learning a keyboard (circa 1986). application architecture includes signature verification, CAD input (see also Proficient), forms data entry, templates for Lotus/123 spreadsheet. Handwriting recognition board supports third-party tablets. [Carpenter86] Carpenter, G.A. and Grossberg, S. "A Massively Parallel Architecture for a Self-organizing Neural Pattern Recognition Machine", manuscript in preparation, Authors' address: Center for Adaptive Systems, Department of Mathematics, Boston University, Boston, Massachusetts 02215 Adaptive recognition for static OCR recognition: gives example of "simple" 5x5 characters, but patterns.



Comments: Does not match human recognition of patterns (random 5x5 bits as input). [Casey86] Casey, R.G. "Text OCR by Solving a Cryptogram", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 349-351 Adaptive recognition for static OCR using clear-text and "unknown" type-font text for a cryptogram: but is clear-text always known? [Casio86a] Casio "Casio IF-8000 Calculator", Collection from Pocket Computing Museum, http://cdecas.free.fr/computers/pocket/if8000.htm, 2002 1984 Calculator (this reference says 1986) with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad Same reference includes Apple Newton Message Pad 110 [Casio86b] Casio "Digital Diary Casio IF-8000 Operation Manual", Casio, 1984/1986 Two-side (Refalo-style) calculator with small touchscreen display and calculator keypad on the left, alphabetic keypad on the right. Electronic ink note-taking on the touchscreen. Memos and notes can include both (typed) text and electronic ink notes. Calendar and address book. Compare with similar Casio calculator with handwriting recognition for calculator input. [Chunbiao86] Chunbiao, G. and Guorong, X. "Automatic Recognition of Printed Chinese Characters by Four Corner Codes", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 10131015 6763 Kanji/Chinese characters in minimal Chinese standard (3000 common): much more than Japanese standard (1493) Chain codes/segments: Chinese/Kanji characters in only nine kinds of primitive segments [CinaVJ86a] Cina, Vincent J. Jr.; White, Steve R.; and Comerford, Liam "ABYSS: A Basic Yorktown Security System: PC Software Asset Protection Concepts", IBM Research Report Number RC 12401, December 18, 1986 ABYSS: A Basic Yorktown Security System for PC software asset protection: copy-protection, DRM. Cites to cryptography, does not give details. Protected (trusted) processor, execution right, application data store. Relies on electronic tokens (e.g. smart cards) [CohenM86] Cohen, M. and Grossberg, S. "Neural dynamics of speech and language coding: developmental programs, perceptual grouping, and competition for short-term memory", Human Neurobiology, 1986, Vol 5, pp 1-22 According to Daniel Bullock's letter from BU in 1986, Cohen did handwriting recognition in 1970. Theoretical adaptive work: "random growth" of human brain [Cordella86] Cordella, L.P. and Marcelli, A. "Normalization and Decomposition of Thin Lines Representing Handprinted Characters", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 723-725 Decomposing characters into segments (curves, arcs) that are invariant with respect to handwriting variability (compare to chain-code segments) Structural analysis beats the pants off of statistical / adaptive methods Cites Suen/Shillman paper on U-V discrimination Recognition by grammar/syntax of (chain-code) segments [Crane86] Crane, H.D., and Ostrem, J.S. "Confusion Grouping of Strokes in Pattern Recognition Method and System", United States Patent 4,573,196, February 25, 1986, assigned to Communication Intelligence Corporation, Menlo Park, California. Confusion matrix for Chinese/Kanji character recognition, using distance and curvature metrics [Currit86] Currit, P.A., Dyer, M., and Mills, H.D. "Certifying the Reliability of Software", IEEE Trans. Software Engineering, Vol SE-12 No 1, January 1986, pp 3-14 The accepted approach to software development is to specify and design a product -- and then to test selectively with cases perceived to be typical of those requirements. Frequently the result is a product which works well against inputs to be tested but which is unreliable in unexpected circumstances Testability of software: software-engineering regressions tests Statistical testing of software can predict reliability: however, it cannot tell you about the existence, severity, or number of remaining errors Sonic tablet digitizer: 0.1mm accuracy over 1.0 meter square active area (but then says +-2.0mm after all calibration!) Sonic tablet digitizer: local drafts throw off same as 1 degree C temperature Sonic tablet digitizer: calibrate to correct for air speed vs temperature [DeBruyne86] de Bruyne, P. "Compact Large-Area Graphic Digitizer for Personal Computers", IEEE Computer Graphics and Applications, Vol 6 No 12, December, 1986, pp 49-53 Sonic digitizer tablet: ultrasonic shock wave: wireless/cordless cursor/stylus, using spark gap sonic pen, accuracy/error is 0.1% of distance measured [Derouault86] Derouault, A. and Merialdo, B. "Natural Language Modeling for Phoneme-to-Text Transcription", IEEE Trans. Pattern Analysis and Machine Intelligence", Vol 8 No 6, November 1986, pp 742-749 Statistical modelling and grammatical rules combined for context in French natural language [DhawanSK86a] Dhawan, Satish K. "Field-coupled pointing device", United States Patent 4,564,835, January 14, 1986 Mouse tablet: tablet has array of electromagnetic (coil) or electrostatic single elements in a grid matrix,



mouse motion determined by number of changes of phase passed over. Mentions some elements being driving out of phase with others to resolve "ambiguity points" when mouse sensor is directly over a physical pixel/element. [DosterW86a] Doster, W. "Designing a Document Analysis System", tutorial at 8th International Conference on Pattern Recognition, Paris, France, 27-31 October, 1986 Cited in Baird86a [DosterW86b] Doster, W. "private letter of May, 1986", AEG Aktiengesellschaft, Forschungsinstitut Ulm, Sedanstrasse 10, D-7900 Ulm, West Germany. Doster also has a DCR (instead of OCR) character data base collection. [DowningD86a] Douwning, Duglas and Covington, Michael "Dictionary of Computer Terms", Barron's 1986(hardcopy book) Technical dictionary of computer terms 1986 [Ewing86] Ewing, J., Mehrabanzad, S., Sheck, S., Ostroff, D. and Shneiderman, B. "An Experimental Comparison of a Mouse and Arrow-Jump Keys for an Interactive Encyclopedia (abstract only)", International Journal of ManMachine Studies, Jan 1986, pp. 29-45. abstract only: see also Ewing85 User-interface on pointing methods: applies to mouse emulation on a tablet? Poor typists prefer mouse for pointing, good typists prefer arrow keys. [Firdman86] Firdman, H.E. "Components of AI Systems", AI Expert, Vol 1 No 1, 1986, pp 81-85 General comments on what an AI system "is" DCR as Artificial intelligence: "domain specific: rules of thumb extracted from experts" [FlurryGA86a] Flurry, Gregory A. "Real-time rub-out erase for an electronic handwriting facility", United States Patent 4,633,436 assigned to International Business Machines Corp, Armonk, New York, December 30, 1986 Stylus patent for tablet digitizer: Looks very similar to erase function of Wang Freestyle: real-time rubout erase on back end of stylus. [FoleyJD86] Foley, J. et al "Managing the Design of User-Computer Interfaces", Report available from Computer Graphics Consultants, Incorporated, 616 G Street SW, Washington DC 20024, 1986 Jim Foley's consulting report on user interface subsystems [Forethought86a] Forethought Inc. "FileMaker Plus Guide to FileMaker Plus Templates", Forethought Inc, 1986 Database development system for MacIntosh by Nashoba Systems. [Fu86] Fu, K.S. "A Step Towards Unification of Syntactic and Statistical Pattern Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 8 No 3, May 1986, pp 398-404. Previously published in Vol 5 No 2, March, 1983 Could also be listed as Fu83: was re-printed in same journal Attributed grammar, control diagram, semantics, statistical pattern recognition, syntactic pattern recognition, syntax-semantics tradeoff Subpatterns, substructure (primitives) of pattern as features There is yet no systematic method to select appropriate syntactic and semantic complexities (features) for a specific pattern recognition problem Gives examples of distorted hand-written "E"s, and a possible tree/heirarchical representation of the structure for recognition [GanesanA86a] Ganesan, Apparajan and Weise, Wayne A. "Method and apparatus for capacitive keyboard scanning", United States Patent 4,651,133, March 17, 1987 Capacitive keypad/keyboard of keys on a grid: ratiometric measurement of total relative capacitance of all other keys in the row, thereby eliminating problems from variations and capacitance. Also detection of multiple key closures (without limit) by comparing both by row and by column to determining a key closure. Capacitors can be cross-point connected: multi-touch [GarciaJD86a] Garcia, John D. "Personal Identification Apparatus", United States Patent 4,621,334, November 4, 1986 Authenticate user biometrically using keystroke latency/delay pattern of typing on keyboard. Cites to Saltzer 1975. [GarrettLN86a] Garrett, L. Nancy, Smith, Karen E. and Meyrowitz, Norman "Intermedia: Issues, Strategies, and Tactics in the Design of a Hypermedia Document System", Proc. of 1986 ACM Conf. on Computer-supported cooperative Work, pp. 163-174 Hyperlinks in electronic document images in the text [GillEN86] Gill, E.N. "An Evaluation of the Documented Requirements of the SSP UIL and a Review of Commercial Software Packages for the Development and Testing of UIL Prototypes", NASA/ASEE Report N87-16761, Contract NST 01-002-099, August 8, 1986 Summary report of 4GL / UIL User Interface Languages for the proposed Manned Space Station: cites Pencept PenPad for touch/stylus based user interface using a digitizer tablet, including Word and PenCAD gesture-based ("Recognition Macro") interfaces [GrooverMP86a] Groover, Mikell P.; Weiss, Mitchell; Nagel, Roger N.; and Odrey, Nicholas G. "Industrial Robotics: Technology, Programming and Applications", McGraw-Hill, 1986 (hardcopy book)



Cited in Lindemann10. multitouch / tactile sensor: page 152, figure 6-5, Lord Corporation LTS 300 Series with 80x80 resolution, each point reports force magnitude. [Guanxiong86] Guanxiong, Z. and Hongyuan, W. "Angle Coding and it's Application in Pattern Recognition", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1001-1003 Chain-codes: eight-direction angle codes (Freeman codes) [Hase86] Hase, M., Suzuki, G., and Hisayasu, I. "A Method for Extracting Marked Regions from Document Images", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 780782 Separating graphics and text characters from an image [HeckbertPS86a] Heckbert, Paul S. "Survey of Texture Mapping", Proc. Graphics Interface '86, Vision Interface '86, May 1986 pp. 207-212 Survey of papers on texture mapping: texture mapping two phases: map texture space to object space (surface), then map object space to 2D screen space. Transformations can be combined into one. Similar to image warping, correction for geometric distortion. Orthographic/Perspective projections. [HeckbertPS86b] Heckbert, Paul S. "Survey of Texture Mapping", IEEE CGA, Nov 1986, pp. 56-67 Later version of Graphics Interface paper, same title. [HekmatpourS86a] Hekmatpour, S. and Ince, D.C. "Forms as a Language Facility", A.C.M. SIGPLAN Notices, Vol 21 No 9, September 1986, pp 43-48 Quotes Gehani82: Proposes forms (for data entry: fields etc.) as so important to deserve own data type / objects in a language. (Object-based programming answers this.) Compare explanation with Pencept and other forms packages for handwriting recognitions. [Hidai86a] Hidai, Y., Ooi, K., and Nakamura, Y. "Stroke Re-ordering Algorithm for On-line Hand-written Character Recognition", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 934-936 BLRT/chain-code: segments at eight discrete slants [HidaiY86b] Hidai, Y., Ooi, K., Nakamura, Y., and Kurosawa, Y., "Hand-written characters segmenting system", European Patent 177319, April 9, 1986, assigned to Toshiba. Hand-written characters segmentation system has tablet determining boundary between characters according to stroke duration (inter-stroke timeout) and coordinates data (proximity height from tablet). [HillRD86a] Hill, Ralph D. "Supporting Concurrency, Communication, and Synchronization in Human-Computer Interaction: The Sassafras UIMS", ACM Trans. on Graphics Vol 5 No 3, July 1986, pp 178-210 UIMS User Interface management system supporting concurrent (multi-stream) input from multiple devices: e.g. mouse/digitizer and keyboard. Example of two-handed (multi-touch) paint program. Multiple concurrent virtual devices on a touch tablet (see Buxton). [Holbaek-Hanssen86] Holbaek-Hanseen, Erik, Braten, Knut, and Taxt, Torfinn "A General Software System for Supervised Statistical Classification of Symbols", Proceedings of ICPR International Conference on Pattern Recognition, Paris, October 27-31, 1986, pp 144-149 Handwriting character recognition using grid lengths of chain code skeletons, boundary tracing, "auxiliary features" [Honeywell86a] Honeywell Information Systems "MULTICS Administration Maintenance and Operations Commands", Honeywell Information Systems GB64-00A, November 9186 Command-line documentation for Multics segmented operating system: all files memory-mapped, access control lists (for files and all code segmenets) use syntax expressions for access permissions and priviledges. access granted to person.project.tag, supports wildcards. Both discretionary (ACL) and nondiscretionary/mandatory (AIM access isolation mechanism) access control. AIM has minimum/maximum range level. [HongoY86a] Hongo, Yasuo and Nitta, Yoshio "Pattern Recognition Apparatus", United States Patent 4,628,533, December 9, 1986, assigned to Fuji Electric Company, Limited, Kanagawa, Japan. Character recognition by pixel matching on OCR [HudsonSC86a] Hudson, Scott E. and King, Roger "A generator of direct manipulation office systems", ACM Trans. on Information Systems, Vol 4 No 2, April 1986, pp. 132..163 HIGGENS - Human Interface Graphical Generation System: Direct manipulation (generator) UI for office entities, included undo/reversal/recovery mechanism. Drag object to trash can to delete. Xerox Star as direct manipulation system. [HullJJ86a] Hull, Jonathan J. "Hypothesis generation in a computational model for visual word recognition", IEEE Expert, 1986, pp 63-70 Context in optical character recognition of type fonts: Cited in Sinha88 [HullJJ86b] Hull, Jonathan J. "The Use of Global Context in Text Recognition", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1218-1220 Context via "nearest neighbor" words from a dictionary Cognitive: OCR recognize words by overall outline



of vertical bars, etc., not actual individual letters Cites Shillman on use of human cognitive psychology, but this time it's on word-level recognition [HullJJ86c] Hull, Jonathan J. and Srihari, S.N. "A computational approach to word shape recognition: Hypothesis generation and testing", Proceedings of IEEE-CS Conference on Computer Vision Pattern Recognition, June 1983, pp 156-161 Hypothesis testing for optical character recognition of font characters: tree search -- compare with PenCept pair-wise binary comparison? Cited in Bozinovic89 [HullsLR86a] Hulls, L.R. "Polyphase Digitizer", United States Patent 4,570,033, assigned to Numonics Corporation, Lansdale Pennsylvania, February 11, 1986 Phase-shifting tablet digitizer, using three-terminal grid winding. Correction for edge effects (hockey-stick) by adjusting X value when Y value indicates close to edge, and vice versa. [Hutchins86] Hutchins, E., Hollan, J. and Norman, D.A. "Direct Manipulation Interfaces", in User-Centered System Design, D.A. Norman and S. Draper, editors, Erlbaum Associates, Hillsdale, New Jersey, 1986, pp 87-124 Cited in JohnsonJ89 Direct-manipulation graphical user interface: model objects must behave as user expects real objects to behave that the image represents: automating response is not enough. Cites to Buxton and Minsky on using pressure and torque for touch input. [ICSA86] ICSA "CRS (Character Recognition System) software product description", Intelligent Computer Systems and Applications, Inc., Box 1555, Station H, Montreal Quebec H3G 2N5 Canada, 1986 Ching Suen's commercial company to sell OCR character recognition software. [Inagaki86] Inagaki, Naoki "Key-In Device", United States Patent 4,578,811, March 25, 1986 Zone-based / matrix based character recognition, using the keypad of the Casio 8000 calculator as the input table. Compare: Unistroke/ single stroke character recognition recognition software. [JacobRJK86a] Jacrob, Robert J.K. "A specification language for direct-manipulation interfaces", ACM Trans. on Graphics, Vol 5 No 4, October 1986, pp. 283-317 Direct-manipulation user interfaces GUI definition, using inheritance of graphical objects. Describes context-sensitive gestures (p. 308). [JaegerRP86a] Jaeger, Robert P. "Patterned Conductive Ink Touch Panel", United States Patent 4,625,075, November 25, 1986 Resistive sheet/film touch digitizer using transparent conductive ink. Pattern of electrodes at edges compensates for distortion of resistance. [JandrellLHM86a] Jandrell, L.H.M. "Data Input Device with a Circuit Responsive to Stylus Up/Down Position", United States Patent 4,575,580, assigned to Astec International, Limited, Kowloon, Hong Kong. Resistive sheet tablet using a force/pressure-sensitive conductive sheet Gives fix for wild-points at starts and ends of strokes from up/down contact resistance on resistive sheet tablets [JelinekF86a] Jelinek, F. "Markov Source Modeling of Text Generation", unpublished manuscript, December 1986. Author's address: Continuous Speech Recognition Group, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 Speech: probabilistic language models for vocabulary context Cites need for billions of samples to get statistically reasonable vocabulary probabilities [JelinekF86b] Jelinek, F. "Self-Organized Language Modeling for Speech Recognition", unpublished manuscript, December 1986. Author's address: Continuous Speech Recognition Group, IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 Speech recognition: has large data-bases of business correspondence for vocabulary [Jorgensen86] Jorgensen, C. and Matheus, C. "Catching Knowledge in Neural Nets", AI Expert, December 1986, pp 31-38 General overview of neural networks, perceptrons, adelines, etc. "credit assignment problem" in perceptrons and adaptive recognition Neural nets making a comeback: future still not proven [Kable86] Kable, R.G. "Electrographic Apparatus", United States Patent 4,600,807, assigned to Scriptel Corporation, Columbus, Ohio, July 15, 1986 Scriptel patent on resistive-sheet, electrostatic/capacitive-coupling digitizer tablet. Mentions correction matrix for entire surface of a digitizer tablet [KaehlerT86] Kaehler, Ted and Patterson, Dave "A Taste of Smalltalk", Norton and Company, New York, 1986 Introduction to Smalltalk language and environment using Towers of Hanoi programming example. (Electronic file.) [KankaanpaaA86] Kankaanpaa, Arto "An Advanced display System with Natural Interactivity", Proc. of Eurographics 86,European Computer Graphics Conference and Exhibition, Lisbon, 25..29 August, 1986, pp 186..193 Text editing with single-stroke (mostly) gestures on integrated touchscreen digitizer and LCD display, suggests electroluminescent or plasma display. [Kao86] Kao, H.S.R., Van Galen, G.P., and Hoosain, R. "Graphonomics: Contemporary Research in Handwriting",



North-Holland Publishers, Amsterdam-New York-Oxford-Toronto, 1986, Number 37 of "Advances in Psychology", Stelmach, G.E. and Vroon, P.A., editors. [Kasturi86] Kasturi, R., Shih, C., and Fletcher, L.A. "An Approach for Automatic Recognition of Graphics", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 877-879 Separating graphics and text characters from an image [KatayamaH86a] Katayama, Hildeji; Katoh, Keisuke; Kita, Junya; and Shimada, Motowo "Digital contrast control circuit for display unit", European Patent Application EP0172343A2, Feb 26, 1986 Contrast control from keyboard using up/down counter circuit. [Kay86a] Kay, Alan "A Talk by Alan Kay: summary of video interview Alan Kay talks about Sketchpad, Grail, the Dynabook", http://www.newmediareader.com/cd_samples/Kay/; The History of the Personal Workstation, 27 May 1986 Grail system with freehand input of letterforms, boxes, lines, corrections to previous drawings, Sketchpad, Dynabook. Video includes Grail system showing handwriting recognition (with lightpen) and scratchout gesture. Screenshots from video in file. File also has notes on KiddiKomp (later version of Dynabook), Flex Machine (with tablet) ca. 1970. [Kay86b] Kay, Alan "Alan Kay explains Sketchbook, Grail, and the Dynabook", Available from YouTube (2009): http://www.youtube.com/watch?v=A6h-zDOggYQ Grail system with freehand input of letterforms, boxes, lines, corrections to previous drawings, Sketchpad, Dynabook. Video includes Grail system showing handwriting recognition (with lightpen) and scratchout/rub-out gesture. [Kondo86] Kondo, S. and Attachoo, B. "Model of Handwriting Process and its Analysis", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 562-565 Features: distortions of end points on strokes do not affect human classification of handwriting, of middles of strokes does affect recognition Stroke structure (order) very important in handwriting process, and therefore should be important in character recognition Big problem in handwriting recognition is finding/extracting features not subject to handwriting distortion. Since set of all characters is not clear cut, any definition based on "invariant" features of a class is likely to be contradictory. Simulate handwritten characters: problem is finding out what base forms are that you should vary on. [KovedL86a] Koved, Larry and Shneiderman, Ben "Embedded Menus: Selecting Items in Context", CACM, Vol 29 No 4, April 1986, pp. 312-318 Touchtext: Pop-up menus showing pick-list of alternative words in a spellchecker UI. [Kurosawa86] Kurosawa, Y. and Asada, H. "Attributed String Matching with Statistical Constraints for Character Recognition", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1063-1067 Chain code: use primitive segments and to string matching Exhaustive depth-first search of syntactic string for primitive segments Katakana and alphanumeric characters with BLRT-like primitive segments Combinatorial explosion of production rules Combination of primitive-segment syntactic matching and statistical constraints BLRT-chain codes give multiple-matches (3 or so), so second, ad-hoc comparison analysis for each possible pair of characters [LamSW86a] Lam, S.W. and Baird, H.S. "Performance Testing of Mixed-font, Variable-size Character Recognizers", Computing Science Technical Report No. 126, AT&T Bell Laboratories, Murray Hill, New Jersey, November 6, 1986 Gives statistics for size of test set to justify recognition performance claims of a given percent with a justifiable confidence level To claim 99.5% recognition accuracy with 95% confidence, need 1500 samples To claim recognition accuracy improved from 99% to 99.5% with 95% confidence, need 5000 samples Refers to clustering and partitioning algorithms for feature selection for Bayesian statistics OCR static recognition system was tested without ligatures, without several "special" characters 99.5%+ performance requires very large, carefully-designed tests Test performance of 99.5% was with common cases omitted, and some confusions forgiven For OCR, scanner digitizer performance is critical OCR scanner/digitizer: has short term fluctuations in sensitivity (re-scanning? was there internal averaging?) Testing/training done only with high-quality data, not low-quality For OCR digitizer scanners, scanning performance is not uniform or repeatable page-to-page OCR tested and trained only on data of good quality, but question was raised Resolution: 24 pixels not enough data accuracy for OCR OCR at 200 dots/inch resolution started to fail at 12 point type size Refers to tools to collect automatically a very large (168,000) character test set data base [LaubeM86a] Laube, M. "Audiographic Terminal", ITT Electrical Communication, Vol 60 No 1, 1986, pp 45-50 Says maximum frequency of handwriting is 15 Hz, therefore 40 points/second digitizing rate (don't believe it!). Resistive-sheet tablet for handwriting, force/pressure-sensitive tablet (!!). Handwriting/voice/display terminal, like Interspec [Leedham86] Leedham, C.G. and Downton, A.C. "On-line recognition of Pitman's handwritten shorthand -- an



evaluation of potential", International Journal of Man-Machine Studies, Vol 24, 1986, pp 375-393 9 degree variation in angle of nominal vertical and horizontal strokes Short and long strokes overlap in length Textbook writing (shorthand) is NOT the most recognizable to humans Human recognition: textbook shorthand is 93.5%, student is 97.5%, expert is 48% For small writing, hooks and loops are as big as many of the strokes Unconstrained shorthand input Intended straight strokes consistently tend to be curved Has gall to say "recognition would be better if writing were neater" Ignored strokes if too close to preceeding stroke Stroke length variability is 20% in length Says strokes were "lost" (thrown out) if not classified as "legal" Did not deal with overdrawn/intentionally retraced strokes Says recognition would get better with user practice (human factors/user interface) "no amount of post-processing (context?) will fix segmentation and stroke classification errors" Test protocol intentionally slow to get better quality data Test protocol defeated linguistic context, resulting in users having to concentrate more Refers to need for better (non-simplified) recognition algorithms Cites ergonomic problems of tablet styli Cites "writer's unfamiliarity with the system and capabilities" as performance factor [LetteraC86a] Lettera, C., Masera, L., Paoli, C. and Proinelli, R. "Use of a Dictionary in Conjunction with a Handwritten Text Recognizer", Proc. ICPR(86), pp. 699-701. 1986 Cited by Marlin Eller, Microsoft Pen Computing group [LiebermanH86] Lieberman, Henry "Using Prototypical Objects to Implement Shared Behavior in Object Oriented Systems", ACM SIGPLAN Notices, Vol 21 No 11, November 1986, pp 214-223 Prototype-based languages versus classes: examples include turtle and pen object for drawing, cites to SELF and Smalltalk [Linus86a] Linus Technology "Personal correspondence Linus technology", Linus Technology, 1986 Correspondence file on Linus Technology: handwriting recognition / touchscreen laptop. [Litvin86] Litvin, Y. "Private letter of August 4, 1986", Author's address: Skylight Software, 2 Charles Street, Bedford, Massachusetts 01730 DCR/on-line handwriting recognition not deterministic: there is no formal criteria for correctness (not true: he misunderstood "deterministic") [LongleyD86a] Longley, Dennis and Shain, Michael "Dictionary of Information Technology, Second Edition", Oxford University Press, 1986 (hardcopy book) Technical dictionary [LuH86] Lu, H.E. and Wang, P.S.P. "A Comment on 'A Fast Parallel Algorithm for Thinning Digital Patterns'", CACM, Vol 29 No 3, pp 239-242 Skeletonization/line-thinning, special notes on endpoints NOT to be deleted: compare with smoothing/preprocessing algorithms for digitizer tablet data by Pencept [MaYL86a] Ma, Y.L., Jang, S.Y., and Ma, C. "Pattern Recognition by Circular Layer Code Approach", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 783-785 Chinese OCR, 97.5% on 4 characters tested with 20 samples each (!!!) [Maamari86] Maamari, F. and Plamondon, R. "Extraction of the Analog Pentip Position, Velocity and Acceleration Signals from a Digitizer", Neuronal and Motor Aspects of Handwriting, North-Holland Publishers, J.S.R. Kau, Editor, 1985 Analog filter on digitizer output gives "improved accuracy" (!!??!!) Low-pass filtering on tablet digitizer lets you get "better" time resolution (?) [MacNelly86] MacNelly, Jeff ""Shoe" comic: "I wish I sat behind someone with decent handwriting"", "Shoe", November 21, 1986. Taken from Boston Globe newspaper, Boston, USA, Page 102 Student in a school class during a test: Fill in the blanks: "To the victor belong the *snails*." That can't be right ... I wish I sat behind someone with decent handwriting. (Humorous illustration of the problems of imperfect handwriting recognition, and of the difficulty of human recognition of handwriting) [MaedaY86] Maeda, Y., Yoda, F., Matsuura, K., and Nambu, H. "Character Segmentation in Japanese Handwritten Document Images", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 768-772 [Maier86] Maier, M. "Separating Characters in Scripted Documents", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1056-1058 AI artificial intelligence was seen as panacea Cited in Wolf86 [Mandler86] Mandler, E. "conversation of September, 1986", AEG Aktiengesellschaft, Forschungsinstitut Ulm, Sedanstrasse 10, D-7900 Ulm, West Germany. One-stroke "T" more common in Europe [Mangione86b] Mangione, P.A. "SSI's Phonetic Engine (TM)", Speech Technology Magazine, Mar/April 1986 Speech Systems phonetic feature speech recognition [Mangione86c] Mangione, P.A. "Phonetic Recognition - The Basic Building Block", Speech Technology Magazine, April/May 1986 Speech Systems phonetic feature speech recognition



[Mangione86d] Mangione, P.A. "What about the user?", Proceedings of Speech Tech '86, April 28-30 1986, New York, New York, pp 154-156 Marketing V.P. did paper on Speech Systems' phonetic speech input product [MantasJ86a] Mantas, J. "An overview of character recognition methodologies", Pattern Recognition, Vol 19, 1986, pp 425-430 Cited in Baptista88 [MantasJ86b] Mantas, J. "A Fuzzy Decision Operator in Recognizing Greek Hand-drawn Characters", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 753-755 Pattern recognition is not just pattern classification, but also design of recognition systems Statistical versus syntactic/linguistic pattern recognition (does not like statistical) Statistical/Bayesian recognition cannot be done for handwriting because a priori probabilities cannot be calculated for every feature [Meisel86] Meisel, W.S. "Implications of Large Vocabulary Recognition", Proceedings of Speech Tech '86, April 2830, 1986, New York, New York. Speech Systems speech recognition using phonetics (sloppy cursive): co-articulated speech, vs isolated, connected, continuous Speech: storing multiple-word (multiple-char) segments as special "words" (BLRTs/variability): phoneme detection reduces variability to a set of rules at phoneme level Ambiguity: groups of words (chars) vs single words: "there are four" vs "therefore" Dictionary look-up/search: speed and accuracy of recognition unrelated to Size of dictionary, but only to complexity of syntax Adaptive vs user-independent: controversy over "speaker-independence" Adaptive: must begin at a usable level even so, otherwise bad human-factors (frustration) Adaptive: correction for content, not for error (user wants a different word) Adaptive: separate enrollment session a bad user-interface idea for "normal" speakers Penalty for speaker-independence too high (frustration) Variability: phoneme speech system can extrapolate/predict words after adaptive samples Trade-off between accuracy and constraints on user in speech: What if user uses bad English? [Microsoft86a] Microsoft Corporation "Microsoft Excel User's Guide Version 1.03 with Business Graphics and Database for the Apple Macintosh", Microsoft Corporation, 1986 (hardcopy manual) Excel 1.03 spreadsheet processing for the Apple Macintosh - shows integration with Apple menus. Contains update for version 1.5 describing File contains additional materials: [MooreCA86a] Moore, Cecil A. "Two-layer Touch Tablet", United States Patent 4,587,387, May 6, 1986 Two-layer resistive grid digitizer tablet. Carbon ink used as conductor, is resistive 100K-200K per bar/conductor, or conductive silver ink, voltage divider along conductors. [Morasso86] Morasso, P. "Understanding Cursive Script as a Trajectory Formation Paradigm", in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (North-Holland), 1986, pp 137-167 Hand gestures and handwriting: study of movements. Measures curvature profile, not just velocity profile, of handwriting motion. Claims no handwriting information at frequency above 10 Hz, shows peak at 5 Hz and D.C. (but his samples are mostly loop motion, not fast block printing). Shows plots of low-pass frequency filtering on handwriting data at 10 Hz and below (frequency content of dynamic writing motion: implies only low-rate digitizer tablet needed if it is accurate enough?) Author intends to collect lots of allograph handwriting samples to build a future recognizer for cursive script. [Morita86] Morita, Toshiaki; Horii, Masahiro; Tasaka, Shigeru; Hirose, Hitoshi "Handwriting Character Recognition Device", United States Patent 4,607,386 assigned to Sharp Kabushiki Kaisha, Osaka, Japan, August 19, 1986 Chinese handwritten character recognition without regard to stroke order [Murase86] Murase, H., and Wakahara, T. "Online Hand-Sketched Figure Recognition", Pattern Recognition, Vol 19 No 2, pp 147-160, 1986 Sketch context: must be valid connections to other flowchart symbols 97.2% accuracy: test protocol was only 20 people, nine symbols, four times each Mentions lots of user-interface work on on-line sketch recognition For "standardized" sketch symbols, high variability (no regularity) in stroke order Says stroke order variable, but end/connection points are always the same Refers to stroke connection variability for "sketched" symbols [MyersBA86] Myers, Brad A. and Buxton, William "Creating Highly-Interactive and Graphical User Interfaces by Demonstration", Proceedings of SIGGRAPH '86, pp 249 ff. Direct-manipulation user interface. Simulated mouse device with three buttons on display screen. [NagyG86a] Nagy, G., Seth, S., and Stoddard, S.D. "Document Analysis with an Expert System", in "Pattern Recognition in Practice", Gelsema, E.S. and Kanal, L.N., editors, Elsevier Science Publishers B.V. (North-Holland), 1986 Tiling/windows applied for format of written/printed documents [NagyG86b] Nagy, G.R., Seth, S., Einspahr, K., and Meyer, T. "Efficient Algorithms to Decode Substitution Ciphers with Application to OCR", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 352-355 Adaptive recognition using clear-text and "unknown" type-font text for a cryptogram: but is clear-text



always known? [Nakagawa86a] Nakagawa, Masaki, Manabe, Toshihiko, Aoki, Katsuo, Ikeda, Yuji, and Takahashi, Nobumasa "Online Handwritten Character Recognition as a Japanese Input Method", Proceedings of ???, pp 191-196 on-line handwriting recognition system for Japanese, JOLIS-1: chain codes (8-directional), 29 primitive strokes types by dictionary look-up [Nakagawa86b] Nakagawa, M., Aizawa, T., Komoda, C., Ideda, Y., and Takahashi, N. "Syntactic Pattern Recognition with Stochastic Dissimilarity in Japanese On-line Input Systems (JOLIS)-1/1.5", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1059-1061 Chain-code: Japanese recognition by classifying strokes into primitive strokes Chain-code: eight stroke directions/segments Chain-codes: extension to elastic matching [NespoulousJ86a] Nespoulous, J.; Perron, P.; Lecours, A.R. "The Biological Foundations of Gestures: Motor and Semiotic Aspects", Lawrence Erlbaum Associates, Hillsdale, NJ 1986 Defines gesture [Nestor86] Nestor, Inc. "Nestor Press clipping file", Nestor, Inc. Synthetic neuron pattern-recognition technology, applied to OCR and DCR handwriting recognition [OedR86a] Oed, R. "telephone conversation of July 17, 1986", AEG Aktiengesellschaft, Forschungsinstitut Ulm, Sedanstrasse 10, D-7900 Ulm, West Germany. You get different writing forms after the first 3-4 lines out of 20 [OedR86b] Oed, R. "conversation of September, 1986", AEG Aktiengesellschaft, Forschungsinstitut Ulm, Sedanstrasse 10, D-7900 Ulm, West Germany. Gesture/symbols are modified characters: slash-M, "DF" [Ogozalek86] Ogozalek, V.Z. and Van Praag, J. "Comparison of elderly and younger users on keyboard and voice input computer-based composition tasks", in Human Factors in Computing Systems, Proceedings of CHI 83 Conference, 1983, Boston. Cited in Martin89 Martin89 cites as saying speech recognition and typing just as fast a user-interface for typing documents [OlsonB86] Olson, Bruce T. "Is your Handwriting Legible Enough for the Computer?", Journal of Law enforcement Report writing, Vol 3, No 1, December 1986 Pencept PenPad for capturing police reports using handwriting recognition [OritaM86a] Orita, M., Kanasaki, M., Toda, Y., Mishima, T., Suzuki, M., Onuma, C., and Takatoo, M. "Image extraction using density distribution characteristics", European Patent 198481, October 22, 1986, assigned to Hitachi, Limited, Tokyo, Japan. OCR scanner preprocessing to extract character images from background clutter [PCMagazine86] PC Magazine "Best of 1985 - Pencept Penpad", PC Magazine, January 14, 1986, Ziff-Davis Publishing Company Review of best PC products of 1985 -- describes Pencept PenPad with PenCad software, use of "pen macros" / letter gesture commands [PavlidisT86a] Pavlidis, Theo "A vectorizer and feature extractor for document recognition", Computer Vision, Graphics, Image Processing, Vol 35, 1986 Cited in Kahan87 [PavlidisT86b] Pavlidis, Theo "A Critical Survey of Image Analysis Methods", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 502-511 Image processing / recognition design requires many methods, not just one theoretical basis A digitizer tablet good enough for writing input [PearsonG86a] Pearson, Glenn and Weiser, Mark "Of Moles and Men: The Design of Foot Controls for Workstations", Proc. CHI '86, April 1986, pp. 333-339 Various foot controls (foot mouse, "tush-mouse", foot touchscreen, etc.). Mentions multipoint/multi-touch touchpad, using multiple-finger gestures for typing regardless of location (Shackelford) [Pencept86a] Pencept "PENWARE (tm) Pendraw (tm) II User's Guide", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 [Pencept86b] Pencept "Penpad 300 User's Guide", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 [Pencept86c] Pencept "Penpad 320 User's Manual", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 (hardcopy book) Handwriting recognition tablet: shows single-stroke and multiple-stroke characters shapes in one set. Includes system disk with mouse driver to adapt tablet input (including command templates, character recognition) to mouse/keyboard input. [Pencept86d] Pencept "Writing Samples", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 Collections of unconstrained, natural writing samples taken from hotel receipts, university bulletin board



notes, bills of sale, and cross-word puzzles from the magazines in waiting rooms and airlines from various North American and European cities, showing true variability effects [Pencept86e] Pencept "Software Toolkit for the Penpad 310 and Penpad 320", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 (hardcopy book) [Pencept86f] Pencept "Pencept PenPad with PenCAD", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 Pencept PenPad interface for CAD systems: described as unique form of digitizer tablet with handwriting/printing character recognition. [Pencept86g] Pencept "Penpad Penpad 300 (TM) Digitizing Tablet Product Information", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 Electromagnetic grid tablet, no handwriting recognition. [Pencept86h] Pencept "Introducing PenCad -- A Software Interface that puts CAD on a Pad", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 PENCAD -- CAD templates and gestures ("recognition macros") for tablet with AutoCAD , other CAD systems. [Pencept86i] Pencept "PenPad Digitizer and Intelligent Tablets -- Using Artificial Intelligence to Make Your Computer Faster and Easier to Use", Pencept Inc., 39 Green Street, Waltham, Massachusetts 02154, 1986 Pencept PenPad 300, 310, 320 electromagnetic tablets, handwriting recognition, command macros (gestures) [Pencept86j] Pencept "Penpad 310 User's Guide", Pencept Division of Numonics, 39 Green Street, Waltham, Massachusetts 02154, 1986 (hardcopy manual) Handwriting recognition tablet, compatible with CalComp, Summagraphics, etc. On-tablet templates. [PennywittKE86a] Pennywitt, Kirk E. "Robotic Tactile Sensing", BYTE magazine , 1986, p. 177-ff Overview of tactile/touch sensors robotics: pressure sensing, slip (shear) sensing, temperature sensing. In particular PVDF polyvinylidine flouride piezoelectric polymer (elastomeric) over array of circular electrodes: compare with Boie 1974 multi-touch touchpad sensor. Also 16x16 array of anisotropic elastomeric conductor line arranged in grid, each of 256 contact point could be measured separately for pressure (more pressure == greater area of contact). Multiple grid-sensing architectures, some with high resolution. [Perex86a] Perex Ltd "Multipad from Perex", Perex Ltd., Reading, UK, 1986 Product literature on Perex Multipad Model 7500 portable (and desktop) touchpad digitizer data entry station: pre-printed forms with data entry icons (or virtual keyboard) recognized by bar code, use inputs data by taping/touching on form. Paper forms then usable as back-up physical record of data. One-line 40/80 character display. Compare with DataPad, which also had handwriting recognition. [PetersonJL86a] Peterson, James L. "A note on undetected typing errors", CACM, Vol 29, 1986, pp 633-637 spelling correction using word lists compiled from published dictionaries, how to detect miss-typed/spelled word is another known word. Gives statistics for undetected errors, but a bit vague on how correctness was specified. [Pickering86] Pickering, J.A. "Touch-sensitive screens: the technologies and their applications", International Journal of Man-Machine Studies, Vol 25, 1986, pp 249-269 Tutorial/review of tablet digitizer/touchscreen technologies that work with a finger touch, integrated with a display. Parallax error in touch-screen model. Capacitive, SAW acoustic surface wave, conductive/resistive film, infra-red optical (using shadow interpolation), surface reflection, strain gauge mounted overlay, velocity-based piezoelectric touch sensing. Mentions Knowlton visual keyboard as touch-input system. Eye dominance (parallax?), finger obscuring, use of stylus. Abstract device mapping (virtual tablets / virtual devices). Describes limitation of not detecting simultaneous multiple touches (multi-touch). Mack89 on tablet technology: [PlamondonR86a] Plamondon, R. and Baron, R. "On-line Recognition of Handprint Schematic Pseudocode for Automatic Fortran Code Generation", Proceedings of the 8th International Conference on Pattern Recognition, October 27-31, 1986, Paris, pp 741-744 Graphical program editing user-interface using DCR [PlamondonR86b] Plamondon, R. and Baron, R. "A dedicated microcomputer for handwritten interaction with a software tool: system prototyping", Journal of Microcomputer Applications, Vol 9, 1986, pp 51-60 On-line character recognition for graphical software tool User-interface goal: must interface to application without changing application Too much feedback distracts the user from the writing task [Postl86] Postl, W. "Detection of Linear Oblique Structures and Skew Scan in Digitizer Documents", Proceedings of 8th International Conference on Pattern Recognition, Paris, France October 1986, pp 687-689 Cited in Biard86a [PoujoisR86a] Poujois, Robert "Capacitive Keyboard Structure", United States Patent 4,614,937, 1986 touch-sensitive (proximity) capacitive keyboard, wiring on opposite sides of substrate, describes matrix wiring of keys as to reduce connections: multi-touch



[Quest86a] Quest Automation "Datapad puts your pen on-line to a computer", Quest Automation 1986 Handprinting recognition data entry terminal: uses two layers of conductive/resistive film, stylus presses together, floating handrest to keep hand from pressing on touch tablet. 40-character LED display. Application show user identifying document by writting in ID number, along with data. [RadioElectronics86] Radio Electronics "Equipment Report - Pencept Penpad 320", Radio Electronics, January 1987, pp 22 ff What a nice thing Pencept recognition macros (gestures) are for a user-interface [ReiffelL86a] Reiffel, Leonard: Jung, Wayne D. "System for Sensing Spatial Coordinates", United States Patent 4,603,231, July 29, 1986 capacitive/resistive sheet tablet, stylus is receiver. Generate AC voltage across conductive (resistive) sheet in X and in Y: measure strength of signal with gain correction for height gain correction used to determine Z proximity as well. Can run X and Y signals simultaneously if different frequencies, sheet may have nonuniform resistance with correction map. Can use more than one sheet, can be any number of surfaces. Gain for height is determined sum of signals measured with voltage from left-right and right-left. Compare with Turner paper on electric field sensing from 1970. Compare with ScripTel at Wang. [Rhyne86] Rhyne, J.R. and Wolf, C.G. "Gestural Interfaces for Information Processing Applications", IBM Research Report RC 12179 (Log #54544) 9/2/86, 1986, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598 Description of IBM's early informal study on what editing marks subjects use on spreadsheets Report on IBM's prototype gesture/handwriting Lotus interface Early description of IBM's electronic ink hardware tablet and display Refers to "gesture language" for the qualitatively similar set of symbol marks and operations subjects use to edit spreadsheets Refers to gesture language, combining gesture (pointing) and menu selection, gesture (command) and handwriting (data) Accuracy of recognition has great effect on user interface (need to confirm, not just correct) Recognition errors/accuracy: user assumes his or her writing was correct, so errors and means to correct must be very obvious For a small gesture symbol set, intra-user variability was found to be small For a small gesture symbol set, inter-user variability was usually three or so "qualitatively similar" (ambiguous/confusable) symbols IBM had no answer on reconciling small display character size with larger handwritten character size for electronic ink User interface for electronic ink: enlarge area of interest, let user write handwritten characters normal size on enlarged image Optical/visual parallax, also ranging parallax (pen tilt) Pen tilt error of 0.06" tablet digitizer accuracy led to pointing errors and slow user interaction rate effects on user interface Handwriting substantially worse on slick glass surface than on paper for electronic ink, much lower recognition accuracy User interface problems for handwriting are big: no obvious ideas on how to solve [RichmondD86a] Richmond, Dana "Computer Terminal", United States Design Patent 283,026, March 18, 1986 Design of PenCept PenPad 200 handwriting terminal, showing digitizing tablet in front of a VT-100 like display with no keyboard / keyboardless [RothfjellRE86a] Rothfjell, Rolf E. "Method and Device for Signature Verification", United States Patent 4,581,482, April 8, 1986, assigned to Esselte Security Systems AB, Stockholm, Sweden. Signature verification using a constrained writing form to get alignment points for segmentation (!) [SabourinR86a] Sabourin, R. and Plamondon, R. "Preprocessing of Handwritten Signatures from Image Gradient Analysis", Proceedings of the 8th International Conference on Pattern Recognition, October 27-31, 1986, Paris, pp 576-579 Static (OCR) signature verification system: test protocol included no forgery attempts [ScheiflerRW86a] Scheifler, Robert W. and Gettys, Jim "The X Window System", ACM Trans. on Graphics, Vol 5 No 2, April 1986, pp 79-109. Also as Internal Report, Laboratory for Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, October 1986 Description of the X-Window graphical user interface system. Pointing input devices include touchscreens, tablets, lightpens. See Gettys later work on RandR rotating windows extension in X windows. Transparent windows (drawn in parent window, so a sub-window?), but only as fully transparent and thus a clipping region for drawing: says other mechanisms usually better, no details. Brief mention of mouse/tablet/joystick/etc. input. Device independence without in GKS-style query for capabilities of devices. [SchellRR86a] Schell, Roger R. and Denning, Dorthy E. "Integrity in trusted database systems", Proc. 9th DoD/NBS Computer Security Conference, Gaithersburg, MD, September 1986 Discussion of DoD Trusted Computer System Evaluation Criteria (DOD 85). Trusted computer system, not only about secrecy (most security literature) but also integrity (control of unauthorized update/modification). Integrity: accuracy/correctness/validity. [SchomakerL86a] Schomaker, L.R.B. and Thomassen, A.J.W.M. "On the Use and Limitation of Averaging Handwriting Signals", in in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (North-Holland), 1986, pp 225-238



Low-pass average filter on digitizer data so smoothed, but with anchored endpoints where velocity is zero Describes anchoring inflection (zero velocity) points on handwritten characters as similar to dynamic time warping for speech recognition Time-normalization of cursive handwriting; Force/pressure sensitive pen used in handwriting research; Capacitive coupling, resistive-sheet digitizer tablet [SchultzJ86a] Schultz, Jan "A history of the Promis technology: an effective human interface", Proc. ACM Conf. on The history of personal workstations, 1986, pp 159-182 Promis: Early Digiscribe SAW touch-screen user interface system for medical records, 1967, user interface mostly point-and-click [ScriptWriter86a] Beasely, Richard "ScriptWriter personal correspondense, Tom Beasely of Data Entry Systems regarding PHIGS standards", Personal correspondence, 1986 Discussion of joint presentation to PHIGS graphical standards body concerning multi-modal input from handwriting recognition tablets from PenCept, CIC: gesture input with associated gesture/text character and positional information, compare wit multi-touch. Peter Bono presentation to PHIGS body. [Scriptel86] Scriptel "SPC-Series Technical Report", Scriptel Corporation, 4145 Arlingate Plaza, Columbus, OH 43228, May, 1986 Capacitive/electrostatic sheet digitizer tablet [SearbyAD86a] Searby, A.D. and Bowman, D.W. "Stylus Devices Responsive to Pressure Changes for Use in Videographic and Like Apparatus", United States Patent 4,580,007, April 1, 1986, assigned to Quantel, Limited, Surrey, England. Force/pressure-sensitive digitizer stylus with improved life and performance [SearbyAD86b] Searby, Anthony D. "Video Processing System for Picture Rotation", United States Patent 4,611,232, September 9, 1986 Rotation of digitized video image (in frame buffer) by bit-shift operations in real time. [Sejnowski86] Sejnowski, Terrence, J. and Rosenberg, Charles R. "NETtalk: A Parallel Network that Learns to Read Aloud", Technical Report JHU/EECS-86/01, John Hopkins University, Baltimore, MD, 1986 Hinton diagrams; training input consisted of phonetic transcriptions of English text, not actual spoken recording. [Shneiderman86a] Shneiderman, Ben "Human-Computer Interaction Research at the University of Maryland", SIGCHI Bulletin, January 1986, Vol 17 no 3, pp. 27-32 Published version of 1985 manuscript. User-interface studies on menu structure: globally visible menu helps User-interface: arrow keys for pointing vs mouse User-interface: manuals are better than on-line help (why?) User-interface: user preference for touch-screen pointing tablet/display over keyboard [Shridhar86] Shridhar, Am. and Badreldin, A. "Recognition of Isolated and Simply Connected Handwritten Numerals", Pattern Recognition, Vol 19 No 1, pp 1-12, 1986 Refers to stroke connection within and between characters [SibertJL86a] Sibert, John L.; Hurley, William D.; and Bleser, Teresa W. "An object-oriented user interface management system", Proc. SIGGRAPH '86, pp. 259-268 GWUIMS George Washington University User InterMace Management System: early object-based UIMS, message passing as buffered messages (?) rather than as method calls: virtual buttons, direct manipulation. Touchscreens tablets and mouse, character recognition (gesture recognition: compare early Alan Kay video). Franz LISP on Sun Microsystems workstation. [Signify86] Signify Inc. "Sign-On product information", 9005 Red Branch Road, Columbia Maryland 21045, 1986 (see also ATI). Signature verification product: now called ASI/Autosig Systems Inc.. [Srinivasan86] Srinivasan, S., Palaniswamy, K., and Natarajan, A.E. "Machine Recognition of the Indian Language Characters Using a Tree-structure Based On Primitives", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 726-728 [StefikM86a] Stefik, M; Bobrow, D.G.; Lanning, S; Tatar, D; and Foster, G. "WYSIWIS Revised: Early Experience with Multi-User Interfaces", Proc. CSCW '86, December 3-5, 1986, Austin Texas, pp. 276-290 Principles of whiteboard/chalkboard/blackboard systems: What You See Is What I See. Boardnoter freestyle sketching, erase gesture (not a shape) selects whole object not just pixels; with multiple users, visual clutter of windows a problem in the user interface, rooms and doors instead. Multi-touch? [Suen86] Suen, C.Y. "Human Recognition of Handprinted Characters and Distance Measurements", in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (North-Holland), 1986, pp 213-224 Needed 29x39 resolution to get decent handwriting recognition Simple (not serious) recognizer algorithm using pixel template matching, then entropy, information content, and nearest neighbor distances: did not correlate to human recognition Human use multiple distance measurements (multiple features) to recognize characters Says entropy, hamming distance, linear and cross correlation, centre of gravity, Mahalonobis distance of template matching were all poor distance functions



[Summagraphics86a] Summagraphics "Bit Pad Two Technical Reference", Summagraphics Corporation, August 1986 User's manual for Summagraphics Bit Pad Two (magnetostrictive?) tablet digitizer. Described proximity /out-of-prox position sensing, absolute coordinate mode and relative (mouse) mode. Both stylus and puck, stylus is an inking pen stylus. [Summagraphics86b] Summagraphics "SummaSketch: The world's best selling graphics Tablet", Summagraphics Corporation, 1986 SummaSketch: 6x9, 12x12, 12x18 line of electromagnetic tablets. Puck ("cursor") or stylus with side button, target is drawing market. [Suydam86] Suydam, W.E. "Approaches to Software Testing Embroiled in Debate", Computer Design, Vol 25 No 21, November 15, 1986, pp 49-55 Long, informal review article on software testing and reliability David Parnas: Only actual use will bring out the mistakes (not testing) Parnas: need for regression testing Parnas: problem of classical testing is unweighted likelihood of failure Parnas: testing must consider seriousness of different types of failure Parnas: testing must differentiate between important errors and insignificant errors Parnas: (Titanic effect) software design implements a mental model of real world: must test the model AI testing: people think it's different from software testing: it is not AI testing: when mechanical engineering was a new as AI, had not invented the right angle yet Dijkstra: testing shows presence of bugs, not absence Testing: must distinguish between active failure (real bug) vs passive (design omission) Testing: only field testing finds "real" errors Testing: unanticipated data can seriously screw a program [SwinehartDC86a] Swinehard, Daniel C.; Zellweger, Richard J. Beach; and Hagmann, Robert D. "A Structural View of the Cedar Programming Environment", ACM Trans. on Prog. Languages and Systems, Vol 8 No 4, October 1986, pp 419-490 Cedar OS at Xerox PARC. Cedar language is prototype-based (?) object-oriented language, garbage collection / automatic storage management, reference types instead of pointers, tiled window management. Graphics tablet and mouse both supported: single serial buffer of time-stamped input events. Mentions windows/objects with transparent colors (alpha blending), complains about the lack of transparent color support on system. [TagushiY86a] Tagushi, Y., and Yamanami, T. "Position Detecting Apparatus", United States Patent 4,617,515, October 14, 1986, assigned to Wacom Company, Limited, Japan. Cordless stylus digitizer tablet with magnets in stylus over a grid of wire loops [TanakaT86a] Tanaka, Toshinori, and Kobayashi, Shunsuke "Entry of Data and Command for an LCD by Direct Touch: An Integrated LCD Panel", SID 86 Digest of Technical Papers, Society for Information Display, 1986, pp 318-320 Touch-entry-device integrated tablet/display using deformation from touch force/pressure on LCD pixel cells: pixels degrade, but refreshing fixes them. Capacitance between two grid layers of ITO on top and bottom of LCD pixels. [TanieK86a] Tanie, K.; Komoriya, K.; Kaneko, M.; Tachi, S.; and Fujikawa, A. "A High Resolution Tactile Sensor", In "Robot Sensors", Vol 2. Tactile and non-vision sensors pp. 189-198, Springer Verlag, 1986 Touch sensor (touchpad) using array of phototransistors detecting light changes from total internal reflection: compare with FTIR by Jeff Han. Detects pressure distribution / 3D profile: multi-touch. Cites to Boie 1984. [TannerPP86a] Tanner, Peter P.; MacKay, Stephen A.; Stewart, Darline A.; and Wein, Marceli "A Multitasking Switchboard Approach to User Interface management", Proc. SIGGRAPH '86, Vol 20, No 4; pp. 241-248 User interface management system: refers to multiple virtual devices implemented on a tablet, applications need not know that the devices are virtual. [TanzawaS86] Tanzawa, Setsu "Recording pen", Japanese Patent Application Number S60-88861 Light-sensing optical stylus digitizer for handwriting input, using X and Y lines in different colors. Compare with Anoto. [TeulingsH86a] Teulings, H., Thomassen, A.J.W.M., van Galen, G.P. "Invariants in Handwriting: The Information Contained in a Motor Program", in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (North-Holland), 1986, pp 305-315 Spatial characteristics are more invariant in handwriting than time or force characteristics (? for signature verification?) But because handwriting is "efficient", time characteristics in handwriting are invariant on trained sequences, like signature verification [TeulingsH86b] Teulings, H., Mullins, P.A., and Stelmach, G.E. "The Elementary Units of Programming Handwriting", in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (NorthHolland), 1986, pp 21-32 Handwriting data low-pass filtered from 16 Hz to 48 Hz frequency There is no one, single grapheme unit of handwriting: units depend on what the subject is writing



[TitusJ86a] Titus, Jon "Technology Update: Digitizing tablets offer choices of formats, operating modes, and pointers", EDN Electronics Design News, April 17, 1986, pp. 69-74 Report on digitizing tablets: mentions dead zone / minimum drag distance (press and hold?), mechanical joint digitizers, sonic 2D and 3D tablets/digitizers, formats. Stylus/puck referred to as "pointer". Vendors include Hewlett-Packard HP, SAC Science Accessories, Scriptel, Summagraphics, Tectronix, Numonics, Hitachi, Aristo Graphics, Altek, Calcomp, GTCO. Stream, point, poll, switched-stream modes. [TognazziniB86] Tognazzini, Bruce "You Had To Be There", report on User Interface Reference Model workshop at SIGCHI '86 conference, SIGCHI Bulletin, Vol 18 No 2, October 1986, pp 21-24 Uses example of "graphical ambiguity/context" in recognition when discussing user-interface issues [Treisman86] Treisman, A. "Feature and Objects in Visual Processing", Scientific American, November 1986, pp 114B-125 Visual recognition consists of low-level features (automatic), then identification Low-level perception is lines, spots, edges, locations, orientation Properties/features conjoined only at higher level of human recognition (context): "distractors" delay human recognition [VanGalen86] Van Galen, G.P., Meulenbroek, R.G.J., Hylkema, H. "On the Simultaneous Processing of Words, Letters and Strokes in Handwriting: Evidence for a Mixed Linear and Parallel Model", in "Graphonomics: Contemporary Research in Handwriting", Elsevier Science Publishers B.V. (North-Holland), 1986, pp 5-20 Parsed characters in script handwriting into character segments at velocity zeros (sharp corners) Gives maximum velocity of handwriting in cursive script as 11 cm/sec Alternated turning direction vs continuous loops has big impact on handwriting velocity (of course, silly!) Beginnings of words written at slower speed [Wakahara86] Wakahara, Toru "personal correspondence", July 28, 1986 Personal correspondence, ambiguous pairs of characters in Japanese Katakana, Kanji. Transparent Electromagnetic digitizer in front (?) of flat display. [Ward86] Ward, J. "Method and Apparatus for Removing Noise at the Ends of a Stroke Caused by Retracing", United States Patent 4,608,658, August 26, 1986, assigned to Pencept Re-trace elimination to pre-process handwritten character shapes before recognition [Watanabe86] Watanabe, Y. "letter to J.Ward", Author's address: Terebijon Gakkaishi, Hachinohe Institute of Technology, Department of Engineering, Hachinohe, Japan, 25 November, 1986 Mentions co-work between Watanabe and Jiro Gyoba Interchange of looping and cusping is seen frequently in writing Japanese Kanji characters written to OCR standard are LESS human-recognizable than "sloppy" writing At 300-500 msec/stroke, writing time for Kanji Japanese slower than Roman Constraints: Japanese have characters they can read, but do not know how to write Japanese writers do not conform to formal writing constraints in practice [Websters86] Merriam-Webster Inc "Moon Type", Webster's Third New International Dictionary of the English Language Unabridged, Copyright 1986, ISBN 0-97779-201-1 Show Moon Type, a simplified single-stroke / Unistroke alphabet used as embossed printing for the blind, popular before Braille became dominant [Wikipedia86a] Wikiepedia "Macintosh Programmer's Workshop", Wikipedia.org, fetched April 2010 Macintosh Programmer's Workshop MPW SDK/IDE, 1984..1986, supplanted by Project Builder IDE which became Xcode. SDK, not sure what it has for building applications: describes a special version of MAKE. [Wolf86a] Wolf, C.G. "Can People Use Gesture Commands?", SIGCHI Bulletin, Vol 18 No 2, pp 73-74 Gesture input - user interface. IBM simple usability study (18 subjects) for "move" gesture consisting of an arrow to indicate what gets moved to where, with different gestures (lasso, bracket, underline) to indicate selection. Describes editing gestures, such as cross out gesture to delete a word, for stylus-based devices. [Wolf86b] Wolf, C.G. "Can People Use Gesture Commands?", IBM Research Report FC 11867 (#53105), 4/17/86 Gesture input - user interface. IBM simple usability study (18 subjects) for "move" gesture consisting of an arrow to indicate what gets moved to where, with different gestures (lasso, bracket, underline) to indicate selection. Describes editing gestures, such as cross out gesture to delete a word, for stylus-based devices. [WuL86] Wu, L. and Weng, F. "Chain Code for a Line Segment and Formal Language", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 1124-1127 Some properties of chain codes (straight line theorem) very hard to test [YamamotoK86a] Yamamoto, K., Yamada, H., Saito, T. and Sakaga, I. "Recognition of Handprinted Characters in the First Level of JIS Chinese Characters", Proceedings of the 8th International Conference on Pattern Recognition, Paris, October 1986, pp 570-572 Relaxation method with 98.6% success on Kanji/Chinese characters Testing with ETL-8 and ETL-9 "standard" Chinese writing samples in Japan ETL-9 standard Kanji Chinese samples is 600000 samples of 3036 characters [YamamotoK86b] Yamamoto, K. and Saito, T. "Pattern Reading System", United States Patent 4,566,124, January 21, 1986, assigned to Agency of Industrial Science and Technology, Ministry of International Trade and Industry, Tokyo, Japan.



Handwriting character recognition by tracing contours, local extrema/outermost points, chain codes Template matching on direction chain codes, polygonal approximation to pattern for OCR handwriting recognition [Yau86] Yau, S. S. and Tsai, J.J.-P. "A Survey of Software Design Techniques", IEEE Trans. Software Engineering, Vol SE-12 No 6, June 1986, pp 713-721 General survey article on software engineering biblio85 (Prev page)

Main Page







Main Page


[AEG87] AEG Aktiengesellschaft "Polyform: The Future Data Entry Workstation", product literature, AEG Information Systems, Postfach 2154, D-7750 Konstanz, West Germany, 1987 AEG's OCR product for typed and handwritten character recognition [AbramsMD87a] Abrams, Marshall D. and Jeng, Albert B. "Network Security Protocol Reference Model and the Trusted Computer System Evaluation Criteria", IEEE Network Magazine, Vol 1 No 2, April 1987 pp. 24-33 Review of applying "trusted system" standards by DoD (Red Book, 1985) to network security. Cites to other on (mathematically-defined) formal security models. Authentication/identification of all network entities, Mandatory access controls based on sensitivity of information, discretionary access control like ACLs, Communications integrety, Service availbility, accountability (logging). [AdlerR87a] Adler, Robert "Touch control system for use with or having a three-dimensionally curved surface", United States Patent 4,642,423, February 10, 1987 Sonic/acoustic wave digitizer/touchscreen for curved monitors: takes into account great-circle distance across curved plate, sonic transducers only at corners. Several references to Raleigh wave, elastic waves, infrared light-beam touch screens. Uses regular faceplate of monitor. [Adobe87a] Adobe "Adobe Illustrator User's Manual Ver. 1.0", Adobe, Inc., Jan 1987 Includes GUI with graphical elements. Gestures/Pop-up menu or keyboard: Cited in "Issues for locationindependent interfaces" [AhmedP87a] Ahmed, pervez and Suen, C.Y. "Computer Recognition of Totally Unconstrained Handwritten ZIP Codes", Int'l. Jnl of Pattern Recognition and Artificial Intelligence, Vol 1 No 1, 1987 Static/OCR recognition of handwritten zip codes. Segmentation as separate step before and independent of feature extraction. Primary features are end point direction, holes and enclosed curves. Tates 84% to 98% correction rate, but does not state whether per-character or per-zipcode, or false positive vs. rejects. [Anatex87] Anatex "Personal Writer Product Description", available from Anatex S.A., Xavier Maury, President, 18 Rue Troyen, F-75017 Paris, France, 1987 (see also "Personal Writer"). Adaptive handwriting recognition product from Anatex, France. In training procedure, if certain characters too similar (e, l, m n), tells user to change writing style. See also PersonalWriter89: company changed



name. [AndersonPR87a] Anderson, Patrick R. and Winfree, L. Thomas jr., eds. "Expert Witnesses: Criminologists in the Courtroom", SUNY Press, 1987 (hardcopy book) Collection of sociological papers on the experiences and roles of expert witnesses in criminal cases (not directly on the legal aspects of expert witnessing) [ArnautLY88] Arnaut, Lynn Y and Greenstein, Joel S. "Human factors considerations in the design and selection for computer input devices", Computer graphics: technology and applications, Academic Press, Boston, 1988 ISBN 0126399700 touchscreens: capacitive, resistive, crossed-wires, acoustic (using same elements for sound generation and detection): force (pressure, torque, and X/Y values from strain gauges at sides of touchscreen. Shows virtual slide and on/off switches circa 1985 on touchscreen. Touch tablets: elographics with stylus. Cites to Buxton for virtual input devices on a tablet. [Asbo87] Asbo, E.L., and Tichenor, H. "Method and Apparatus for Dynamic Signature Verification", United States Patent 4,646,351, February 24, 1987, assigned to Visa International Service Association, San Mateo, California. Signature verification using tip force (a.k.a. pressure), and length and angle of segments [AuerCM88a] Auer, Carol M.; Castagno, Daniel L.; Haley, Allen W. Jur.; Moore, Harry H. IV; O'Leary, Sean E.; Paley, Steven J; and Rutt, Thomas E. "Computer Interface Device", United States Patent 4,725,694, February 16, 1988 Slate-format/tablet-format touchscreen/pen computer with simulated keyboard on display. Keyboard/keypad layout, and labeling/function of keys can be changed dynamically. Return key removes virtual keyboard from display. Compare with Linus WriteTop? [BTG87] British Technology Group "Handwriting encoder and software for automatic signature verification", "Handwriting encoder may provide the access key for future systems users", Report CR 129827, 101 Newington Causeway, London SE1 6BU, 1987 Combined pen and smart-card in 1987: invented by Colin Hilton, inventor: British Technology Group advisor: Dr. John Parks. Handwriting input/recognition system available for license from England [Baird87] Baird, H.S. "Feature Identification for Hybrid Structural/Statistical Pattern Classification", Laboratory report, 24-Feb-1987. Author's address: AT&T Bell Laboratories, 2C-557, Murray Hill, NJ 07974 Reports on OCR system (with Pavlidis) Training speed "very slow" on a statistical OCR recognition system Adaptive recognition: including "unrecognizable" characters (too small) aided training on normal size OCR Adaptive recognition: very large-scale trials necessary to determine actual performance Statistical recognition: in practice, "hand-crafted" (cognitive/functional) rules will always be needed as well Good 99%+ OCR results, but only for "excellent quality" input [Baptista88] Baptista, G. and Kulkarni, K.M. "A High Accuracy Algorithm for Recognition of Handwritten Numerals", Pattern Recognition, Vol 21 No 4, 1988, pp 287-291 Multi-layered system syntactic and deterministic (feature table driven) approach to handwritten O.C.R. numeral recognition based on human visual cortex Criticizes syntactic character recognition: says it is unwieldy, and cannot be used in a trainable system Diagram with heirarchy of human visual processing, from cells to brain stem to relations O.C.R. using noise removal, thinning, internal segmentation (like chaincodes) [BarkerBA88] Barker, B.A., Hernandez, I.H., and Machart, B.H. "Integrated multiple data editor", United States Patent 4,723,209, February 2, 1988, assigned to IBM Corporation. Integrated multiple data editor uses super-block structure of objects sets and flow attributes for text object sets [BierEA88a] Bier, Eric A. "Snap-Dragging: Interactive Geometric Design in Two and Three Dimensions", UC Berkeley Report UCB/CSD-88-416, May 1988 Snap to grid vs. heuristic constraints in three dimensions 3-D similar to compass and ruler for pointing and positioning, in technical illustration system. Slope, angle, distance constraints for lines, circles, planes and spheres (alignment object) specified by user. [Blatt88] Blatt, Louis "Character Recognition and Added Value", internal memorandum, Wang Laboratories, September 27, 1988 Review of user-interface human-factors ergonomics advantages/disadvantages in a portable handwriting recognition product: just a rehash of Wolf87 on gestures. User interface: ease of learning may be more important than ease of use. [Blesser87] Blesser, B. "Position Indicating Apparatus for Use in a Digitizing Tablet System", United States Patent 4,638,119, January 20, 1987, assigned to Pencept, Incorporated, Waltham, Massachusetts. Pencept digitizer/tablet patent: two-coil pen stylus, side button, force (a.k.a. pressure) transducer [Blesser87a] Blesser, B., and Prentice, T. "Digitizing Tablet System", United States Patent 4,644,102, February 17, 1987, assigned to Pencept, Incorporated, Waltham, Massachusetts. Pencept digitizer patent



[Bokser88a] Bokser, Mindy R. "Pattern Classification Means for Use in a Pattern Recognition System", United States Patent 4,773,099, September 20, 1988 1988, assigned to The Palantir Corporation, Santa Clara, California. Pattern recognition (for characters) using a set of "ringed zones" in feature space to define boundary space and confidence probability Contains interesting diagram for a discussion of one character overlapping/invading the feature space boundary of another. [Bokser88b] Bokser, Mindy R. "Means for Resolving Ambiguities in Text Based Upon Character Context", United States Patent 4,754,489, June 28, 1988, assigned to The Palantir Corporation, Santa Clara, California. Context in character recognition similar to di-grams, tri-grams, n-grams: aggregate probability score of all character matched against a dictionary of words [Booth87] Booth, D.S., Bryden, M.P., Cowan, W.B., Morgan, M.F., and Plante, B.L. "On the Parameters of Human Visual Performance: an Investigation of the Benefits of Antialiasing", unpublished manuscript, 1987. Authors' address: Computer Graphics Laboratory, University of Waterloo, Waterloo, Ontario Canada N2L 3G1 Quotes Allen Newell: engineering understanding in user-interfaces and human factors more important than psychological mechanisms [Borland88a] Borland International "SideKick Plus for SideKick Users: The Professional Desktop Manager", Borland International, 1988 Introduction to features of SideKick Plus over SideKick. Text editor, Time planner/calendar, telephone dialer for DOS desktop. Enhanced version of SideKick. Character-mode windows. Screen dialing: Can search display screen for telephone numbers. [Borland88b] Borland International "SideKick Plus: The Professional Desktop Manager -- Owner's Handbook", Borland International, 1988 User manual/guide for Sidekick Plus desktop PIM personal information manager for DOS. Phonebook database with automatic dialing. Quick-dial dials numbers at cursor position on screen. Phonebook automatically searches screen for phnumber: regular Sidekick does this by default. (page 172) [BrassLB88] Brass, Robert L.; Glaberson, John; Mason, Richard W.; Santulli, Scott; Roth, G. Thomas; Feero, William M., and Blalaska, Richard K. Jr. "Method and Apparatus for Transforming Digitally Encoded Data into Printed Data Strips", United States Patent 4,728,783, March 1, 1988, assigned to Cauzin Systems, Incorporated Two-dimensional bar code printed with matrix printer, strip data can be arbitrarily long. Contrast to Sekendur patterns [Brault87] Brault, Jean-Jules and Plamondon, Rejean "Handwritten Curve Partitioning Based on Geometric and Sequential Information", Proceedings of the Third International Symposium on Handwriting and Computer Applications, Ecole Polytechnique de Montreal, Canada, 1987, pp 50-52 Segmentation of handwriting using local extrema / points of high curvature [BrittsS87] Britts, Stefan "Dialog Management in Interactive Systems: A Comparative Survey", SIGCHI Bulletin, Vol 18 No 3, January 1987, pp 30-42 Cites to 1986 Ward paper as example that UI / Dialog management systems for user interfaces do not accommodate new types of devices, such as handwriting recognition input. [Burgess87] Burgess, Ken L. "Digitizing Method and Apparatus for Compensating Cursor of Stylus Velocity Errors", United States Patent 4,686,331, August 11, 1987 Correction for bowing due to velocity on digitizing tablet using double alternating X and Y measurements [Buxton87a] Buxton, W. "Research Frontiers and Unsolved Problems", in Readings in Human-Computer Interaction: a Multidisciplinary Approach, Chapter 14, 1987 Gesture and character recognition: shorthand gesture user-interface for music. [Buxton87b] Buxton, William and Kurtenbach, Gordon "Editing by Contiguous Gesture: A Toy Test Bed", Computer Systems Research Institute, University of Toronto, Toronto, Ontario Canada M5S 1A4, 1987 Suggested symbols for gesture recognition operations: lasso, angles, directional strokes, single-line crossout. Combine lasso-drag-"C" gestures in one combined mark. Double lasso for selection with exceptions. [CCIT88a] CCITT International Telegraph and Telephone Consultative Committee "Draft Recommendation x.509: The Directory-Authentication Framework", CCITT International Telecommunication Union, November 1988 Draft standard for CAs certification authorities from ITU: includes introduction to RSA public key cryptosystems, defnition of PKI certificates. [CIC88a] CIC "Handwriter (R) ProFicient (R) AutoCad Enhancer User's Guide / Proficient Series Software Drivers for AutoCad, Synthesis, NC Programmer, etc.", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Gesture-like handwriting user interface for office automation and data entry from CIC. Compare with PenCAD offering by PenCept, Ledeen Recognizer. [CIC88b] CIC "Handwriter (R) WRITE-ON (R) Series Software Drivers for Microsoft Windows, WordPerfect, Pagemaker, etc.", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Gesture-like handwriting forms data entry from CIC Tap different control areas to shift from upper to



lower, etc. [CIC88c] CIC "Handwriter (R) Customization tools: FormManager (TM), FormMapper (TM), etc.", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Signature verification software from CIC [CIC88d] CIC "Handwriter (R) On-line Dynamic Signature Verification (TM)", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 [CIC88e] CIC "Handwriter (R) Data Entry System, Layout Design System (TM) Manual Version 1.1, May 1988(TM)", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Forms-design package for handwriting, using handwriting: separate writing areas for upper and lower case, numerics, Command templates (on-tablet menus), etc. File includes product description materials: Layout Design System for Customized Computer Application in Handwriter Series [CIC88f] CIC "Handwriter (R) Data Entry System User's Guide(TM)", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Shows recognition macros / gestures, special shape for erase (inverted U): features copied from PenPad product [CIC88g] CIC "Handwriter (R) Write-On(TM) Form Manager User's Guide Version 1.0", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 User guide / installation guide for CIC HandWriter forms application software: shows command and digitizer input areas on tablet for forms-layout design program using HandWriter tablet with handwriting recognition. Field types/context in recognition: check box, date, fixed point, integer alphanumeric, etc. Command-line interface. [CIC88h] CIC "Proficient AutoCAD Enhancer", Communication Intelligence Corporation, 333 Ravenswood Avenue PN 257, Menlo Park, CA 94025, 1988 Template for AutoCAD on DOS/PC using handwriting recognition and pre-defined menus on the tablet, external to the application. Compare with Pencept CAD products. Electronics board noted as working with multiple third-party digitizer tablets. [CLCS88] Chinese Language Computer Society "Proceedings of 1988 International Conference on Computer Processing of Chinese and Oriental Languages", August 29 - September 1, 1988, Toronto, Canada. Handwritten and speech Chinese, Japanese/Kanji recognition [CallahanJ88a] Callahan, Jack; Hopkins, Don; Weiser, Mark; and Shneiderman, Ben "An Empirical comparison of Pie vs. Linear Menus", Proc. CHI '88, pp. 95-100 Pop-up pie menus: permits multiple motions, action is selected on mouse-up / end-of-stroke to confirm correct choice. Other references cite for T-Cube pie-menu unistroke handwriting (recognition) input from Apple. Press-and-hold (press-and-hold-still), pressing and holding with any of three mouse buttons to invoke pop-up menu. [CardelliL88a] Cardelli, Luca "Building User Interfaces by Direct Manipulation", Proc. UIST '88, pp. 152-166 Dialog editor for creating graphical UI user interfaces. Refers to buttons for rotations and reflections of bitmap image being constructed for an icon/cursor. [Casio87a] Casio "Casio PF-8000 Calculator (IF-8000)", Collection from Pocket Computing Museum, http://cdecas.free.fr/computers/pocket/museum.htm, 2002 1984 Calculator (this reference says 1987) with touchscreen input, and also zone-based handwriting recognition on a touch-film keyboard/keypad Same reference includes Kyocera Refalo (1991) Same reference includes Sony Magic LInk, General Magic OS Same reference includes EO Personal Communicator, 1994, PenPoint OS from GO Same reference includes Simon from IBM 1994: Phone built into a portable computer Same reference includes Nokia 900 communicator: fax/phone/browser Casio IF8000 and PF-8000: same product? [Casio87b] Casio "Casio Handhelds Product Information (in Japanese)", http://member.nify.ne.jp/m_sai/museum/pda.html and other sources, fetched 2002 Information on Nokia and other handhelds: Sharp PA-8500 (1988), Casio PF-8000 with handwriting recognition touchscreen/touchpanel (1984), Casio DK-5000 full keyboard (1990), HP 95LX full keyboard (1991), HP 100 LX, HP 200 LX, IBM Palm Top PC 110 (with touchpanel?) 1996, IBM ChipCard TC-100 credit card format (1995), Sharp PA-M1 (1996), NTT DoCoMo Pocketboard (1998), NEC Mobile Gear MC-P1 (with Pen/GEOS?), Refalo, Casio PB-100, CVB-1000 touchscreen organizer watch, Casio IF-8000 touchscreen organizer with electronic ink on touchscreen, Casio NX-6000 / Planeo PDA, Sharp IQ-9200 with touchscreen PDA/organizer, Casio IT-2000, [CaswellNS88a] Caswell, N.S. "Introduction to input devices", Chapter in "Input Devices", Sol Sherr Ed. Academic Press, Boston, 1988 ISBN 0126399700 (electronic book) Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets Section "Digitizers and input tablets" by Thomas E. Davies, Gerard Mathews, and Paul D. Smith Coordinate transforms for



calibration on digitizer tablet. Active surface tablets vs. Touch Screens. Refers to rudimentary touch screen composed of switches. Cites problems of acoustic, voltage gradient (resistive film), acoustic touch, multilayer matrix-encoded digitizer tablets. Touchpad integrated with keyboard. Comparison of mouse and tablet. see also Sherr88: check Barker Engineering Library. multi-touch capable? [CaswellNS88b] Caswell, N.S. "Introduction to input devices", Chapter in "Input Devices", Sol Sherr Ed. Academic Press, Boston, 1988 ISBN 0126399700 (electronic book) (See also SherrS88a) [ChiaoYueh88] Chiao-Yueh, Lin "Method and Means for Automatically Coding and Inputting Chinese Characters in Digital Computers", United States Patent 4,758,979, July 19, 1988 Kanji/Chinese character recognition using a special digitizer stylus Gesture-like handwriting user interface for AutoCad from CIC [ClarkWA87] Clark, Wesley A. "The LINC was early and small", History of Medical Informatics: Proc. of ACM conference on History of medical informatics, Bathesda Maryland, 1987, pp. 51-73 Also available at www.digibarn.com: Description of LINC "personal" laboratory computer with DECTape/LINC-tape. Graphical input for cursor crosshairs (without a mouse) using analog knobs on A/D converter module. Citations to TX-O and TX-2 computers (which ran Spacewar and Sketchpad) [ComerfordLD88a] Comerford, Liam David and White, Steve Richard "Manipulating Rights-To-Execute in connection with a software copy protection mechanism", European Patent Application EP 0268139A3, May 25, 1988 Same as US Patent US5109413. DRM/licencing control for software: conditioned on number of executions, period of time, backup of software license encrypted. [Compucon87] Compucon Services Corporation "letter from Bruce D. Holenstein to Jean Renard Ward", 83 Main Street, Newton, New Jersey 07860, June 23, 1987 Letter from Compucon looking for somebody to buy their character recognition software: does OCR only: they don't admit it doesn't do handwriting [Cooper88] Cooper, Leon N., Elbaum, Charles, Reilly, Douglas L., and Scofield, Christopher L. "Parallel, Multi-Unit, Adaptive, Nonlinear Pattern Class Separator and Identifier", United States Patent 4,760,604, July 26, 1988, assigned to Nestor, Incorporated, Providence, Rhode Island. Sort of a neural-net recognizer for patterns not linearly recognizable [CraneHD88a] Crane, Hewitt D. and Ostrem, John S. "Process and Apparatus Involving Pattern Recognition", United States Patent 4,718,102, January 5, 1988, assigned to Communication Intelligence Corporation, Menlo Park, California Loosely-worded patent on a first pass recognition for handwriting (Kanji) that intentionally leaves in ambiguous categories (initial classification), then a second pass for disambiguation Described basic strokes, then a word/stroke dictionary for Kanji/Chinese recognition Contains review of commercial work in Kanji/Chinese recognition. Cited for Unistroke. [DataEntry88] Data Entry Systems "product information", 1988: see ScriptWriter. [DataSystems87a] DataSystems UK Ltd "The PAD", DataSystems UK Ltd, 35 Holly Avenue, Breaston Derbyshire DE7 3BG, United Kingdom AC plasma display integrated with high resolution transparent digitizer: digitizer type not stated. VME bus,68000 tablet processor controller, 200 points/second, 0.001" nominal resolution. See also National Physical Laboratory. [DaviesTF88] Davies, Thomas F., Matthews, H. Gerard, and Smith, Paul D. "Digitizers and input tablets", "Input Devices", Sol Sherr Ed. Academic Press, Boston, 1988 ISBN 0126399700, Chapter 4 (electronic book) (hardcopy book) (See also SherrS88a) Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets [DayBW88a] Day, Benjamin W. Jr, Gillon, Alexander C., and LeConte, Raoul A. "Touch Screen Form Entry System", United States Patent 4,763,356, August 9, 1988 User-Interface Form input with pen stylus, field highlighted as they are selected, also pops up appropriate GUI tool for that field: pop-up graphical/virtual keyboard, date pad, numeric pad, calculator, electronic ink drawing/writing field. No handwriting character recognition? [DhawanSK87a] Dhawan, Satish K. "Electrostatic Pattern-Coupled Digitizer", United States Patent 4,705,919, November 10, 1987 Charge-ratio electrostatic tablet digitizer [DiederichJ87a] Diederich, Jim and Milton, Jack "Experimental Prototyping in Smalltalk", IEEE Software, May 1987, pp. 50-64 Smalltalk IDE development environment: modeless development (rapid turnaround on code/execute, in this case via interpreter) speeds up development, object-based programming with strict encapsulation tends to make code messes be a high levels of abstraction, not low. Pluggable views, similar to MacApp IDE. [Digitalk88] Digitalk Inc. "Smalltalk/V 286 Tutorial and Programming Handbook", Digitalk Inc., Los Angeles



California, 1988 Introduction to Smalltalk language and environment. (Electronic file). [Dilella87] Dilella, Antonio "Method and Apparatus for Isolating Image Data for Character Recognition", United States Patent 4,680,803, assigned to NCR Corporation, Dayton, Ohio, July 14, 1987 Separates hand-written characters on checks prior to recognition: segmentation by looking at joins, feedback from recognizer [Doran88] Doran, David "Trackballs and joysticks", Appears in Computer graphics: technology and applications, Academic Press, Boston, 1988 ISBN 0126399700 Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets [DruinA88a] Druin, Allison "NOOBIE: the animal design playstation", ACM SIGCHI Bulletin, Vol 20 No 1, July 1988, pp, 45-53 Large stuffed animal designed to interact with children, respond to physical squeezes and touches with sounds. Mostly a study of making machines interactive: compare with later work on animated faces and teddy bears at M.I.T. [Dunkley88] Dunkley, Rowland A. and Pugsley, Peter C. "Sign Verification", United States Patent 4,752,965, June 21, 1988, assigned to The De La Rue Company, PLC, London, England. Portable signature verification system using a force/pressure-sensitive pad/tablet [El-Sheikh88] El-Sheikh, T.S. and Guindi, R.M. "Computer Recognition of Arabic Cursive Scripts", Pattern Recognition, Vol 21 No 4, 1988, pp 293-302 O.C.R. system for Arabic script from typewritten (but cursive, like handwriting) characters Uses Fourier descriptors for characters, topological classifier for dots, stress marks [Ellozy88a] Ellozy, H.A. et al "The paper-like interface (video)", IBM internal video shown in office lobby circa 1989: web links in paper file Description of paper-like interface PLI projects at IBM: see refs. Video also shown at CHI89. [Etherington87] Etherington, H.J.C., Joslin, P.C., and Dunkley, R.A.G. "Sign Verification", United States Patent 4,680,801, July 14, 1987, assigned to The De La Rue Company PLC, England. Signature verification [EvansJW88a] Evans, John W. "Capacitance-variation-sensitive touch sensing array system", United States Patent 4,733,222, March 22, 1988 Capacitive grid touchscreen using diamond shapes on electrodes, separate determination at each node. Shows alternative spiral patterns instead of X/Y grid. Used for touch-sensitive keyboard, may also be used for touch sensing, or finger in "close proximity". Interpolation of multiple readings to get better position. Refers to transparent conductive material for grid so that touchscreen may overlay a CRT display. Conductor traces on opposite sides of board to avoid cross-over. multi-touch capable? [FeldmanJA88a] Feldman, Jerome A., Fanty, Mark A., Goddard, Nigel H., and Lynne, Kenton J. "Computing with Structured Connectionist Networks", CACM, Vol 31 No 2, February 1988, pp 170-187 Overview article on neural networks: asserts merging connectionist/relaxation/adaptation AI/patternrecognition with algorithms+data structures/inference/representation would be good, but it's just speculation [Filliman87] Filliman, Mark D., and Granzow, Robert H. "Personal Identification Method and Apparatus", United States Patent 4,656,662, April 7, 1987, assigned to NCR Corporation, Dayton, Ohio. A patent on the idea of doing signature verification (of any means) using a lightpen and a display [FordDM87a] Ford, D.M.; Higgins, C.A.; Brocklehurst, E.R. "The Electronic Paper Project", Proc. 3 Int'l Symp. on Handwriting and Computer Applications, Montreal 1987, pp. 194-196 Report on paper-like interface / Electronic Paper project PAD. Free-hand editing using proofreader's marks gestures by BSI standard BS5261(1976) [Fox87] Fox, A.S., and Tappert, A.S. "On-line External Word Segmentation for Handwriting Recognition", unpublished manuscript, authors' address: Computer Sciences Department, IBM T.J.Watson Research Center, PO Box 704, Yorktown Heights, NY 10598, 1987 Asserts external segmentation (separate from recognition) is faster, more interactive Cites 0.5% segmentation error, but with six very careful writers Cites that sophisticated segmentation can be done using extra information from having boxes Asserts delayed strokes (t, i crossings and dots) always done at end of each word [Fox88a] Fox, A.S., Greanias, E.C., Kim, J., and Tappert, C.C. "System for Automatic Adjustment and Editing of Handwritten Text Image", United States Patent 4,727,588, February 23, 1988, assigned to International Business Machines Corporation, Armonk, New York. Scribble-editor: gesture recognition? user interface for text-editing/word-processing of handwritten annotation electronic ink Automatic editing and adjustment system for handwritten text images using processing algorithm to identify groups and to smooth random fluctuations in handwritten information [Fox88b] Fox, A.S., Kim, J., and Tappert, C.C. "Reco machine interfaces", IBM Research Report RC14053,



September 1988 Tappert89, describing IBM's handwriting recognition [Fukunaga87] Fukunaga, Y., Kuzunuki, S., Shojima, H., Yokoyama, T., Koga, K., Hirasawa, K., and Kawada, S. "Apparatus for Recognizing and Displaying Handwritten Characters and Figures", United States Patent 4,641,354, assigned to Hitachi, Limited, Tokyo, Japan, February 3, 1987 User-interface: ink erasure for electronic ink and handwriting recognition Patent on electronic ink where handwritten strokes are erased automatically as the strokes for a character are recognized and the ASCII result is displayed instead [FurukawaM88a] Furukawa, Mikio; Tahara, Kazatoki; and Kunishi, Yosuke "Touch-operated See-through Coordinate Input Unit", United States Patent 4,725,696, February 16, 1988 Resistive-film transparent digitizer using grid of conductors between spacer dots: multiple wires between dots for relable operation. Electroconductive ink or paint, or thin wires of metal. [GallantSI88a] Gallant, Stephen I. "Connectionist Expert Systems", CACM, Vol 31 No 2, February 1988, pp 152169 Connectionist / neural network overview Chinese input system using bar-coded "digitizer" and radical identification Chinese input with over 6000 characters (not Kanji/Japanese) [GerbEletronik87a] Gerb Elektronik "CHI-EASY Product Description", (in German and Chinese), Gerb Elektronik Berlin, Roedernallee 174-176, 1000 Berlin 51, West Germany, 1987 [GiuseD88a] Giuse, Dario "LISP as a Rapid Prototyping Environment: The Chinese Tutor", Lisp and Symbolic Computation, Vol 1, 1988, pp. 165-184 Lisp as rapid-prototyping language and UIMS for user interfaces [GoodmanD88a] Goodman, Danny "The Complete HyperCard Handbook, Second Edition", Bantam Books, 1988 HyperCard reference and tutorial: browser for forms and information connected by hyperlinks (within the set of hypercard stacks), scripts/calculations associated. [Goshtasby88] Goshtasby, A. and Ehrich, Roger W. "Contextual Word Recognition Using Probabilistic Relaxation Labeling", Pattern Recognition, Vol 21 No 5, 1988, pp 455-462 Spelling context correction after recognition: does not use confidence figure from recognition about how good a character is, or what else came close Post-processing spelling context correction: examples show incorrect correction Spelling context correction using probabilities of letter pairs Describes relaxation process for spelling context correction by factoring in the merit figure for each character's % confidence level [GouldJD87a] Gould, J.D. and Salaun, J. "Behavioral Experiments on Handmarkings", Proceedings of CHI+GI Conference on Human Factors in Computing Systems and Graphics Interface, Toronto, April 5-9, 1987, pp 175181 IBM study of what gestures/symbols are most "natural" for user interfaces for editing drawing and text: data captured using Macintizer tablet, but no recognition of gestures by computer in system. Pigtail and "X" gestures for delete gestures, circle and arrow gestures for highlighting, arrow gesture for move, lasso gesture for select, etc. [GouldJD87b] Gould, J.D. and Salaun, J. "Behavioral Experiments on Handmarkings", A.C.M. Trans. Office Information Systems, Vol 5 No 4, October 1987, pp 358-377 Sloppy IBM study of what gestures/symbols are most "natural" for user interfaces User interface / gesture recognition: "natural" symbols are circles for groupings, arrows for operators and target position User interface / gesture recognition: design of the actual user interface will actually determine whether gesture symbols are useful, not the use of gestures per se "natural" user-interface study of gesture symbols for text editing / word processing, picture editing [GouldJD87c] Gould, J.D., Alfaro, L., Finn, R., Haupt, B., Minuto, A., and Salaun, J. "Why reading was slower from CRT displays than from paper", Proceedings of the 1987 CHI+GI Conference: Human Factors in Computing Systems and Graphics, Toronto, Ontario, Canada, 1987 Graphic user-interface hardware: what makes a display easy to read [Goy88] Goy, Carl "Mice", Computer graphics: technology and applications, Academic Press, Boston, 1988 ISBN 0126399700 Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets [Graphonomics87] International Graphonomics Society "Membership directory", March 1987, c/o Department of Experimental Psychology, University of Nijmegen, PO Box 9104, 6500 HE Nijmegen, The Netherlands. Index of researchers in handwriting analysis, signature verification, and character recognition [Greanias87] Greanias, E.C., J.J., Verrier, G.F., and Donaldson, R.L. "Combined finger touch and stylus detection system", United States Patent 4,686,332, August 11, 1987, assigned to International Business Machines Corporation, Armonk, New York. Integrated transparent tablet/display for data processing. Flexible film to fit on curved CRT display. Stylus detection capacitive digitizer and finger touch touchpad in one device, integrated with a display.



Capacitive(?) grid. Not clear that it can sense multiple simultaneous finger touch points (multi-touch), but can sense stylus and finger input simultaneously. Compare with Scriptel. Touch detected by strength of signal: 919 patent says this sometimes gave false touches due to variation in signal between grid lines. Bimanual/multi-touch. flexible ITO on flexible film, whole thing is transparent [Greanias88] Greanias, E.C., Schroeder, K.F., and Ruffino, L.V. "Minimum parallax stylus detection", United States Patent 4,764,885, August 16, 1988 Parallax error minimalization for display device / tablet digitizer - enables operator to focus attention on displayed location of cursor instead of stylus location. Emphasizes display of tracking cursor to remediate problems from parallax displacement of reported position from physical position of stylus. [GreensteinJS88] Greenstein, Joel S. and Muto, William H. "Keyboards", Computer graphics: technology and applications, Academic Press, Boston, 1988 ISBN 0126399700 Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets [Gundersen88a] Gundersen, Steven C. "Measure of Distinguishability for Signature Verification", United States Patent 4,736,445, April 5, 1988, assigned to International Business Machines Corporation, Armonk, New York. Signature verification where the most reliable criteria are applied first, then the less reliable criteria or similarity score. Uses X/Y accelleration and pressure/force. Several references to Hurst. [Gundersen88b] Gundersen, Steven C. and Worthington, Thomas K. "Signature Verification Algorithm", United States Patent 4,789,934, December 6, 1988, assigned to International Business Machines Corporation, Armonk, New York. Signature verification using spatial domain, then successively acceleration in frequency domain and force/pressure profile for a coherence score [HCI87] Advance Program "Second International Conference on Human-Computer Interaction", Honolulu, Hawaii, August, 1987 [HallAD88a] Hall, Anthony D., Cunningham, James B., Roache, Richard P., and Cox, Julie W. "Factors Affecting Performance Using Touch-entry Systems: Tactual Recognition Fields and System Accuracy", Journal of Applied Psychology, Vol 73 No 4, 1988, pp 711-720 (abstract only) Study of minimal acceptable optical parallax, angle (standing/sitting, vertical/horizontal touchscreen), size of target on touchscreens/electronic-ink tablet: how much to offset the cursor affects usability. Bigger targets are easier to target. [HallAD88b] Hall, A.D., Cunningham, J.B., Roache, R.P. and Cox, J.W. "Factors affecting performance using touchentry systems: tactual recognition fields and resolution characteristics", Technical Report No TR-29.0787, IBM Corporation, Research Triangle Park, North Carolina, 1988 Beringer89: integrated tablet/display hardware for user-interface? [Hamel88] Hamel, K. "NoteWriter computer software", Computer Software, University of British Columbia, Vancouver, Canada, 1988-1994 Cited by Brown in 1998: unistroke music notation handwriting recognition system [HelanderM88a] Helander, Martin et al, eds. "Handbook of Human Computer Interaction", Elsevier Science Publishers, 1988 Includes chapters Direct Manipulation, Graphics Challenges in Designing Object-oriented User Interfaces, Input Devices (Gesture-based input), Rapid Prototyping for User Interface Design (Wizard-of-Oz) [HellmanME87a] Hellman, Martin E. "Software Distribution System", United States Patent 4,658,093, April 14, 1987 DRM: One-time-use enabling codes for software. Can authorize number of times to use before needing new license authorization. Requires private key on each device for non-transferability. [Hernandez87a] Hernandez, I., Baeker, B.A., Himelstein, C.S. and Machart, B.H. "Business chart arm editing method", United States Patent 4,674,043, June 16, 1987, assigned to IBM Corporation, Armonk, New York. Business chart arm editing method updating predefined data to reflect result of editing action [Hernandez87b] Hernandez, Irene H., Himelstein, Carol S., and Wang, John S. "Method of Editing Graphic Objects in an Interactive Draw Graphic System Using Implicit Editing Actions", United States Patent 4,686,522, August 11, 1987, assigned to IBM Corporation, Armonk, New York. User-interface on graphics drawing/editing: one button to pull up a menu, a second button to select: mode of action persists until stopped User-interface: editing in handwriting/graphics is to modify (several steps in a row), not correct (one step) as in text editing [Herrndobler87] Herrndobler, I. and Machalitzky, C. "Schreiben mit Uli: Schreiblehrgang", Paul List Verlag, Munich, West Germany, 1987 German handwriting textbook to show national variation in handwriting styles (for script) [HigginsCA88a] Higgins, Colin Anthony; Leedham, Graham; and Duckworth Roger James "Machine editing of freehand work", UK Patent Application GB 2 199 169 A, June 29, 1988 Integrated transparent touchscreen panel (transparent position-sensing membrane) and flat LCD display, with menu-driven user interface for handwriting/text recognition and sketch recognition/editing.



[HillRD87a] Hill, Ralph D. "Supporting concurrency, Communication, and Synchronization in human-computer interaction - The Sassafras UIMS", ACM Trans. on Graphics Vol 5 No 3, July 1986, pp. 178-210 UIMS user-interface development system: not single-input stream or single application focus, instead Local Event Broadcast Method LEBM and Event-Response Language ERL for defining UI for multiple input devices, which may also be used for distinct tasks. Simultaneous use of mouse and touch-sensitive tablet for two-handed paint. Bimanual/multi-touch for two pointers? [Hilton88] Hilton, Colin Sefton "Method and Apparatus for capturing information in drawing or writing", European Patent 0 276 109, January 18, 1988 Signature verification tablet using optical sensor: refers specifically to recognizing characters (position) on paper which has different shading or color shading. Optical digitizer with specially-printed paper and stylusess which reads shading of two colors in X and Y gradient, or shades of grey with different lighting sources. Compare with Sekendur. [Himelstein87a] Himelstein, Carol Sue and Wang, John Shihyan "Perspective projection", European Patent 220461, May 6, 1987, assigned to IBM Corporation. User interface in 3-dimensional drawing: computer computes the 3rd depth dimension by user specifying a vanishing point [Himelstein87b] Himelstein, C.S. and Wang, J.S. "Editing method for interactive drawing", United States Patent 4,683,468, July 28, 1987, assigned to IBM Corporation. User interface for sketch input: match pointing position to vertices and endpoints of objects: editing subobjects within objects Identify objects by endpoints of writing motion [Himelstein87c] Himelstein, C.S. and Wang, J.S. "Editing method for interactive drawing", United States Patent 4,686,522, August 11, 1987, assigned to IBM Corporation. Identify objects by endpoints of writing motion [HiroakiN88a] Hiroaki, Negishi; Masaaki, Daimon; and Hiromi, Iwakura "Method for Displaying Window in Windowing System", Japanese Patent JPH02114319 (A), October 25, 1988 (Machine translation to English) Translucent/semi-transparent windows so that user can see objects underneath. Figure 6: transparent on-screen keyboard window. [HoltCM87a] Holt, C.M., Stewart, A., Clint, M., and Perrott, R.H. "An Improved Parallel Thinning Algorithm", CACM, Vol 30 No 2, February, 1987, pp 156-160 Skeletonization/line-thinning on OCR, using 8 compass directions Line thinning with special provision for stair-case elimination [HopkinsD87a] Hopkins, Don "Directional Selection is as Easy as Pie Menus!", 4th Usenix Computer Graphics Workshop, March 30, 1987 (abstract only) Abstract of talk for Usenix on pie menus / marking menus. Menus can be organized semantically: counterpart operations in opposite radial directions, etc. [Huang87] Huang, J.S. and Chung, M.-L. "Separating Similar Complex Chinese Characters by Walsh Transform", Pattern Recognition, Vol 20 No 4, pp 425-428, 1987 O.C.R. for typed Chinese characters, first separation is by zoned template matching method, then by Walsh transform Says 5401 Chinese characters commonly used [HueckingEE87a] Huecking, Ernst E. and Klemmt, Harl-Heinz "Electronic data input keyboard comprising keys provided with conductive contacts", United States Patent 4,709,228, November 24, 1987 Matrix (row and column) keyboard allowing N-key roll-over / multiple key rollover for any number of keys. Row conductors are scanned sequentially by applying a voltage signal, column conductors receive scanning pulse of each row, so that every key can be read regardless of how many keys are depressed. Diode on each key switch prevents phantom key presses from multiple keys. Compare with Kaplow multi-touch, capacitive matrix keyboards and touch sensors. [IBM87a] IBM Corporation "IBM Scientists Demonstrate Personal Computer with Advanced Speech Recognition Capability", IBM Press Release, April 7, 1986 Talkwriter speech recognition, claims 95 percent accuracy, system must be trained to individual speaker's phonemes [IBM87b] IBM Corporation "Three-Axis Touch-Sensitive Pad", IBM Technical Disclosure Bulletin Vol 29 No 8 January 1987, pp, 3451-3453 (partial copy) switch under touchpad/touchscreen so that user can "click" mouse button by pressing touchpad down. Alternative is force-sensitive third layer (implies two-layer touchscreen similar to elographics?) with higher activation force. Compare with MicroTouch mousepad/UnMouse? [InfoWorld87a] Walkenbach, John "Product Comparison: Desktop organizers", InfoWorld, April 29, 1987, pp 43-47 Comparison of products including Borland SideKick 1.5 with phone dialer: also Desktalk from Advance Logic Systems, Homebase 2.5 from Brown Bag Software, HQ 1.02 from TEK Microsystems, Maestro 1.0 from Spectre Software, Metro 1.0 from Lotus Development Corp., PC-Desk from Software Studios, Polywindows Desk Plus 1.2 from Polytron Corp., Pop-Up Desk Set Plus from Popular Programs Phone dialer



from screen on Borland SideKick -- perhaps others also. [IshinoH88a] Ishino, Hiroaki "Optical Character Reader", United States Patent 4,751,743, June 14, 1988, assigned to Sumitomo Electric Industries, Limited, Osaka, Japan. Detecting edge of form and proper paper alignment for a hand-held optical scanner for OCR [KableRG87a] Kable, Robert G. "Position Responsive Apparatus, System and Method having Electrographic Application", United States Patent 4,678,869, June 7, 1987 Electrostatic grid digitizer tablet with grid of correction tables to transform detected coordinates to correct coordinates. Transparent conductors in two layers of grid thinner at crossing points to reduce electrostatic/capacitive coupling. [KableRG87b] Kable, Robert G. "Stylus for position responsive apparatus having electrographic application", United States Patent 4,695,680, September 22, 1987 Shielded stylus for electrostatic grid digitizer, stylus is the receiver [Kahan87] Kahan, S., Pavlidis, Theo, and Baird, H.S. "On the Recognition of Printed Characters of Any Font and Size", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol PAMI-9 No 2, March 1987, pp 274-288 Confusion cases disambiguated through contour analysis on OCR Ligatures / merged characters broken and re-classified, then layout and linguistic context processing Many OCR systems fail accuracy if document is slightly tilted Error rate for manual typing / Human OCR is about 3 per page Practical OCR performance must be 99.9% accuracy, no substitution errors, just rejection Robustness is more important than accuracy (?) Lists many different kinds of character blobs/joins for OCR Linguistic context by certain very simple context heuristics, not grammar and punctuation rules Use UNIX spell dictionary for context manipulation in OCR Use of spelling dictionary does not help much correcting mis-recognition errors Test of OCR system, but omitted small punctuation, disconnected characters, ligatures, and special characters: confusable errors not counted (!) Bayesian classifier assumes statistically independent features, which is not really true Performance of current OCR recognition system are glaring engineering problems, not science Need more theory of human reading ability and automatic design (?) of recognition classifiers For OCR, ligatures are usually accidental serif joins or double-o joins [KahnRE88a] Kahn, Robert E. and Cert, Vinton G. "The Digital Library Project. Vol. 1: The World of Knowbots (draft): An Open Architecture for a Digital Library System and A Plan for Its Development", Corporation for National Research Initiatives NRI, March 1988 Concept study for on-line digital library system DLS (as a national infrastructure): Indexing, registration of users, accounting/billing. "Knowbot" server would track source (author), copyright, ISBN / document ID, publisher, royalty compensations. [Kamal88] Kamel, Khaled and Iman, Ibrahim "A computerized transcription system for cursive shorthand writing", Proceedings of IEEE SouthEastCon, Knoxville, Tennessee, pp. 336-339, April 1988 Cited by Goldberg for Unistroke recognition [Kankaanpaa88] Kankaanpaa, A. "FIDS - A Flat-Panel Interactive Display System", Laboratory of Information Processing Science, Helsinki University of Technology, 02150 Espoo 15, Finland, February 1987: IEEE Computer Graphics and Applications, Vol 8 No 3, March 1988, pp 71-82 Electronic ink paper sent to me for review. Proof-reader's symbols in 1988 paper? simulation of recognition using a keyboard. User interface: electronic ink with gesture/special symbols for editing word-processing text with handwriting. Discusses "orthogonality" of gesture/command handwriting recognition symbols. Resistive tablet Touch-pen from Sun-Flex with two rings (?) as touch switch / tip switch. [Kaplan88] Kaplan, Samuel J., King, Jonathan J., and Sagalowicz, Daniel "Knowledge based processor for application programs using conventional data processing capabilities", United States Patent 4,783,752, November 8, 1988, assigned to Teknowledge, Incorporated, Palo Alto, California. Interface between artificial intelligence rule-based symbolic reasoning software and applications programs [Kashioka88] Kashioka, S., Shima, Y., Miyatake, T., and Ejiri, M. "Method for Producing a Standard Pattern for Pattern Matching", United States Patent 4,783,831, November 8, 1988, assigned to Hitachi, Limited, Tokyo, Japan. Used for machine vision, locating parts [KayA87a] Kay, Alan "Doing With Images Makes Symbols - Communicating with Computers", Video: Distinguished Lecture Series, Apple Computer, 1987 Video presentation on Dynabook, includes demonstration from 1960's (?) of gesture input in a drawing program using handwriting recognition of capital letters, GRAIL system, Sketchpad [KerrickDD88a] Kerrick, D.D. and Bovik, A.C. "Microprocessor-based Recognition of Handprinted Characters from a Tablet Input", Pattern Recognition, Vol 21 No 5, 1988, pp -525-537 D.C.R. system in 32Kbytes of combined RAM/ROM: features are endpoints, corners, joins, angle, curvature, aspect ratio, relative positions D.C.R. system: three levels: low-level features and binary decision tree, binary tree for absence of features, high-level evaluation specific to remaining candidate characters D.C.R. system: retrace only allowed for certain lower-case characters: a, g, h, m, n, q, r, u, y Corner extraction



using Freeman77 Stroke/character segmentation using possible fitting of strokes into a recognized character (bad idea: Easy to create lots of exception cases) This is NOT a robust system. There are too many ways to fool it -- and the authors don't cite much existing work, either. Uses HPR (Handprint Recognition) instead of D.C.R. or On-line character recognition Word segmentation based on overlap of 40% or more [KimJ87a] Kim, J. "Gesture Recognition by Feature Analysis", IBM Research Report RC 12472 (Log #56035) 1/28/87, IBM T.J. Watson Research Center, PO Box 704, Yorktown Heights, NY 10598 It takes more time to write a gesture than a character Asserts character recognition methods will not work for gestures Could not deal with "vocabulary explosion" by including rotational, inversion variants for gestures Recognizing all variants of a letter takes too much time and space (!) Gesture variation effects: non-linear scaling, rotation, mirror-image inversion, reverse direction Usually, a gesture by itself does not make much sense: compound phrases only Gesture recognition (at IBM) using twelve-direction chain codes [KimJ88a] Kim, Joonki "On-line gesture recognition by feature analysis", Proceedings of Vision Interface '88, Edmonton, June 6-10, 1988, pp 51-55 Gesture recognition "differs from" handwriting recognition. Gesture recognition by direction changes (compare to Buxton SSSP reference), thus less subject to differences in rotational, mirror-image, reverse direction, size orientation for cut, pigtail, circle, other gestures. Gestures not independent of application, separate grammatical module for each application operation. Example application is Lotus spreadsheet. Gesture / direct manipulation interface to a spreadsheet program (Lotus 123), part of PLI paper-link interface projects at IBM. First stage of recognizer uses chain codes, second stage is custom-programmed for the gesture shapes. [Kimpan87] Kimpan, C., Itoh, A., and Kawanishi, K. "Fine classification of printed Thai character recognition using the Karhunen-Loe`ve expansion", IEE Proceedings, Vol 134 Pt E No 5, September 1987, pp 257-264 Handwriting recognition for Thai: features are Karhuen-Loe`ve with maximum eigenvalue Features for Thai: Freeman chain codes, concavities and convexities, similar arc pairs: not conclusive results: 100% on training set, "lower" on test data (but not given) Cites other work on Korean, Arabic, Thai handwriting recognition [Kinoshita88] Kinoshita, Akiyoshi "Character Input System", United States Patent 4,724,423, February 9, 1988 Simple zone-type handwriting character recognition reading the strokes on the calculator keyboard [KishiH88a] Kishi, Hajimu and Tanaka, Kunio "Graphic display method for displaying a perspective view of an object on a CRT", United States Patent 4,754,269, June 28, 1988 Matrix algorithm for displaying an image of an object in a 3D projection on a display. Perspective view includes rotation. Compare with CRC Mathematical tables? [KobayashiK88] Kobayashi, K., Sakurai, A., and Sato, G. "Character Recognition Method and System Capable of Recognizing Slant Characters", United States Patent 4,757,551, July 12, 1988, assigned to Ricoh Company, Limited, Tokyo, Japan. Translating scanner input into direction codes: line-thinning. Recognition by histogram of contour direction/chain codes on OCR handwritten numbers. Asserts that slant/tilt varies among handwriting by different people, but consistent for one user [Kondo87] Kondo, Shozo "Some Considerations on Stroke-Structure of Characters Using a Model of Handwriting Process", Proceedings of the Third International Symposium on Handwriting and Computer Applications, Ecole Polytechnique de Montreal, Canada, 1987, pp 72-73 [KramerSM88a] Kramer, Steven M. "On Incorporating Access Control Lists into the UNIX Operating System", Proc. USENIX UNIX Security Workshop, Aug 23..30, 1988, pp. 38..48 Report on vendor proposals for ACLs access control lists on UNIX, in response to DoD "Orange Book" (see DoD85). Cautionary essay on rush to implement vs. consideration of impacts on rest of operating systems, especially DAC discretionary access control (permissions/rights set by user) vs. mandatory access control (permissions set by system on user). Multiple ways to combine ACLs with traditional UNIX OGO (Owner, Group, Superusr) definitions. Wildcard syntax in ACLs. Denial of rights ("deny" permission) complicates things more. Extended permissions: examples of ownership (right to set permissions), extend (make file larger), truncate file, delete. [KreinPT88a] Krein, Philip T. and Meadows, R. David "The Electroquasistatics of the Capacitive Touch Panel", IEEE Conf. Proc. 1988 88CH2565-0/88/0000-171 Seems to be same as KreinPT90a: Indium Tin Oxide ITO resistive film touchscreen errors [KruegerMW88a] Krueger, Myron W.; Hinrichsen, Katrin; Gionfridd, Tom; and Sonnanburg, Joan "Videoplace '88", Studio in the Museum of Natural History, Vernon CT. June 10, 1988 Videoplace: video of exhibit using optical (TV image) detection of multiple fingertips (multi-touch), singletouch and multi-hand gestures for manipulating graphical objects and animations. [KurlanderD88a] Kurlander, David and Bier, Eric A. "Graphical Search and Replace", ACM Computer Graphics, Vol



22 No 4, August 1988, 113-120 Scriptable/pattern-based technique for bulk editing of graphical images, by specifying a transformation of graphical objects whose properties, color, shape, etc. match a pattern or example. Scripting referred to as mouse-click macros. [Kurta87a] Kurta Corporation "IS/PenMouse", Kurta Corporation, 1987 Technical specification on Kurta IS/PenMouse digitizer tablet, includes examples of binary format on serial port. Stylus or puck, each with additional buttons (on side of stylus, etc.) [Kurta88a] Kurta Corporation "Kurta out-does the competition with Architecture Templates", Kurta Corporation, 1988 On-table menus/command templates for VersaCAD, AutoCAD, electrical symbols, other PC/DOS CAD applications. Compare with similar offerings from CIC/ Pencept: this without handwriting recognition or gestures. [Kurta88b] Kurta Corporation "Kurta out-smarts the mouse", Kurta Corporation, 1988 On-tablet menus/command strips, described as additional user-programmable Function Keys for use with Apple MacroMaker for keyboard shortcuts. Cordless stylus to 12-button cursor (puck). [Kurtzberg87] Kurtzberg, J.M. "Feature Analysis for Symbol Recognition by Elastic Matching", IBM Journal of Research and Development, Vol. 31 (1) (1987) 91--95 Combines feature analysis with elastic matching to filter out unlikely prototypes: pruning features for preprocessing recognition a number of strokes, points per stroke, relative heights of endpoints / maxima of strokes [KustanovichY87a] Kustanovich, Yosef "Surface-area pressure transducer and line-selection circuit for use therewith", United States Patent 4,644,801, February 24, 1987 Grid of capacitive pressure sensors with compressible material between the plates of the capacitors. Each capacitor can be measured individually. Used for biomedical instrumentation (pressure profile of foot) [Kuzunuki87] Kuzunuki, Soshiro, Shojima, HHiroshi, Yokoyama, Takanori, Fukunaga, Yashushi, and Hirasawa, Kotaro "Method for Designating a Recognition Mode in a Hand-Written Character/Graphic Recognizer", United States Patent 4,680,804, August 14, 1987, assigned to Hitachi, Limited, Tokyo, Japan. Projections of strokes on to X and Y axes for recognition: mode setting: strokes larger than a certain size are annotation, not characters. Simple size context to distinguish characters and drawing marks [KwokPCK88a] Kwok, P.C.K. "A Thinning Algorithm by Contour Generation", CACM, Vol 31 No 11, November 1988, pp 1314-1324 Skeletonization/line-thinning with grey-scale information [LaLondeW88a] LaLonde, Wilf and Pugh, John "Smalltalk: Graphics Through the Looking Glass", JOOP, August 1988, pp. 52-68 Implementation of a magnifying-glass tool (with transparent window) in Smalltalk: "white hole" (transparent window section) with a border to highlight. [LakinF87a] Lakin, Fred "Visual Grammars for Visual Languages", Proc. AAAI-87, pp. 683-688 Two-dimensional visual languages using spatial parsing (e.g. mathematical expressions), shows handwritten input using tablet digitizer. Contains additional references back to 1971, for visual parsing of sketches of houses, etc. [LapeyreJM87a] Lapeyre, James M. "Electro-Optical Position Determining System", United States Patent 4,688,833, August 25, 1987 Optical digitizer/tablet for large area, two optical sensors on the edges of the tablet with triangulate to a light LED on the stylus [LeeS88a] Lee, Seongwhan, and Kim, Jin H. "Automatic verification of seal imprints using attributed stroke graph matching", Proceedings of the International Conference on Computer Processing of Chinese and Oriental Languages, 1988, pp 66-70 Lee,S89 [LeeS88b] Lee, Seongwhan, and Kim, Jin H. "Recognizing hand-drawn symbols in engineering drawings", Artificial Intelligence in Engineering: Robotics and Processes", Computational Mechanics, Southampton, 1988, pp 179-200 Lee,S89 [Leung87] Leung, C.H., Cheung, Y.S., and Wong, Y.L. "A Knowledge-Based Stroke-Matching Method for Chinese Character Recognition", IEEE Trans. Systems, Man, and Cybernetic, Vol SMC-17 No 6, November 1987, pp 9931003 Relaxation method for distance from feature-space template with reduced computational load for Chinese recognition Minimum-distance template matching in feature space Trained/adaptive system, but with specific knowledge of writing Chinese characters, too Tested on 240 Chinese characters only, notes that extension to 4000 will add problematically to compute time User training sample collection (for only 240 Chinese characters!) is admitted to be tedious [Licklider88] Licklider, J.C.R. "Interview with J.C.R. Licklider, Conducted by William Aspray and Authur Norberg",



28 October, 1988. Charles Babbage Institute, Center for the History of Information Processing, University of Minnesota, Minneapolis. Available at http://www.cs.utexas.edu/users/umair/cs370/interviews/oh150jcl.rtf Mentions Herb Teager's early work on a capacitive (?) coupling digitizer tablet, and the alternative project of a mouse: Page 20, with mouse and light-pen. [Linus87a] Linus Technologies "LINUS (tm) Write-Top User's Guide", Linus Technologies Inc, 1889 Preston White Dr., Reston, VA 22091 Shows writing/editing with handwriting recognition on a single surface display/tablet digitizer [Linus87b] Linus Technologies "Linus Write-Top / TRW Write-Top: Linus 1000", Posting at oldcomputers.net, fetched 2014 Touchscreen laptop computer, show calibration and application screens [Linus88a] Linus Technologies "Linus plans tiered Write-Top channel strategy", Product announcement for WriteTop handwriting recognition product, Computer and Software News, June 13, 1988 Product announcement, portable handwriting recognition product for Linus Technologies: Write-Top. Additional materials in file: Ralph Sklarew and Robert Nadeau, George Mason University: original name was "Techbook" [Linus88b] Linus Technologies "Equipment/Device: Linus Technologies Write-Top, ca. 1988", Linus Technologies (physical device) Early portable pen-computing, predecessor to PenPoint from GO, Microsoft PenWindows and Tablet PC. Touch-screen digitizer connected to display unit. (Stylus missing). [Linus88c] Linus Technologies "Linus Just-Write and Just-Write II Software Reference Manual", Linus Technologies, Inc, 1988 Handwriting recognition text-editing application. Desktop application shows text listing of files (similar to File Commander) on DOS operation system, for executing an opening files. Also has virtual keyboard. [Linus88d] Linus Technologies "Portable Computer Runs on Handwriting", Insight Magazine, August 22 1988 Linus Write-Top portable computer: Handwriting recognition directly on touchscreen, healthcare/nursing applications [LittlePD88a] Little, Paul D. "Audio/telephone communication system for verbally handicapped", United States Patent 4,785,420, November 15, 1988 On-tablet menus as talking UI for verbally handicapped using a touch digitizer: Chalkboard from Powerpad. Note that Powerpad (later models) accepted multi-touch input, resolution 0.1 inches. [LiuzzoJG87a] Liuzzo, James G. and Proettea, William J. "Keyboard with arrays of function keys", United States Patent 4,698,618, October 6, 1987 Physical keyboard with function keys near numeric row. Key to adjust display brightness. [LuP87] Lu, P.-Y. "professional resume", for work in on-line handwriting recognition Resume for former employee of Communications Intelligence Corporation. Device Driver and Mouse Emulator for digitizing tablet [Lukis87] Lukis, L.J. "Method and Apparatus for Correcting X-Y Position Measurements", United States Patent 4,679,241, July 7, 1987, assigned to Moore Business Forms, Incorporated, Grand Island, New York. Resistive-sheet digitizer tablet with correction table calibration Several references to British patents for portable handwriting recognition dynabook: diagrams show Moore Business Forms handwriting product (portable) [MaarseFJ88a] Maarse, F.J., Janssen, H.J.J., and Dexel, F. "A special pen for an XY-tablet", preliminary manuscript, University of Nijmegen, Department of Experimental Psychology, Montessorilaan 3, PO Box 9104, 6500 HE Nijmegen, The Netherlands, 1987. Published 1988 in "Human-Computer Interaction: Psychonomic Aspects", Springer Verlag, pp. 353-360 Measuring pen stylus tilt angle using peak levels of signal picked up in electromagnetic tablet: digitizer which reports stylus angle [Machart88] Machart, Beverly H., and Wang, John S. "Freehand Drawing Containing Invisible Lines", United States Patent 4,757,549, July 12, 1988, assigned to International Business Machines Corporation, Armonk, New York. Stroke segmentation in sketch recognition: group strokes together in a signature, or any free-hand writing or scribble, if they are close enough together Compares stroke starting and ending positions with previous storage ending to detect termination of a signature [MaedaK87] Maeda, Kenichi, and Nitta, Tsuneo "Pattern Recognition Apparatus and Method for Making Same", United States Patent 4,651,289, March 17, 1987, assigned to Tokyo Shibaura Denki Kabushiki Kaisha, Kawasaki, Japan. After training, a speech/handwriting recognition system stores deltas from previously learning patterns on misrecognition, not entire pattern Reduce memory size for pattern dictionary by storing deltas from originals, not whole patterns [MakkuniR87a] Makkuni, R. "A gestural representation of the process of composing Chinese temples", IEEE Computer Graphics and Applications, Vol 7 No 12, December 1987, pp 45-61



[Malzbender87] Malzbender, T. "Permuted Trace Ordering Allows Low-Cost, High-Resolution Graphics Input", Hewlette-Packard Journal, June 1987, pp 4-7 Description of Hewlett-Packard's digitizer electrostatic tablet circuitry. By permuting traces in an X/Y grid, and reporting the numbers of which traces which detected a signal in decreasing strength, able to interpolate very high accuracy without relying on absolute measurements of signal strength. Amplifier built into stylus. HP 45911A digitizing tablet. [Mandler87] Mandler, Eberhard "Advanced Preprocessing Technique for On-Line Script Recognition of Nonconnected Symbols", Proc. 3rd Int. Symp. on Handwriting and Computer Applications, pp. 64-66, July 1987 Cited by Marlin Eller, Microsoft Pen Computing group [MatthewsHG88] Matthews, Henry G.; Zalenski, Thomas; Barbetti, Jamie L; and Mletzko, Al "Menu for a Charge Ratio Digitizer", United States Patent 4,788,386, November 29, 1988 Summagraphics charge-ratio tablet, with tapered traces and varying-width traces on a PC board [MatzkeKE88a] Matzke, Karl E. and Schick, Paul W. "Touch Activated Control Method and Apparatus", United States Patent 4,736,191, April 8, 1988 Circular capacitive touchpad (for keyboard), radial plates: which plate touch controls direction of movement of cursor, pressure (as inferred from change in capacitance) determines velocity of cursor. Tap and dwell input on touchpad for control functions. Dwell/press-and-hold. [McArthurD87a] McArthur, David; Stasz, Cathleen; and Hotta, John Y. "Learning Problem-Solving Skills in Algebra", RAND Note N-2595-NSF, Prepared for The National Science Foundation, May 1987 (abstract only) see 1990 publication by same authors [McAvinneyP88a] McAvinney, Paul "Method and apparatus for isolating and manipulating graphic objects", United States Patent 4,746,770, May 24, 1988 Optical beam touchscreen device, multiple fingers. See McAvinney bio reference: states was first optical multi-touch device. [McCaskillRA88a] McCaskill, R.A., Wang, J.S., and Repass, J.T. "Printed document processing", United States Patent 4,739,314, April 19, 1988, assigned to IBM Corporation. User-interface: specifying measurements of printed document objects by pointing to them [MelBW88a] Mel, Barlett W.; Omohundro, Stephen M.; Robison, Arch D.; Skiena, Steven S.; Thearling, Kurt H.; Wolfram, Stephen; and Luke, Young T. "Tablet: The Personal computer of the Year 2000", Communications of the ACM, June 1988, pp 638-646; also Report No UIUCDCS-R-88-1406/UILU-ENG-88-1711, U. Ill. at UrbanaChampaign, February 1988 Forerunner proposal to iPhone/iPad? Winning proposal in Apple-sponsored competition to envision the computer of the future. Touchscreen (stylus for high resolution input) used as a mouse. Gesture for turning on is to touch two corners. Voice input/output (multi-touch? or just didn't know tablet technology?): no other gestures?. Built-in camera for scanning/digitizing documents, video e-mail, videoconferencing, GPS with complete internal map storage. Social apps for finding interesting people nearby. Interactive group gaming. [Microsoft87a] Microsoft Corporation "Microsoft Excel Arrays, functions and Macros", Microsoft Corporation, 1987 (hardcopy book) Description of Microsoft Excel spreadsheet program for Windows [Microsoft87b] Microsoft Corporation "Reference to Microsoft Word Word Processing Program Version 3.0 for the Apple Macintosh", Microsoft Corporation, 1987 (hardcopy manual) Word 3.0 Word processing for the Apple Macintosh - shows integration with Apple menus. Contains update for version 3.01 describing differences from Word 1.0 for the Macintosh. [Microsoft87c] Microsoft Corporation "Learning Microsoft Word: Word Processing Program Version 3.0 for the Apple Macintosh", Microsoft Corporation, 1987 (hardcopy book) Word 3.0 Word processing for the Apple Macintosh - shows integration with Apple menus. Contains update for version 3.01 describing differences from Word 1.0 for the Macintosh. [Microsoft88a] Microsoft Corporation "Descriptions of Expert and Standard Mouse Menus", support.microsoft.com, article 23837 / Q23837, approximate date 1988 Description of MENU.COM, *.MNU files, MENUMAKE.DOC, MENUREAD.ME. Mouse menus allow for support of mouse in DOS applications that are not "mouse-aware". MENU.COM uses static menu definitions, expert menus are *.COM files with specific knowledge of application. Compare with keyboard macros, and with recognition macros by PenCept and CIC for handwritten gestures. File contains additional references concerning mouse menus for Application That Do Not Use the Mouse, standard keyboard layouts Keyboard Styles fro IBM and IBM Compatibles, Using Microsoft MOuse Menu Software with MS-DOS 4.x-6.0 (MenuMaker for mouse menus not compatible with extended key codes) etc. [MiletzkiU87a] Miletzki, U; Doster, W.; Fogaroli, G.; Lobl, H; and Moulds, P. "Paper Interfaces for Office Systems", ESPIRIT '86, pp. 373-397, 1987 Cited in FordDM91a: regarding gestures



[Mishima88] Mishima, T., Kanasaki, M., Takatoo, M., and Ota, H. "Half-tone image recognition / Method and apparatus for recognizing pattern of halftone image", European Patent 287995, October 26, 1988, assigned to Hitachi, Limited, Tokyo, Japan. Image processing: recognize halftone texture in a scanned image, to do OCR character recognition without binarization, so that contrast/brightness does not affect recognition [Mochinaga88] Mochinaga, N., Moto, T., and Ogata, Y. "Pen-type Character Recognition Apparatus", United States Patent 4,751,741, June 14, 1988, assigned to Casio Computer Company, Limited, Tokyo, Japan. Pen stylus which measures lateral pressure/force using strain gauges for handwriting recognition, instead of an X/Y digitizer : used for handwriting recognition / character recognition [Morishita87] Morishita, T., Ooura, M., and Ishii, Y. "A Kanji Recognition Method Detecting Incorrectness in Writing", Proceedings 1987 International Conference on Computer Processing of Chinese and Oriental Languages", pp 67-74 Claims 99.7% recognition of Kanji characters "with errors", 99.9% for "correctly written" Chinese characters, used to check for correct writing style in Japanese, "Standard Writing" by the Japanese Council on the National Language [MullerMJU88a] Muller, Michael J. "Multifunctional Cursor for Direct Manipulation User Interfaces", Proc. CHI '88, pp. 89-94 Cursor showing state of current selected tool: suggests first selecting tool and then selecting object to apply it to (in the meantime, cursor indicates which tool is "active), rather then first select object and then select operation/command to apply to it. Update (pop-up?) menus/icons on mouse input? Mentions implementation for 1/2/3-button mouse, with single- and double-click actions. [MyersBA87a] Myers, Brad. A. "Creating Interaction Techniques by Demonstration", IEEE CG&G Computer Graphics and Application, September 1987, pp. 51-60 Peridot: GUI generator for direct-manipulation interfaces, action of user interface can be defined by defining default/initial parameter values (internal values) and running an example -- and then editing code. Supports two-handed bimanual (multi-touch) input by supporting multiple locator input devices. [MyersBA88a] Myers, Brad. A. "The Garnet User Interface Development Environment: A Proposal", CarnegieMellon University Technical Report CMU-CS-88-153, September 1988 In Garnet system papers: set of UIMS/UI design tools for direct manipulation GUIs. Common Lisp IDE, but objects use prototype inheritance, not classes. Any change in property of an object immediately updates display of object (e.g. color management, color properties, or uncovering a window) [NCSC87a] National Computer Security Center "Trusted Network Interpretation", NCSC-TG-005 Version 1, Library No. S228,526, 31 July 1987, Gallagher, Patrick R. Jr. Director. Available at csrc.nist.gov (2015) Extension of NCSC Trust System Evalution Criteria (Dod85) to trusted network systems and components. Communications integrity, physical security: specifics of encryption outside of scope. [NYTimes88a] Lewis, Peter "The Executive Computer -- Exotic Gizmos for Aiding Workers", November 20, 1988, New York Times ComDex: Private Eye head-mounted single eye display monitor, Reflection Technology Ink: Wang Freestyle combining text, handwriting (electronic ink notes), voice recording, electronic mail (via FAX) and filing. Mentions Scriptel tablets. [Nadler87] Nadler, Morton "private communication with Morton Nadler", August, 1987 This fellow was in handwriting character recognition in the 1960's , on ANSI handprint-standard committee as GE representative NADLer Inc., Blacksburg VA (circa 1986) [Nakamura87a] Nakamura, Shoichiru, and Kable, Robert "Electrographic System and Method", United States Patent 4,650,926, assigned to Scriptel Corporation, March 17, 1987 Correction grid matrix for digitizer: divide into small correction rectangles [NationalPhysicalLaboratory87] National Physical Laboratory NPL "Electronic Paper: The Electronic Paper Collaboration", National Physical Laboratory NPL, Division of Information Technology and computing, Tedding, Middlesex TW11 0LW, United Kingdom Scriptel digitizer and plasma display, with handwriting/cursive recognition. Commercial development by DataSystems UK. Says "Electronic Paper is nearly here" (sic). [Nestor87] Nestor, Inc. "NestorWriter Product Description", available from Nestor Incorporated, One Richmond Square, Providence RI 02906. Cited in --->"Computing with Neural Networks", High Technology, Vol 7 No 5, May 1987, pp 28-29 References to Nestor's work on handwriting recognition product [Nestor88] Nestor, Inc "Press/correspondence file on Nestor, Inc.", (personal correspondence) References to Nestor's work on handwriting recognition products and technology using neural nets. Mike Buffer, Leon Cooper. [NielsenJ87a] Nielsen, Jakob "CHI+GI '87 Trip Report", http://www.useit.com/papers/tripreports/chi87.html Mentions demonstration of DataGlove, with a hand-gesture of opening the fingers of the hand to clear a



graphic screen (swipe gesture?). Demo of Alternate Reality Kit with direct manipulation of simulated objects using a mouse. (See Alternate Reality Video) [NielsenJ88a] Nielsen, Jakob "CHI '88 Trip Report", http://www.useit.com/papers/tripreports/chi88.html Refers to "two-cursor" problem, but this has nothing to do with multi-touch multiple touch inputs on a digitizer or tablet, it is a cognitive problem of needing to point simultaneously to a command (e.g. "paste" icon) and a target (position in text). [Numonics88a] Numonics Incorporated "#2200 Digitizer Pad Technical Information", Available from Numonics Incorporated [Numonics88b] Numonics Incorporated "CADCommand product information", Available from Numonics Incorporated, Montgomeryville PA Pencept PenCAD and PenPad 320 sold under Numonics name: recognition macros / gestures, handwriting recognition [Numonics88c] Numonics Incorporated/Terminal Display Systems Limited "MAX - Multi-Axis Input Digitizer", "ZedPEN product literature", Terminal Display Systems, Lower Philips Road, Blackburn England BB1 5TH, 1988 5-axis digitizer (X,Y, theta/phi/rho), plus force/pressure sensor, plus "flick" accelerometer sensor [Numonics88d] Numonics Incorporated "Numonics Corporation covers computer graphics from architecture to zoology", Numonics Corporation, 1988 Numonics product line (pre-Pencept), shows electromagnetic and electromechanical 2D digitizer tablets. Models 1210, 1220, 1240, 1250 1224, 1224CF electromechanical digitizers, Models 200, 2300, 2400 tablets, with 16-button cursor/puck, stylus. Electromechanical include firmward for computing areas. [OoiK87a] Ooi, K., Hidai, K., Kurosawa, Y., and Nakamura, Y. "Method and Device for Handwritten Letter Recognition", United States Patent 4,685,142, August 4, 1987, assigned to Kabushiki Kaisha Toshiba, Kawasaki, Japan. Kanji/Japanese recognition using standard strokes, in a standard stroke order Attempt to remove strokeorder properties from Kanji recognition using a heirarchical dictionary of standard strokes, and matching as you go [OritaM87a] Orita, M., Kobayashi, Y., and Kuboo, Y. "Tree-structure recognition dictionary preparation system", United States Patent 4,658,429, April 14, , 1987, assigned to Hitachi, Limited, Tokyo, Japan. Order tree nodes in a recognition dictionary of templates, so features with the most power are highest in the tree [OritaM87b] Orita, M., Mishima, T., and Kobayashi, Y. "System and Method for Preparing a Recognition Dictionary", United States Patent 4,682,365, July 21, 1987, assigned to Hitachi, Limited, Tokyo, Japan. Hierarchical tree-structure for pattern recognition decision tree: method to minimize search time by organizing tree (for Kanji recognition) Cites prior art (1975) on tree-structured pattern recognition dictionaries for Kanji/Japanese recognition [Ovonics88] Ovonic Imaging Systems OIS, Inc. "E-Z image and "OIS Telepad" product literature, 1988", 1896 Barrett Street, Troy, MI 48084, 313-362-3140 EZ-Image is a clear digitizer tablet, resistive sheet: electronic ink with integrated LCD display Telepad is an integrated digitizer/display unit Uses the term "electronic paper" (similar to "electronic ink") [PearsonG87a] Pearson, G. and Weiser, M. "Exploratory Evaluations of Two Versions of a Foot-Operated CursorPositioning Device in a Target-Selection Task", poster paper, CHI+GI 87 Conference, Boston, Massachusetts, April 1987. Authors' address: Heterogeneous Systems Lab, University of Maryland, College Park, Maryland. Foot-operated pointing device, similar to mouse [PenMouse88a] PenMouse Technologies "Equipment/Device: PenMouse device, ca. 1988", PenMouse, PenMouse Technologies (physical device) Roller-ball mouse in a pen-shaped housing, does not report absolute position. Not the same as the IS/PenMouse digitizer tablet from Kurta. [Pencept88a] Pencept "PenPad Digitizer and Intelligent Tablets -- Using Artificial Intelligence to Make Your Computer Faster and Easier to Use", Pencept division of Numonics, Inc., 39 Green Street, Waltham, Massachusetts 02154, 1988 Pencept PenPad 300, 310, 320 electromagnetic tablets, handwriting recognition, command macros (gestures) [PersonalWriter88a] Personal Writer Inc. "Personal Writer PW15 SL User Manual", Personal Writer Inc., 1801 Avenue of the Starts Suite 507, Los Agneles CA 90067 (hardcopy book) Handwriting recognition tablet for MacIntosh, unconnected/discrete letters -- no boxes, but characters must be widely spaced. Trainable recognition. On-tablet menus and tablets with drawing, recognition, command, and menu areas, macro definitions, mouse areas. Recognition makes use of personally defined text dictionaries. Keyboard emulation, mouse emulation with tablet. Remarkably similar functionality to PenCept PenPad and CIC HandWriter. Includes sheets for user to write 5 examples of each character/symbol shape to be recognized.



[Phillips87] Phillips, M. "Several simple tests can help you choose the correct digitizer", Computer Technology Review, Vol VII No 1, January 1987 Mark Phillips / Ted Kuklinski tablet tests: includes tilt measurements [PierK88a] Pierk, Ken; Bier, Eric; and Stone, Maureen "An Introduction to Gargoyle: An Interactive Illustration Tool", Proc. Int'l. Conf. on Electronic Publishing, Document Manipulation and Typography, Nice France, April 1988, p. 223-238 Gargoyle graphical drawing editor: uses snap-dragging instead of snapping to alignment grid for precise positioning. Snap-dragging: objects may have "gravity", causing snap to a control point or to a surface/line if the cursor comes close enough. Refers to Cedar context-sensitive pop-up menus: compare to marking menus -- user can hold cursor over object, and long-press on any mouse button to get the menu -- pressand-hold? [Platshon88] Platshon, Mark "Acoustic touch technology adds a new input dimension", Computer Design, March 15, 1988, pp 89-93 Technical description of Elographics digitizer technology, surface acoustic wave (SAW). SAW technology for force/pressure-sensitive "Z" on touch tablet digitizer. Refers to finger as a "pliable pointer", system can infer force from effect of increasing contact area: pressure/force gestures e.g. light touch for drag, firm/heavy touch for click/activate. [Pobgee88a] Pobgee, P.J. "Prototype System for Interactive Input of Cursive Information", National Physical Laboratory, Teddington (England), Division of Information Technology and Computing, Report No NPL-DITC125/88, 1988 (abstract only). NTIS citation index: refers to pointing and handwriting recognition combined in a GUI application. User interface and editing and graphics manipulation for handwriting recognition. [Pobgee88b] Pobgee, P.J. "EPT, a Dynamic Tutorial for Introducing New Users to Electronic Paper", National Physical Laboratory, Teddington (England), Division of Information Technology and Computing, Report No NPLDITC-134/88, 1988 (abstract only). NTIS citation index Integrated tablet and display with sample handwriting applications [PotterR88a] Potter, R., Weldon, L., and Shneiderman, B. "Improving the accuracy of touch screens: An experimental evaluation of three strategies", Proceedings of CHI conference, Washington, D.C., May 15-19, 1988. Published by the Association for Computing Machinery, New York, 1988, pp 27-32 Touch screens advantageous/desirable because do not consume workspace, no moving parts. Accuracy limitations due to visual parallax: target on touch-down, target on drag-to-touch; target on lift (to be patented) -- does not note that target-on-lift typically employed in mouse systems, albeit for different reasons. Mentions highlighting before lift, so that user can tell what would be selected if user lifted pen then. Mack89 digitizer user-interface hacks [Reilly87] Reilly, Douglas, L., Scofield, Christopher, Elbaum, Charles, and Cooper, Leon, N. "Learning System Architectures Composed of Multiple Learning Modules", Proceedings of IEEE First Annual International Conference on Neural Networks, June, 1987, Vol II, pp 495-503 Training time is shorter if you train one module to each shape class, rather than many modules to share training, in a Nestor neural network. [Reilly88] Reilly, Douglas L, Scofield, Christopher, Gouin, Philip R., Rimey, Raymond, Collins, Edward A., and Ghosh, Sushmito "An Application of a Multiple Neural Network Learning System to Industrial Part Inspection", to be presented at ISA/88, Houston, Texas. Nestor recognition paper on industrial vision parts inspection [Rhyne87] Rhyne, J. "Dialogue Management for Gestural Interfaces", Computer Graphics, Vol 21 No 2, April 1987, pp 137-142. Also IBM Research Report RC 12244, 1986 Temporal grouping/segmentation of written gestures more important than the temporal sequence [RodgersJL87a] Rodgers, James L. and Waterbury, Charles A. "Low power high resolution digitizing system with cordless pen-mouse", United States Patent 4,672,154, June 9, 1987 Capacitive/electrostatic digitizer with X/Y grid. Stylus is battery powered. Interpolation between grid lines. Stylus buttons sensed by changing transmitting frequency of pen slightly. [RosenbergJM87a] Rosenberg, Jerr M. "Dictionary of Computers, Information Processing and Telecommunications, Second Edition", Wiley, 1987 (hardcopy book) Technical dictionary of computer terms [RubinT88a] Rubin, Tony "User Interface Design for Computer systems", Ellis Horwood Books in Computer Science, U. of London, 1988 (hardcopy book) User interface guidelines concerning keyboard commands, error messages, text menus. Section on WIMP Interfaces: keyboard, mouse, joystick, trackball, touch sensitive screens, light pen, speech input / speech recognition. Oldest touchscreen stated as touch wire touch sensitive screens / TSS -- also mentions conductive grid: same thing? WIMP interface/ GUI examples taken from MacIntosh. [RubineD88a] Rubine, Dean and McAvinney, Paul "The Videoharp", Proc. ICMC 88, pp. 49-55



Electronic VideoHarp musical instrument using optical sensing of multiple fingers (multi-touch) and gestures: either harp-like (strum) or piano-like (projected keyboard). Optical sensing on two sides (either side) of transparent instrument, such that instrument can be played with both hands like a harp, or mounted horizontally like a piano. MIDI output. to a synthesizer. Position of finger along a ray detected by shadowing. File contains additional materials showing Video harp construction, frame, and use. See also McAvinneyP90a [Rympalski87] Rympalski, William P., Herstein, James S., Ritenour, Roger L. "Electronic Sketch Pad", United States Patent 4,639,720, assigned to Harris Corporation, Melbourne, Florida, January 27, 1987 Capacitive digitizer integrated with an LCD display: patent on "writing" directly on a display. conformable stylus tip allows for indirect pressure sensing. Mutual capacitance, forms image of objects on display, therefore also multi-touch? [SAC87a] Science Accessories Corporation "GP-7 Grafbar Mark II Operator's Manual", Science Accessories Corporation, 1987, 970 King's Highway West, Southport, Connecticut 06490 Sonic/acoustic digitizer tablet, notes that has slant range correction, resettable coordinate origin, physical diagrams of stylus/pen and cursor/puck. Notes that drafts affect performance, wind changes speed of sound (?). Spark gap as acoustic source. [SAC87b] Science Accessories Corporation "QUESTOR III Model GP-9 two-dimensional sonic digitizer -- large areas -- projected images", Science Accessories Corporation, 1987, 970 King's Highway West, Southport, Connecticut 06490 Sonic/acoustic digitizer tablet, 2D, multi-button puck or stylus, large format. Has single large transducor/sensor bar. Mentions 3D models with up to 9x9x9 foot volume. Special requirements can be addressed. [SAC88a] Science Accessories Corporation "Capture / Create / Analyze / Direct with sonic digitizers", Science Accessories Corporation, 1988, 970 King's Highway West, Southport, Connecticut 06490 Sonic/acoustic digitizer tablet, 3D, up to 9x9x9 foot volume. [SIGSOFT87a] SIGSOFT "Anti-Skid Brakes", quoted from October 1986 issue of Road and Track, A.C.M. SIGSOFT Software Engineering Notes, Vol 12 No 1, page 10, January, 1987 Unanticipated/intermittent input/event causes software system error [SIGSOFT87b] SIGSOFT "Cause of the Mysterious Bay Area Rapid Transit Power Outage Identified", quoted from July 8, 1987 San Francisco Chronicle, A.C.M. SIGSOFT Software Engineering Notes, Vol 12 No 4, page 4, January, 1987 Unanticipated/intermittent input/event causes software system error [SIGSOFT87c] SIGSOFT "Actual Stock Price Fails Sanity Check", contributed by Mark Brader, A.C.M. SIGSOFT Software Engineering Notes, Vol 12 No 4, page 4, October 1987 Unanticipated/intermittent input/event causes software system error [SIGSOFT87d] SIGSOFT "BBC Documentary Filming Causes Library of Congress Computer Crashes", contributed by Howard C. Berkowitz, A.C.M. SIGSOFT Software Engineering Notes, Vol 12 No 4, page 15, October 1987 Unanticipated/intermittent user input/event causes software system error [SIGSOFT88a] SIGSOFT "Runaway mouse problem in popular commercial WP program", contributed by Steven Jones, A.C.M. SIGSOFT Software Engineering Notes, Vol 13 No 4, page 9, October 1988 Press report of mouse/driver (?) but in Word 4.0, Microsoft mouse on PS/2 port generates runaway input (buffer bug?) [Saba87] SABA "Handscan product literature", Saba Technologies, Incorporated, 9300 S.W. Gemini Driver, Beaverton, Oregon, 97005, June 1987 Produce literature on hand-held OCR input scanner Acoustic/sonic 2-D tablet [Satoh88] Satoh, Koji "Method of Character Recognition", United States Patent 4,783,835, November 8, 1988, assigned to Sumitomo Electric Industries, Limited, Osaka Japan. Typed character recognition in a hand-held scanner using rightside and left-side profiles/chain codes, in addition to line features [Schoonard87] Schoonard, James W.; Gould, John D.; Bieber, Miriam; and Fusca, AAngie "A Behavioral Study of a Computer Hand Print Recognition System", IBM Research Report RC 12484 (Log #56157) 2/6/87, 1987, IBM T.J. Watson Research Center, PO Box 218, Yorktown Heights, NY 10598 Compared human recognition accuracy, both by original writer and by another subject, with IBM's machine recognition In adaptive system, adding poorly-formed (atypical) prototypes from handwriting to system is a problem, as accuracy goes down In adaptive system, accuracy goes down as prototypes are added beyond a certain point (so why add them? when to add them?) Handwriting interaction tested for four tasks: copy, compose, spreadsheet, delayed copy/transcription Human recognition accuracy measured as 88.5% to 96.1%, average 93.2% Writing speed for hand-printing is 0.99 characters/second 90 to 111.2 prototypes in adaptive system for individual subject's handwriting Subjects consistently tended to underestimate recognition accuracy of IBM system, because errors in words and inability to correct were so annoying



96% recognition accuracy "not good enough" for commercial use of on-line character recognition [ScottWC87a] Scott, Walter C. "Signature Verification", United States Patent 4,701,960, October 20, 1987, assigned to Texas Instruments Incorporated, Dallas, Texas. Signature verification using matches at local extrema points(?) and something like elastic matching [Sejnowski87] Sejnowski, Terrance and Rosenberg, Charles "Parallel Networks that Learn to Pronounce English Text", Complex Systems, 1:145-168, (1987) Hinton diagrams; training input consisted of phonetic transcriptions of English text, not actual spoken recording. Cited by Marlin Eller, Microsoft Pen Computing group [Sekita88] Sekita, I., Toraichi, K., Mori, R., Yamamoto, K, and Yamada, H "Feature extraction of hand-written Japanese characters by spline functions for relaxation", Pattern Recognition, Vol 21 No 1, January 1988, pp 9-17 Cited in Yu90: for line thinning with context by chain codes [SemioticsDesigners87a] Semiotics Designers "Handwriting recognition product outline: Correspondence file", 2980 Salem Drive, Santa Clara, California 95051, 1987 Carol Anne Ogdin, formerly Mr. Ogdin Lists of sentences using all 26 letters of alphabet in various combinations, for collecting writing samples Sample list of sentences using all 26 letters of the alphabet -source unknown [SherrS88a] Sherr, Sol (Ed.) "Input Devices", Academic Press Inc., 1988, ISBN 0-12-639970-0 (electronic book) Chapter 1, "Introduction to Input Devices", Caswell, N.S. Chapter 2, "Human Factors Considerations in the Design and Selection of Computer Input Devices", Arnaut, Lynn Y. and Greenstein, Joel S. Chapter 3, "Keyboards", Greenstein, Joel S. and Muto, William H. Chapter 4, "Digitizer and Input Tablets", Davies, Thomas E., Mathews, H. Gerard, and Smith, Paul D. Chapter 5, "Mice", Goy, Carl Chapter 6, "Trackballs and Joysticks", Doran, David Chapter 7, "Voice Input Systems", Viglione, Sam S. See also citations for each chapter. Windows input managers / window management, virtual devices. [Shojima87] Shojima, H., Kuzunuki, J. S., and Hirasawa, K. "On-line pattern recognition for hand-written shapes", United States Patent 4,653,107, March 24, 1987, assigned to Hitachi, Limited, Tokyo, Japan. Re-order strokes, feature is quantized curvature, try deleting endsegments or combining sequential strokes On-line handwriting recognition using a dictionary of templates: used in a document preparation userinterface? [ShojimaH88a] Shojima, H., Mifune, T., Mori, J. and Kuzunuki, S. "Method and Apparatus for On-line Recognizing Handwritten Patterns", United States Patent 4,718,103, January 5, 1988, assigned to Hitachi, Limited, Tokyo, Japan. Cites DP Matching Process to Character Recognition of Nikkei Electronics, 1983. Angular vector chain code sequences: matching successive angle differences for line segments in handwriting character recognition; Unanticipated/intermittent input/event causes software system error [SilversteinA88a] Silverstein, Alan; McMahon, Bill; and Nuss, Greg "Adding Access Control Lists to Unix; or, How to Stop Worrying About All that Other Security Stuff and Love UNIX DACLMs", Hewlett-Packard Company, March 12, 1988 (submitted to /usr/group?) cited in various patents: Discussion of practical aspects of different ways of implementing Access Control Lists ACLs in Unix, combinging with traditional access bits. Cites to earlier work at HP since January 1987, an to the National Computer Security Center. Trusted computer base TCB of computer system. Discretionary Access control DAC is right to pass access rights/permission to others. Unix chmod bits as a form of DAC. [Sinha88] Sinha, R.M.K. and Prasada, B. "Visual Text Recognition Through Contextual Processing", Pattern Recognition, Vol 21 No 5, 1988, pp 463-479 Spelling context dictionary where words outside the dictionary are handled with a Viterbi algorithm A priori context (i.e. statistical values for letter pair digrams) vs a posteriori context (checking results against a spelling dictionary) Contains quick overview of everybody's work on spelling or statistical probability context correction Transient dictionary for context: words not in regular Uses weighted confusion matrix of touching/rejected characters for context correction (segmentation) Refers to ambiguous upper/lower-case confusion pairs, other than the obvious In OCR, attempt to fix touching characters by a separate segmentation algorithm on rejected characters common "omitted" words can be used to correct the rest dictionary may occur frequently in document, so most [SklarewR88a] Sklarew, Ralph "Handwritten keyboard-less entry computer system", European Patent 254561a, 1 Janurary 27, 1988, assigned to Linus Technologies, inc. Broad patent on Linus' keyboardless computer product, using handwriting recognition Gives many example screens for Linus/Grid's user interface, showing pop-up windows/rectangles for handwriting entry Gives figures for handwriting digitizer tablet performance: 0.015"/0.005", 150 points/second Linus/Grid userinterface on editing functions/symbols, appear similar to gesture-based user interface [Skylight88] Skylight Software, Incorporated "Handprints 2.0 product information", 323 Andover Street, Wilmington, Massachusetts 01887



Yuri Litvin's small on-line handwriting recognition software product: runs in 13Kbytes (!) [SmithR87a] Smith, Randall "Experiences with the Alternate Reality Kit: An Example of the Tension between Literalism and Magic", IEEE Computer Graphics, Sep. 1987 vol 7 no 9, pp. 42-50 See also video of Alternate Reality Kit ARK: mouse-hand can manipulate literal ("realistic" but 2-D) objects with momentum, bouncing, etc. Other "magical" features such as buttons, changes to simulated gravity, etc. Suggests that multiple mouse buttons (by extension, mouse gestures?) are hard to learn because they are unnatural. [SmithRW87] Smith, R.W. "Computer processing of line images: A survey", Pattern Recognition, Vol 20, 1987, pp 7-15 Cited in Taxt90 [SpeechSystems87] Speech Systems, Incorporated "Phonetic Engine (R) product description", 18356 Oxnard Street, Tarzana, California 91356, 1987 Product literature on phonetic speech recognition system with large dictionary. Phonetic Decoder (TM), Phonetic Profiler (TM), DS100 application development system, [Srihari87a] Srihari, Sagur N. and Hull, Jonathan J. "System to Achieve Automatic Recognition of Linguistic Strings", United States Patent 4,654,875, assigned to Research Foundation of State University of New York, Albany, N.Y., March 31, 1987 Automatic language (actually, character) recognition by converting words to bit strings, and comparing resulting strings to the best match of similar bit strings: stated as applicable to OCR [Srihari87b] Srihari, Sagur N. and Bozinovic, Radmilo M. "A Multi-Level Perception Approach to Reading Cursive Script", Artificial Intelligence, Vol. 33 No 2, p. 217-255, 1987 Cursive script recognition [StefikM87a] Stefik, Mark; Foster, Gregg; Bobrow, Daniel G.; Kahn, Kenneth; Lanning, Stan; and Suchman, Lucy "Beyond the chalkboard Computer: Support for collaboration and problem solving in meetings", CACM, Vol 30 No 1, January 1987, pp. 32-47 Cognoter/Colab: Xerox/PARC whiteboard/teleconferencing system supporting WYSIWIS what you see is what I see, users operated concurrently/in parallel on shared objects. Mouse or pen graphical input. Cites to Wang Freestyle. [StevensMJ88a] Stevens, M.J. "A Text Editor Driven by Hand-Drawn Symbols", NPL Report DITC 124/88, 1988 Cited in FordDM91a regarding gestures [StollC88a] Stoll, Clifford "The Cuckoo's Egg: Tracking a Spy Through the Maze of Computer Espionage", Pocket Books, ISBM 0743411463 Discrepancy of 75 cents in a billing program for internet usage led SysAdmin to first public uncovering of an attacker/hacker group is fraudulent [Suen88] Suen, C.Y. (chair) "Future Challenges in Handwriting and Computer Applications", panel discussion, 3rd International Symposium on Handwriting and Computer Applications, Montreal, May 29, 1987. Panelists: R. Plamondon, C. Tappert, A.J.W.M. Thomassen, J.R. Ward, K. Yamamoto. Summary published January 18, 1988, Prof. Ching Y Suen, Concordia University, Montreal, Canada. Comments included why most handwriting character recognition accuracy reports are "inaccurate" [Summagraphics87] Summagraphics "Digitizer technology paves the way for portable systems", Computer Design, April 1, 1987, page 9 Summagraphics' low-cost charge-ratio digitizer tablet [TaguchiY87a] Taguchi, Yoshinori and Yamanami, Tsuguya "Position Detecting Device", United States patent 4,704,501, November 3, 1987 Electromagnetic grid digitizer, active stylus [Tamura88] Tamura, S. and Kawasaki, S. "Recognition of Sign Language Motion Images", Pattern Recognition, Vol 21 No 4, 1988, pp 343-353 Cheremes - like phonemes or graphemes, but for (hand) sign language, like American Standard sign language [TannerPP87a] Tanner, Peter P. "Multi-Thread Input", Computer Graphics, Vol 21 No 2, April 1987, pp 142-145 User interface discussion of having multiple input devices / multiple streams of input: relates to problem of handwriting combining both position AND command/text input, as in gesture recognition? [TappertCC87a] Tapper, C.C., Suen, C.Y., Wakahara, T. "Research Report: On-Line Handwriting Recognition - a Survey", Research Report RC 14045 (#59748) 12/8/87, IBM Research Division, T.J. Watson Research Center, Yorktown Heights, New York Report on state-of-the-art in handwriting recognition, pen-computing, gestures, curve-matching, elastic matching, etc. as of about 1986 Compare with 1990 publication: Tappert90c [TappertCC87b] Tappert, C.C., Fox, A.S., Kim, J., Levy, S.E., and Zimmerman, L.L. "Handwriting recognition on transparent tablet over Flat Display", Proc. of SID, Vol 28 No 1, 1987, pp 67-73. Also Society for Information Display Digest of Technical Papers, Vol XVII, pp 308-312, May 1986



Cited in Rhyne86: Scriptel Digitizer Rhyne86 cites other version on visual parallax for electronic ink: 0.06" lateral shift causes subject's discomfort. Tappert describes also slippery surface. Cites internal Pencept report by Phillips and Prentice. [TappertCC88a] Tappert, Charles C. "Recognition System for Run-on Handwritten Characters", United States Patent 4,731,857, March 15, 1988, assigned to International Business Machines Corporation, Armonk, New York. Character stroke segmentation/parsing using an iterative approach to find the best aggregate recognition score for the whole line, or for whole word. Recognition method for run-on handwriting repeating receiving, examining and gradings steps until all strokes have been processed and recognized. [TappertCC88b] Tappert, C.C. "A Divide-and-Conquer Cursive Script Recognizer", IBM Res. Rep. RC14070, Oct. 1988 Cited by Marlin Eller, Microsoft Pen Computing group [Tektronix87a] Tektronix "Tektronix Smalltalk Users Manual", Tektronix, Part No. 061-3440-00 1987 Smalltalk language and graphical user interface: describes draw-through or drag gesture for selecting text, in addition to single/double/triple click to select words, sentences, etc. Selection for click also depends on where you click: in middle of word or at start of line. [TexasInstruments87a] Texas Instruments "Texas Instruments Business Edge Guidebook", Texas Instruments, 1987 Hand-held calculator with PowerTouch display, five touchpad buttons whose labels and functions change to set different functions when user hits SCROLL key. File contains additional materials. Also sold as Radio Shack EC-5500. [TexasInstruments88a] Datamath Calculator Museum "Texas Instruments Business Edge Guidebook", Texas Instruments, 1988. datamath.org/Sci/Modern/BusinessEdge.htm Hand-held calculator with touchscreen display, shows labels of functions under function key spots on display. Feature also found in TI-81 (variable function keys), Texas Instruments Personal Banker. [TransImage87] TransImage Corporation "Transimage 1000", hand-held OCR data-entry system for IBM PC XT/AT product announcement, 910 Benicia Avenue, Sunnyvale, California 94086, 1987 Hand-held optical scanner for OCR: M 68000 processor board for IBM PC XT/AT (compare Pencept and CIC) [TsaiRY87a] Tsai, Roger Y. "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-shelf TV Cameras and Lenses", IEEE Journal of Robotics and Automation, Vol RA-3, No 4, August 1987, pp. 323-344 Mathematical method for calibrating a camera (estimating distance to object, location of object, pincushion or other well-behaved distortion of lens) in 3D space using views from a single camera. Uses known 2D target pattern with known points. Contrast with structure-through-motion. [TsudaI88a] Tsuda, I., Shimizu, H., and Hibino, K. "Method and Apparatus of Recognition", United States Patent 4,760,603, July 26, 1988, assigned to Research Development Corporation of Japan, Tokyo, Japan. Strange patent on recognition by looking at oscillation frequencies of hypothetical components stretched between segments of the outline image [Tsugei87] Tsugei, Shinji and Iguchi, Shigeki "Handwritten Character Input Device", United States Patent 4,656,317, April 7, 1987, assigned to Sharp Kabushiki Kaisha, Osaka, Japan. Handwriting recognition for processing billing receipts on-line: patent on any kind of forms input? [UngarD87] Ungar, David and Smith, Randall B. "Self: The Power of Simplicity", OOPSLA '87, pp. 227-242 Description of SELF programming language: prototype-based object-oriented language: prototypes, slots, and behavior. Prototypes combine inheritance and instantiation. Walter Smith said this was the base for NewtonScript. Cites to Smalltalk. No discussion of SDK. [Viglione88] Viglione, Sam S. "Voice input systems", Computer graphics: technology and applications, Academic Press, Boston, 1988 ISBN 0126399700 Cited in LCS/Telegraphics case vs. Schumer for input devices like virtual tablets [Wacom88a] Wacom "User Manual for SD-510 A5 Type Digitizer -- Provisional", Wacom, July 1988 Wacom small-format tablet digitizer, Summagraphics Bit Map Two emulation. Full list of Bit Pad Two sommands (Summagraphics). Proximity is Z height at which coordinates are not correct. Batteryless / nobattery stylus and cursor (puck). [Wada87] Wada, Y., Kobayashi, Y., and Mitsuta, T. "Graphic data design method", European Patent 218246, April 15, 1984, assigned to Hitachi, Limited, Tokyo, Japan. User interface patent on design rule checking (?) drawing constraints on CAD input (?) [WangA87] Wang, A., Ho S., and Mainemer, C.I. "Writing Pad", United States Patent 4,638,118, January 20, 1987, assigned to Wang Laboratories, Inc., Lowell, Massachusetts. Dr. An Wang's digitizer: force/pressure-sensitive digitizer using very thin grid of contact lines, as for LCD's. Force/pressure-sensitive digitizer using coarse, then fine position sensing of X/Y grid. Touch surface has special zones for special computer codes (commands).



[WangLaboratories88a] Wang Laboratories "Video: Wang Freestyle Demonstration", Available on YouTube: video link to SIGGRAPH videos from www.billbuxton.com Want Freestyle desktop stylus/handwriting system, documents may be annotated with dynamic electronic ink and real-time voice recording.. No handwriting recognition, extensive use of direct manipulation gestures. Dwell/press-and-hold. [WangPSP88] Wang, P.S.P. "Knowledge Pattern Representation of Chinese Characters", International Journal of Pattern Recognition and Artificial Intelligence, Vol 2 No 1, 1988, pp 161-179 Not really about a Chinese/Kanji recognition system, but that Chinese characters are complicated (lots of strokes) because they are like English words (lots of strokes from lots of characters), yet are superior in many ways to Roman/Latin alphabets, and keyboard input technolgies for Chinese ideograms have been worked out. [Ward87a] Ward, J. "Design Criteria for and Electronic Tablet Technology with Acceptable Performance for Handwriting Capture and Analysis", Proceedings of the Third International Symposium on Handwriting and Computer Applications, Montreal, Canada, July 20-23, 1987, pp 178-180 [Ward87b] Ward, J. "Issues in the validity of testing protocols and criteria for on-line recognition of handwritten text", Proceedings of the Third International Symposium on Handwriting and Computer Applications, Montreal, Canada, July 20-23, 1987, pp 67-70 [Ward87c] Ward, J. and Phillips, M. "Digitizer Technology: Performance Characteristics and the Effects on the User Interface", IEEE Computer Graphics and Applications, April 1987, pp 31-44 Basic tutorial on tablet digitizer / touchscreen performance and errors: Gaussian noise, wild data, slew rate errors. Mutability of tablet performance and "specsmanship" on specifications. Overview of touchscreens and tablets: grid/sheet, electromagnetic/EMR, electrostatic/capacitive, resistive, sonic. Missing coordinates. Tilt error. Mechanical slop. [Ward87d] Ward, J. (organizer): Sibert John (Moderator): Buffa, Michael G., Crane, Hewitt D., Doster, Wolfgang, and Rhyne, James "Issues Limiting the Acceptance of User Interfaces Using Gesture Input and Handwriting Character Recognition", panel discussion, Proceedings of CHI+GI Conference on Human Factors in Computing Systems and Graphics Interface, Toronto, April 5-9, 1987, pp 155-158 Much work on handwriting character recognition is very over-stated Participants from Nestor, SRI International, CIC, AEG Research Institute, IBM Thomas J. Watson Research Center, Pencept [Ward87e] Ward, J. "Combinatorial Aspects of Variability in Handprinted Text", Proceedings of the Third International Symposium on Handwriting and Computer Applications, Montreal, Canada, July 20-23, 1987, pp 6063 [Ward88] Ward, J. and Kuklinski, T. "A Model for Variability Effects in Hand-printing, with Implications for the Design of On-line Character Recognition Systems", IEEE Trans. Systems, Man, and Cybernetics, Vol 18 No 3, May 1988, pp 438-450 Generative model for shape variations in handprinted and handwritten characters, based on stroke order, angle, hooks, retracing, and a modification of chain codes using local extrema, for on-line handwriting character recognition [WareC87a] Ware, Colin and Mikaelian, Haratune H. "An Evaluation of an Eye Tracker as a Device for Computer Input", Proc. CHI + GI 98, pp. 183..188 Eye tracker better than mouse or tablet as pointing device, because do not need to find location on screen, then move eyes to mouse, then back. (Not sure about that!) Mentions dwell time for selection / mouse click: compare with press-and-hold. Gulf and Western series 1900 eye view monitoring system, determines boundary of pupil. [WareC88a] Ware, Colin and Jessome, D. "Using the Bat: A Six-Dimensional Mouse for Object Placement", Graphics Interface '88, 1988, pp. 119..124 Bat: Using Polhemus Isotrack 6 DOF three-dimensional electromagnetic digitzer as input device for 3D placement: uses relative motion / relative displacement (like a mouse) instead of absolute position, so distortions from nearby metallic objects etc. no a factor. [WeingartSH87a] Weingart, Steve H. "Physical Security for the µABYSS System", Proc. 1987 IEEE Symp. on Security and Privacy, April 27..29, 1987, Oakland, CA, pp. 52..58 Physical security of processors: sealed processor and memory, basically booby-trapped (embedded wires, short out if package drilled, break if opened etc.) if tampered with physically. Notes that it does not address freezing of CMOS memory. [WelbournLK88a] Welbourn, L.K. and Whitrow, R.J. "A gesture based text editor", Proceedings of the British Computer Society, pp 363-371, Cambridge University Press, 1988 Cited in Goodisman91, FordDM91a [WhiteSR87a] White, Steve R. and Comerford, Liam "ABYSS: A Trusted Architecture for Software Protection", Proc. 1987 Symp. Security and Privacy, Oakland, CA, Apr. 27-29, 1987, p. 38..51 Software copy protection / rights authorization based on passing rights-to-execute vi a token. Enforcement



by protected/secure (logically, physically, procedurally secure) processors, encrypted/secure communications between secure processors. Rights-to-execute can be time-limited, dependent on callers, etc. Use count enforced by limited number of use-once rights-to-execute. [Whitrow87] Whitrow, R. and Higgins, C. "The Application of n-grams for Script Recognition", Proceedings of the 3rd Symposium on Handwriting and Computer Applications. Montreal, Canada, 1987 Cited by Marlin Eller, Microsoft Pen Computing group [Williford88] Williford, John D. "Automatic Reference Adaptation During Dynamic Signature Verification", United States Patent 4,724,542, February 9, 1988, assigned to International Business Machines Corporation, Armonk, New York. Continuous adaptive signature verification recognition: samples are updated automatically over time [Wolf87a] Wolf, Catherine G. and Morrel-Samuels, Palmer "The Use of Hand-drawn Gestures for Text-editing", International Journal of Man-Machine Studies, Vol 27 No 1, July 1987, pp 91-102, also available as IBM Report RC 12523 Paper and pencil study of gesture user-interface for editing (proofreader's marks) for user consistency, choice of symbols [Wolf87b] Wolf, Cathy G. "A comparative Evaluation of Gesture and Conventional Interfaces", IBM Research Division Human Factors research report RC13187 (#58999), October 9, 1987 Cited in Blatt88 gesture user interface, single stroke gesture for spreadsheets, with text and pointing recognition [Wolf88] Wolf, Catherine G. "A Comparative Study of Gestural and Keyboard Interfaces", Proceedings of the Human Factors Society 32nd Annual Meeting, 1988, pp 273-276 Gesture user interface on spreadsheets: 72% faster than keyboard alone, preferred by users, using electronic ink integrated tablet/display: shows "preferred" gesture symbols [Xerox87a] Xerox Corporation, Vista Laboratory "Analyst V1.2t Reference Manual", Xerox Special Information Systems, Vista Laboratory, 1987 popup/pop-up menus on three-button mouse for various functions on single-click/double-click. Pop-up menu is context sensitive, depending on what is clicked on and whether there is a current selection. Implemented in Smalltalk-80. Draw-through for drag selection of text. [Yamakawa87] Yamakawa, Tadashi "Character and Figure Processing Apparatus", United States Patent 4,672,677, June 9, 1987, assigned to Canon Kabushiki Kaisha, Tokyo, Japan. User-interface combination of handwriting recognition, and simulated touch-button areas on tablet for Chinese/Kanji word processing Shows a menu of multiple candidate character to select from when the online recognizer for handwritten character isn't sure. [Yoshida88] Yoshida, K., Shimizu, H., and Watari, Massao "Continuous Characters Recognition System", United States Patent 4,764,972, August 16, 1988, assigned to NEC Corporation, Tokyo, Japan. On-line character recognition using inter-stroke handwriting strokes Refers to changes in separate handwritten character variations depending on inter-stroke motion (hooks, continuation marks, etc.) Refers to partially-connected (script-like) writing as continuous writing (don't stop between characters) [ZimmermanTG87a] Zimmerman, T.G., et al "A hand gesture interface device", Proceedings of the ACM CHI + GI '87 Conference on Human Factors in Computing Systems and Graphics Interface, Toronto, Canada, April 5-9, 1987. Association for Computing Machinery, New York, 1987, pp 189-196 Cited in Nielsen90 Z-glove, Date-Glove, Data-Glove using ultrasonic/acoustic position sensors for threedimensional digitizing. Alternative input was magnetic sensing of small magnets on a glove. [ZingherO87a] Zingher, Oded "Data Input Unit and Method for Printing Machines", United States Patent 4,639,881, January 27, 1987 Transparent matrix switch / touch screen, using transparent ink (gold, etc.) on two sheets of mylar in row/column geometry. biblio86 (Prev page)

Main Page







Main Page


[ATT90a] AT&T "UNIX System V Release 4 Programmer's Guide: XWIN Graphical Windowing System Xlib-C Language Interface", Prentice Hall, 1990 Administrative / programming guide for XWIN/X-Windows windowing system on Unix. ACL access control list for network connection, which system programs may be run from. Access rights/permissiongs can be conditioned also on authorization name and data received at server end of connection. [AbadiM90a] Abadi, Martin; Burrows, Mike; Kaufman, Charlie; and Lampson, Butler "Authentication and Delegation with Smart-Cards", Technical Report SRC-RR0-67, 1990; Proc. TACS '91 International Conference on Theoretical Aspects of Computer Software, September 24..27, 1991, Sendai, Japan, pp 326-345 Security and performance analysis of smart-card authentication protocols relying on public/private key encryption/decryption in the smartcard: neither smartcard PIN nor private key ever exposed to login station/computer. [Abramovitz89] Abramovitz, R.G., Anderson Jr., R.L., Busse, F.L., Celarier, S.A., Poublan, J-Y. H., and Shorthill, L.R. "Optical Character Reader", United States Patent 4,809,351, February 28, 1989, assigned to SABA Technologies, Incorporated, Beaverton, Oregon. Hardware design of a hand-held scanner for optical (typed) character recognition [AccessDevelopment90] Access Development Corporation "FormLink product literature", July, 1990. Address: 2000 Commonwealth Avenue, Auburndale, Massachusetts, 02166, 617-332-7200 Forms package, stylus-keyboard electronic-paper user interface, suitable for adding handwriting recognition [ActiveBook89] Active Book Ltd., Cambridge England "Masterstroke: the notebook computer", London Times, September 7, 1989, p 34 Portable touch-screen/handwriting product, with character recognition. Active Book: Dr. Herman Hauser, founder: Olivetti financing [AdolfJ89a] Adolf, Jurine "An Expert System for the Selection of Input Devices", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 335-339 Select input/pointing devices -- Says on-line recognition inferior to almost all other user-interfaces for text input (comparison with Linus)



[AdvancedProducts89] Advanced Products and Technologies "Voice Computer product announcement", IEEE Computer, September 1989, p 98 Portable voice (!) recognition and output computer product [Agilis89a] Agilis Corporation "Product information on the Agilis System (TM) of Handheld Workstations", June 12, 1989. 1101 San Antonio Road, Mountain View, California 94043-1008 Portable touch-screen/handwriting product, but no character recognition or electronic ink user-interface: radio net, Unix/OS-2/DOS File includes additional material: BYTE magazine, "Agilis Hand-Held Workstations: Computing Power in the Field" Aug 1989. [Agilis89b] Agilis Corporation "Agilis Hand-Held workstations: Computing Power in the Field", BYTE Magazine, August, 1989 Portable touch-screen/handwriting product, but no character recognition or electronic ink user-interface: radio net, Unix/OS-2/DOS File includes additional material: BYTE magazine, "Agilis Hand-Held Workstations: Computing Power in the Field" Aug 1989. Hand-held Microsoft PC with modular disk drive module/slice, touchscreen/tablet display, battery packs, etc. [AkihikoM90a] Akihiko, Matsuo; Masume, Hanaeda; nd Atsuko, Tokuhara "Window Display System for Multiwindow System", Japanese Patent JPH03288891 (a), April 5, 1990 (Machine translation to English) Translucent/semi-transparent windows so that user can see objects underneath. [Apple89a] Apple Computer, Inc. "HyperCard User's Guide", Apple Computer, Copyright 1989, Cupertino, CA (hardcopy book) Basic reference for Hypertalk: new buttons may be defined to auto-highlight, includes lasso gesture for graphical editing [Apple89b] Apple Computer, Inc. "HyperTalk Beginner's Guide: An Introduction to Scripting", Apple Computer, Copyright 1989, Cupertino, CA (Hardcopy book) Basic reference for Hypertalk: describes tool palettes for buttons, text, selection: lasso selection gesture triggered by tool palette button [Apple90a] Apple Computer, Inc. "MacIntosh Reference", Apple Computer, Copyright 1990, Cupertino, CA (hardcopy book) Reference manual/user's guide for MacIntosh computer: CloseView screen magnifier for accessibility [Apple90b] Apple Computer, Inc. "Getting Started with your MacIntosh", Apple Computer, Copyright 1990, Cupertino, CA (hardcopy book) Reference manual/user's guide for MacIntosh computer: desk user interface, explains that "press" does not mean press on the mouse, it means press the mouse button [Apple90c] Apple Computer, Inc. "Getting Started with HyperCard IIGS", Apple Computer, Copyright 1990, Cupertino, CA Reference/tutorial for HyperCard: hyperlinked document cards. Examples of IDE/SDK. Mentions HyperTalk scripting language. [Apple90d] Apple Computer, Inc. "HyperTalk Beginner's Guide for the Apple IIgs", Apple Computer, Copyright 1990, Cupertino, CA Reference/tutorial for HyperTalk: scripting language for HyperCard with hyperlinks. Scripting language similarities to JavaScript (on mouseUp, etc.) [AppleProgrammersAndDevelopersAssociation89a] Apple Programmer's and Developer's Association "Hypercard Developer's Toolkit Version 1.0", Apple Programmer's and Developer's Association, 1989 Apple Hypercard documentation from users group. User interface visual effects (scroll or wipe/swipe). AppMaker Application Generator with GUI graphical user interface editor. [AppleProgrammersAndDevelopersAssociation89b] Apple Programmer's and Developer's Association "ResEdit Version 1.2B1", Apple Programmer's and Developer's Association, 1989 Apple Hypercard document from users group. Resource Editor IDE, shows resource built in a file, resource picker. Properties and attributes of individual resources can be edited. MacsBug 6.0 debugger. Includes LightSpeed C: GUI of build utility allows files to be bundled into segments graphically (partition line) for both source and library files (resources also?) [Atari90] Atari Corporation "Portfolio hand-held personal computer", Sunnyvale, California, 94089-1302, 1990 Hand-held (well, hand-holdable) clamshell computer, no touchscreen. MSDos 2.11. Includes calendar, diary, note taker, other PDA functionality. [AtkinsonR89a] Atkinson, Russ; Demers, Alan; Hauser, Carl; Jacobi, Christian; Kessler, Peter; and Weiser, Mark "Experiences Creating Portable Cedar", Xerox PARC Palo Alto Research Center, Report CSL-89-8, June 1989. Also Proc. SIGPLAN '89, Portland Oregon, June 21-23, 1989 Portable compilation of Cedar (successor to Smalltalk) by compiling to "C" and using native "C" compiler, plus implementing part of Cedar in the ported subset. X-Windows client. [Autosig90] Autosig Systems Inc. "Sign/on product information", 1990, PO Box 16050, Irving Texas 75016, 214-



258-8033 Formerly Signify / Sign-on: signature verification product [BarkerBA89] Barker, Barbara A., Hernandez, Irene H., and Machart, Beverly H. "In-line dynamic editing of text or graphics", United States Patent 4,815,029, March 21, 1989, assigned to IBM Corporation. User interface combining free-hand drawing, graphical editing, similar to "Polite" product system by Sheldon Borkin and John Prager, or Xerox Star and Apple MacIntosh with generic functions valid for all types of objects On-line dynamic editing of text or graphics in computer document executing actions on user-selected attributes based on internal knowledge of object type rather than by explicit commands [BarkerBA90] Barker, Barbara A.; Hernandez, Irene H.; Machart, Beverly H. "Method for Modifying Intermingled Text Object and Graphic object within an Object Set Individually or Correspondingly", United States Patent 4,974,194, November 27, 1980 User-interface for stylus/gesture editing of text, using mark-up gestures [BarkerPG89] Barker, P.G. and Manji, K.A. "Pictorial dialogue methods", International Journal of Man-Machine Studies, Vol 31 No 3, pp 323-347, September 1989 User-interface for electronic books: multi-media pointing a la hypertext with pictures [BaumE89a] Baum, E. and Haussler, D. "What size net gives valid generalization?", in Advances in Neural Information Processing Systems, D.S. Touretzky, editor, Morgan Kaufman, 1989 Cited in MartinGL90b Trainable feedforward/back-propogation neural networks, mathematical derivation of size network needed to classify from a size of training sample set with particular expected error bounds. Notes however that this presumes the training set is not biased / is representative. [BeardMH90a] Beard, Marian H.; Caro, Perry A.; Hsiao, Jennifer B.; Mackey, Kevin J.; Sandman, James G.; Steinbach, Gary R.; and Wood, Donald R. "Virtual and emulated objects for use in the user interface of a display screen of a display processor", United States Patent 4,939,507, July 3, 1990 Simulated devices (PCs with simulated/virtual floppies, BIOS etc. etc.) also simulated/emulated virtual printers, etc. in a GUI desktop that is not a PC/DOS (Xerox). Includes a pop-up virtual keyboard that can be moved on the display. Mouse or (other) cursor control device (e.g. touchscreen). [BegejS90a] Begej, Steve "Glove Controller With Force and Tactile Feedback For Dexterous Robotic Hands", Phase I SBIR Final Report, 14 September, 1990, NAS 918308 Prototype for data-glove-like hand controller with tactile/haptic feedback to fingertips and joints. Intended as telepresence (compare virtual reality) controller, remote robotic control. Sensors for joint angles and joint force, 37-taxel tactile touch sensor array in fingertip, tactile display is inflatable (air) fingertip. [BeringerDB89a] Beringer, Dennis B. and Bowman, Mary James "Operator Behavioral Biases using High-resolution Touch Input Devices", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 320-322 Nothing study of visual parallax and targetting error in integrated tablet/display: 3/4mm error on their hardware at 17 degrees viewing angle: software offset only good for one user in one position, so they suggest letting the user just get used to visual parallax [BierEA90a] Bier, Eric A. "Snap-Dragging in in Three Dimensions", ACM SIGGRAPH Computer Graphics, Vol 24 No 2, Mar 1990, pp. 193-204. Proc. I3D '90 Symp. on Interactive 3D Graphics Snap to grid vs. heuristic constraints in three dimensions 3-D similar to compass and ruler for pointing and positioning, in technical illustration system. Gravity, alignment objects (lines, angles, spheres, etc.), smooth-motion affines. Snap-to-grid in three dimensions in 2-D user interface. [BierEA90b] Bier, Eric A. and Goodisman, Aaron "Documents as User Interfaces", Proc. Int'l. Conf. on Electronic Publishing, Document Manipulation and Typography, 199, pp. 249-262 Earliest paper on EmbeddedButtons. Mentions navigation buttons embedded in electronic document (buttons in TOC and in section headings), turn words into buttons that do strike-out or other function (translation/dictionary/spelling-change) when clicked. Buttons also embedded in multimedia documents. Video demonstration. [BierEA90c] Bier, Eric A., Goodisman, Aaron; Wyatt, Doug; ndPier, Ken "Documents as User Interfaces: Video", Xerox PARC demonstration video for EmbeddedButtons, 1990, Available on YouTube.com Demonstration video for EmbeddedButtons: portions of text in a document automatically made active and clickable: e.g. bring up dictionary definition, show a pop-up menu, etc. Tioga text editor, Gargoyle graphics editor, Documents as Control Panels. To accompany 1991 SIGCHI paper. Compare with Visual Basic, PenApps? [BilboMK90a] Bilbo, Mark K. "Using MacWrite: Covers MacWrite II", Que Corporation, 1990 (hardcopy book) MacWrite II word processing / electronic publishing. Find/Replace uses wildcards, then presents user multiple change actions [BlattLA89a] Blatt, Louis A. "Human Factors Support of Gesture Recognition", research report June 16, 1989, Wang Laboratories, 1 Industrial Avenue, Lowell, Massachusetts 01851 Internal Wang report on variations in chain code writing sequences for handwritten alphabetic characters



[BlattLA89b] Blatt, Louis A. "Spreadsheet Gesture Set", research report February 2, 1989, Wang Laboratories, 1 Industrial Avenue, Lowell, Massachusetts 01851 Internal Wang report on what editing marks / gestures subjects used informally on spreadsheets [BlattLA90a] Blatt, Louis A., and Schell, Alan "Gesture Set Economics for Text and Spreadsheet Editors", manuscript, March 3, 1990, Wang Laboratories, 1 Industrial Avenue, Lowell, Massachusetts 01851 Gesture recognition UI study on "most natural" gestures for text and spreadsheet editing, resulted in proofreader's marks [BloodEB89a] Blood, Ernest B. "Device for quantitatively measuring the relative position and orientation of two bodies in the presence of metals utilizing direct current magnetic fields", United States Patent 4,849,692, July 18, 1989 Ascension Technology Corporation: electromagnetic 3D/6DOF position sensing (tablet) using multiple receiving antennas (orthogonally at right angles) and DC direct current magnetic fiel from multiple transmitters. Subtract out Earth's magnetic field. Vague about the exact computation of position: based on direction inferred from relative signal strengths along three axis, and distance inferred from absolute signal strength w.r.t. earth magnetic field as reference? Says with DC signals, no need for calibration (?), no interference from electromagnetic fields (?), but affected by nearby magnetic object. [BloodEB90a] Blood, Ernest B. "Device for quantitatively measuring the relative position and orientation of two bodies in the presence of metals utilizing direct current magnetic fields", United States Patent 4,945,305, July 31, 1990 Ascension Technology Corporation: electromagnetic 3D/6DOF position sensing (tablet) using multiple receiving antennas (orthogonally at right angles) and DC direct current magnetic fiel from multiple transmitters. Subtract out Earth's magnetic field. Vague about the exact computation of position: based on direction inferred from relative signal strengths along three axis, and distance inferred from absolute signal strength w.r.t. earth magnetic field as reference? Says with DC signals, no need for calibration (?), no interference from electromagnetic fields (?), but affected by nearby magnetic object. [BlySA90a] Bly, Sara A. and Minneman, Scott L. "Commune: A Shared Drawing Surface", Proc. COCS '90 ACM SIGOIS and IEEE CS TC-OA conference on Office information systems, Pages 184-192, ACM New York Shared writing/marking/drawing/gesture surface (whiteboard) with voice, transparent digitizer over display with single pen/stylus for each user. All user ink was same color, cursor "pens" different color for each user. Pointing by moving cursor without touching surface. Simultaneous by all users. Transparent digitizer not specified. Gestures were in-air for other users to see what speaker was pointing at, not gesture commands. (proximity sense digitizer: Scriptel?) Identify/differentiate users? Compare with Wang Freestyle? [BoieB89a] Boie, Bob; Mathews, Max; Schloss, Andy "The Radio Drum as a Synthesizer Controller", Proc. International Computer Music Conference (1989), pp. 42-45 Capacitive touch sensor: transmitting antennae in multiple drum sticks (e.g. multi-touch), receiver in flat drum sensors, different drum sticks use different frequencies. Z proximity sensing up to about 6 inches (compare with Scriptel). Bimanual/multi-touch [BoieRA89b] Boie, R. A.; Ruedisueli, L.W.; and Wagner, E.R. "Gesture Sensing via Capacitive Moments", Work Project No. 311401-(2099,2399), AT&T Bells Laboratories, 1989 See also Blonder 93 patent involving capacitive moments, and Radio Drum. Cited in Jaffe 93 essay on electronic instruments, further electronic music references. Some references give the date as 1991. [BorensteinNS90a] Borenstein, Nathaniel S. "Multimedia Applications Development with the Andrew Toolkit", Prentice-Hall, 1990, ISBN 0-13-036633-1 (hardcopy book) Mostly Dialog and other aspects of Andrew UI programming [BorgK90a] Borg, Kjell "iShell: A Visual UNIX Shell", Pro. Chi '90, April 1990, pp. 201-207 Visual interactive shell user interface for UNIX using gestures: Desktop UI using gestures: drag-and-drop document icons on on UI machines such as printer, mailer, fax, etc. Visual programming language iScript for defining actions. [BowersDevelopment89a] Bowers Development "AppMaker, the Application Generator", Bowers Development, Lincoln MA 1989 ThinkC Class Library object-oriented development system for the Apple Macintosh -- forerunner to FrameMaker? Shows UI for creating dialogs on targeted system. GUI IDE for cross-system development. Includes addendum for version 1.1 of Appmaker. GUI IDE allows for custom sliders. [BowersHL90a] Bowers, Harold L. "Interactive Template", United States Patent 4,933,514, July 12, 1990 (Printed patent miss-spells Interactive Template): Patent on CadKey fixed-menu command template to be put on top of a digitizer with command iput, commands being grouped on the template. Compare with other templates of the time, such as PenCad, Pencept template with recognition macros (gestures), and work by Doster and Oed. [BozinovicRM89a] Bozinovic, R.M. and Srihari, S.N. "Off-Line Cursive Script Word Recognition", IEEE Trans.



Pattern Analysis and Machine Intelligence", Vol 11 No 1, January 1989, pp 68-83 (abstract only) Off-line (vs on line) cursive handwriting recognition. Multi-level analysis of handwriting: points, contours, features, letters, and words Reference-line finding on handwriting, letter segmentation based on local minima or low vertical profiles Topological relationships of contours on handwriting for recognition Lists of features used for cursive handwriting recognition Chain encoding for off-line handwriting cursive script recognition Statistical and syntactic context rules for cursive handwritten words Testing done with two subjects told to do fully-connected script, and to maintain reasonable legibility Said 77% accuracy on whole words with context was good test results (!) Testing of off-line cursive script, but used data taken from tablet digitizer, so no scanning/pixel/gray problems Well-known tenet of pattern recognition practice that for problems with large feature space dimensionality, the sizes of the training and testing sets should be roughly the same Well-known tenet of pattern recognition practice that training and testing sets should not be the same to avoid overly optimistic results [BrooksFP90a] Brooks, Frederick P. Jr.; Ouh-Young, Ming; Batter, James, J. and Kilpatrick, P. Jerome "Project GROPE - Haptic Displays for Scientific Visualization", Computer Graphics, Vol 24 No 4, August 1990, pp. 177-185 Haptic output display visualization of molecules (electric field surface) and objects, with force-feedback device. GROPE-1 Device used developed by Margaret Minsky. [BrownE90a] Brown, E., Buxton, W., and Murtagh, K. "Windows on Tablets as a Means of Achieving Virtual Input Devices", in D. Diaper et al (Editors), "Human-Computer Interaction - INTERACT '90", Elsevier Science Publishers, pp. 675-681 See also http://www.billbuxton.com/windows.html Refers to dynamic windows, virtual devices on tablets, tools for defining regions flexibly. Use touch tablet to simulate keypads, mouse, other devices. Compare with Schumer? Mentions emulating multiple devices on multi-touch tablet (Lee '85) [BrownNR89a] Brown, Norman R. and Vosburgh, Ann Marie "Evaluating the Accuracy of a Large-Vocabulary Speech Recognition System", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 296-300 Recognition performance: simple recognition error (how many wrong): adjusted recognition error (how many wrong after removing errors system "not expected" to get right) out of vocabulary words (also removing errors attributed to bad input: tablet or voice input errors? Use of context can ADD errors and reduce recognition accuracy: "drag-along" errors where a "correction" due to context screws up subsequent inputs User interface and recognition %: a (speech) recognition system need very good / easy correction mechanisms for user to correct recognition errors [BuxtonWAS90a] Buxton, William A.S. "A Three-State Model of Graphical Input", Proc. INTERACT 90, pp 449-456 Modeling pointing (stylus, joystick, mouse, touchscreen, lightpen) user interfaces as transitions among three states: touch, proximity, out-of-range. Does not model multiple buttons directly. Tap is no motion in touch. Mentions timeout cue (no motion/dwell gesture) for pointing. Dwell combine with press-and-hold? [ByrneJB89a] Byrne, Jani Gabriel "Competitive Evaluation in Industry: Some Comments", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 423-425 Notes on comparing competitive commercial user interfaces [Byte89a] BYTE Magazine "BYTE Magazine: Annual Awards and digitizing Tablet", BYTE Magazine, Vol 14 no 1, January 1989 "Product Focus: Graphic Details" by Apiki, Steve and Diehl, Stanford. Includes review of Penpad 320 of Pencept: show character recognition CAD templates (AutoCad), controller board. [CIC89a] Communication Intelligence Corporation "Handwriter: Expanding the world of computing", CIC, 1989 Product and technology brochure on CIC Handwriter: ideographic writing, non-typiseds. Several examples of program templates for desk top publishing, none for CAD/CAM. Comparison of user interface on touch tablet with single touches, versus multiple keystrokes for spreadsheet programs Lotus/123 and Multiplan. [CIC90a] Communication Intelligence Corporation "Communication Intelligence Corporation: Handwriter Licensing program for Wang Laboratories", CIC, April 27 1990 Licensing program for CIC HandWriter handwriting/gesture recognition on tablet: compare with Pencept. [CIS89] CIS "Geometry Ball product information", CIS Group, Postfach 10 01 80, D-4060 Viersen 1, West Germany, 1989 6DOF Six-degree of freedom 3-D pointing mouse device: senses torque (yaw, pitch, roll) and direction of force (X/Y/Z), motionless joystick. [CalComp89a] CalComp "CalComp 2300 Series DrawingBoard Digitizers", CalComp Incorporated, 1990 CalComp 2300 digitizer tablet, stylus, cursor/puck. File includes brochure on CalComp graphic input devices [Canon89] Canon "Ai Note IN-3000 product announcement", CanonVoice, February, 1989 (in Japanese). Small, portable notebook-style handwriting input/recognition (PDA) product from Canon, 1980 [CardSK90a] Card, Stuart K.; Mackinlay, Jack D.; and Robertson, George G. "The Design Space of Input Devices", Proc. CHI '90, pp. 117-124 Comparison of user-interface features for tablets, touch pads, light pens, mouse, and other pointing input



devices: resolution, foot print, speed and error performance, accuracy. [CarpenterG89a] Carpenter, Gail and Grossberg, Stephen "Hierarchical neural approach", Boston University press release, 1989 Report of ongoing research for multi-layer neural networks with back-propogation [ChengF89] Cheng, F., Hsu, W., and Chen, C. "Fuzzy Approach to Solve the Recognition Problem of Handwritten Chinese Characters", Pattern Recognition, Vol 22 No 2, pp 133-141, 1989 Fuzzy set recognition applied to Chinese characters Most troublesome problem in Chinese handwriting is great variation among hand-prints Assumptions about statistical probability density in handwriting recognition are highly dubious It is impractical to deal with variations in handwriting with probability densities A similarity measure between strokes in Chinese character based on angle feature, not length Stroke length not a reliable feature in Chinese characters 96% recognition rate on 881 Chinese characters (160 variations each), once rejects are removed [ChowD90a] Chow, Doris and Kim, Joonki "Paper Like Interface for Educational Applications", Proceedings of National Educational Computing Conference, June 20-22, 1989, Boston, Massachusetts, pp. 337..344 Part of the paper-like interface PLI projects at IBM: electrostatic/capacitive digitizer (Scriptel) over LCD display, exploration of gesture input. Teaching block-printing to children by direct comparison with prototype shapes. States that gesture recognition requires a different recognition strategy. Crosswordpuzzle as teaching drill using handwriting recognition. Music editing demonstration modeled after Buxton, but not using same recognition technology. [ClarkAJ90a] Clark, Andrew J. "Do you really know who is using your system? - A Survey of Personal Authentication Techniques", Paper to be presented to the British Computer Society, Technology of Software Protection Specialist Group, February 1990 Review of authentication techniques: Something Known (password), Something Owned, Optically read cards, Smart cards, Password Generators, Personal Authenticators, Physical Characteristics -- Fingerprint, Retinal Scan, Voice Verification. Mentions *dynamic* signature verification as an "involuntary response", cites low hardware reliability of touch-screens: Dr. John Parks of the British Technology Group for signature verification products [ClarkD89b] Clark, D.; Demmel, J.; Hong, J.; Lafferriere, G.; Salkind, L. and Tan, X. "Teleoperation Experiments with a Utah/MIT Hand and a VPL DataGlove", NASA/JPL Space Telerobotics Conference, JPLPublication 89-7, Vol. V, January 1989, pp. 81..89 DataGlove with articulation sensors combined with Polhemus 3D digitizer sensor used for Waldo-like teleoperation of robotic hand. [ClarkDM89] Clark, D.M. "Pattern Recognition Method and Apparatus", United States Patent 4,805,225, February 14, 1989, assigned to The Research Foundation of the State University of New York, Albany, New York. Hierarchical network recognizer, similar to neural nets, to determine the presence of a feature (but how are features weighted?). Cites work on connectionist models for neural nets [ComputerBusinessReview89] Computer Business Review "IBM Announcements - Computer Business Review", Computer Business Review, Issued 1146, March 31, 1989 Report on IBM PLI "paper-like interface", and comparison with Quest Automation Datapad development in the United Kingdom: Quest Datapad 15 years previously (1975?) . [ComputerS89] Computer Shopper "Paper-like Interface -- IBM researchers are exploring a way to use computers in which people simply write on a flat surface", Computer Shopper, June 1989, page 169 Press announcement of IBM's electronic ink handwriting recognition product [CorrentDR89a] Corrent, Dean R. "Computer System User Interface", United States Patent 4,827,410 User interface for transparent digitizer or touchscreen which covers a display and also portions of working surface next to the display. Fixed Menus can be mounted outside the area of the display. [DFM89a] DFM Business Systems Inc. "DBF unveils 3.5-pound hand-held PC", Travelite 286 Product announcement, Info World, September 11, 1989, vol 11 no 37 [DFM89b] DFM Business Systems Inc. "Hand-held CPU for health care uses infrared-link", Travelite 286 Product announcement, Network World, Oct 30, 1989, v6 no 43 [DanisCM89a] Danis, Catalina M. "Developing Successful Speakers for an Automatic Speech Recognition System", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 301-304 Training users to adapt to a particular speech recognition: Problems are too clear at end of words, too mushy at start of words, no word separation, speak too fast Data base /testing: very important to try realistic conditions: realistic conditions include the fact that users adapt to the system [DannenbergRB89a] Dannenberg, R.B. and Amon, D. "A gesture based user interface prototyping system", Proc. UIST '89, 1989, pp 127-132 Slide-switch, two-finger (multi-touch) twist knob, click gestures. Hand-coded recognizer in prototype. Buttons: multitouch? Touchscreen/tablet was McAvinney Sensor Frame. Describes user interface system using object that are responsive to multiple, concurrent gestures (more than one finger); object is



responsive to gesture if point of (first) contact falls within its bounding box. Includes "graphical switch" and "graphical slide switch" to turn on/of. [DarraghJJ90a] Darragh, John J.; Witten, Ian H. and James, Mark L. "The Reactive Keyboard: A Predictive Typing Aid", IEEE Computer, November 1990, pp. 41-49 Graphical/virtual/synthetic keyboard on-screen keyboard on Apple MacIntosh, usable with mouse without physical keyboard. Also includes predictive type-ahead. One use is as accessibility technology, combined with dwell-mouse for handicapped use. [Datakech90a] DataKech, Incorporated "Scriptwriter product information", 15331 Manor Village Lane, Rockville, Maryland 20853 [Datapro89] Datapro Research, Manufacturing Information Systems Data Collection report #450 "Handwriting Recognition Systems", January 1989 Datapro report on Linus Write-Top and Data Entry Systems ScriptWriter handwriting recognition products. [DavisE89a] Davis, Elliot "Pattern Recognition Error Reduction System", United States Patent 4,802,231, January 31, 1989 Error reduction by matching patterns against additional templates of patterns otherwise mis-matched. [Denker89] Denker, J.S., Gardner, W.R., Graf, H.P, Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D., Baird, H.S., and Guyon I. "Neural network recognizer for Hand-written zip code digits", In Advances in neural information processing systems I, D.S. Touretzky, editor. Morgan Kaufman, 1989 Martin,GL90 [DestombesF89a] Destombes, F. "Virtual All-Points-Addressable Tactile Display", IBM Technical Disclosure Bulletin, May 1, 1989, Vol 89 No 5, pp. 135-136 Compare with optacon: small finger-sized display for tactile output. Refers to small display being movable over desktop surface, and sensing absolute position on the surface (such as through mesh/grid of wires: electromagnetic/electrostatic tablet?) [DominessyME89a] Dominessy, Mary E. "A Literature Review and Assessment of Touch Interactive Devices", Technical Note 11-89, AD-A217 668, U.S. Army Human Engineering Laboratory, Aberdeen Proving Ground, Md, October 1989 Review of touchscreen technologies and product 1989. Johnson Switch one-layer (horizontal only) wiregrid capacitive as first touch input device 1983/1967. Cross-wire capactive/electrostatic. Resistive membrane. Capactive film/sheet. Optical LED grid, optical LED from two corners. Elographics SAW surface acoustic wave with Z-axis force/pressure. Notes on parallax (esp. over curved CRT display), dead zones / target size (compare invention disclosure), [Dragon90] Dragon Systems "DragonDictate product announcement", Newton, Massachusetts, March 1990 Commercial voice/speech recognition product [DrummDE89a] Drumm, Donald E. "Electronic Light Pointer for a Projection Monitor", United States Patent 4,808,980, February 28, 1989, assigned to Wang Laboratories, Lowell, Massachusetts. Patent on wall-mounted light pen for projection TVs, displays [DulaneyKL89a] Dulaney, Kenneth L.; Hawkins, Jeffrey C.; Sanford, Lindsay; and McNamara, James H. "Split Screen Keyboard Emulator", United States Patent 5,031,119, July 9, 1991 On-screen simulated touchscreen keyboard, show full CGA display in top half of screen, on-screen virtual keyboard in bottom 1/2 of screen. [DunthornDI90a] Dunthorn, David I. "Virtual Button for Touch Screens", United States Patent 4,914,624, April 3, 1990 Get around lack of proximity sensing in virtual push buttons (with highlighting) on a digitizer touch screen, touch and untouch (sic) events. First touch/untouch does a selection, second touch/untouch actual activates what was selected. Instead add a second touch to the first touch, or move the finger in a particular direction, or press harder, and get a second button or a rotating menu to select/perform the particular action. [Dynabook89] Dynabook Technologies "product literature and press release", October 1989. PC/Computing, October 1989 Rumored portable touchscreen product: handwriting recognition? Any relation to Alan Kay's Dynabook? [EventoffFN89a] Eventoff, Franklin N. "Digitizer Pad", United States Patent 4,810,992, March 7, 1989 Resistive grid digitizer with force/pressure sensing, using a pressure sensitive layer between the two grids. One grid conductive, one grid resistive with resistance gradient. Grid need not be othogonal, shows circular and trapezoidal grids. [Evergreen90] Evergreen Technologies "Gestures product announcement", Jeffrey Siegel, Fox Chapel Center, Suite 401, 19751 Frederick Road, Germantown, Maryland 20874, personal correspondence, March 12, 1990 Handwriting (adaptive?) character recognition product, gesture interfaces? [Exos90] EXOS, Inc. "Dexterous Hand Master exoskeleton product information", August 1990. 8 Blanchard Road, Burlington, MA 01803, 617-229-2075



Data-glove, mechanical version, n-dimensional pointing device including Polhemus 3-D digitizer [Faggin89] Faggin, Federico, and Lynch, Gary S. "Brain Learning and Recognition Emulation Circuitry and Method of Recognizing Events", United States Patent 4,802,103, January 31, 1989, assigned to Synaptics, Incorporated, San Jose, California. Neural net patent Successive stages of recognition thresholds set for neural nets learning Includes neural net feature to resolve mis-recognized patterns confused with a pattern already learned Press announcement and description of IBM prototype signature verification system [FederalCW89] FCW "Smart signatures at FED: Net Prototype uses Biometrics, Smart Cards", Federal Computer Week, May 22, 1989, page1 [FellowsGL89a] Fellows, Gerald L. and Freivalds, Andris "The use of force sensing resistors in ergonomic tool design", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 713-717 Pressure/force sensors for hand-tools (drills, hammers, etc.): also stylus? Compare with FSR touchscreens. [FlemingSS90a] Fleming, S.S. "IBM 384381 Fleming Architecture for Separate User-Interface Software Development", IBM Technical Disclosure Bulletin, DA890-0063 JHB, Vol 33 No. 6A, November 1990 pp. 483-485 Separation of user-interface from application, messages send to application from action bars (menus) or pop-up windows (pop-up menus). [FoksJ90a] Foks, Jack "Technology and open learning", Proc. OLNT'90, Australian Society for Educational Technology Overview of on-line teaching/instruction systems ca. 1990. Describes Wang Freestyle as delayed communication/messaging system, point-to-point and point-to-multipoint (electronic annotation) [FoleyJD90a] Foley, James D.; vn Dam, Andries; Feiner, Steven K.; and Hughes, John F. "Computer Graphics: Principles and Practice, 1990 Edition", Addison-Wesley, 1995 (hardcopy book) Computer graphics reference, 1990 [FooteB89a] Foote, Brian "Class Warfare: Classes vs. Prototypes", Workshop on Objects without Classes, OOPSLA '90, New Orleans, LA Discussion of classless prototype-based languages, including Smalltalk and SELF. No description of development environments. [FrancikE89a] Francik, Ellen and Akagi, Kenichi "Designing a computer pencil and tablet for handwriting", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 445-449 Design of WANG Freestyle tablet and stylus: taper, length, comparison of human factors of tablets. Gives samples of types of writing for tablet and stylus design. Freestyle stylus was engineered to look and feel like a pencil, with an "eraser" button (colored pink) on the rear end of the stylus. Tablet digitizer itself was electromagnetic, designed by Hitachi. [FujiwaraA89a] Fujiwara, A., Takenouchi, M., and Shimizu, M. "Character Recognition Apparatus", United States Patent 4,813,078, March 14, 1989, assigned to Matsushita Electric Industrial Company, Limited, Kadoma, Japan. Optical character recognition for typed Kana using direction codes / chain codes of outer perimeter to compare best matches of lengths of segments [FurukooriH89a] Furukoori, H. "Image Contour Detecting Apparatus", United States Patent 4,817,173, March 28, 1989, assigned to Kabushiki Kaisha Meidensha, Tokyo, Japan. Curve tracing [GTCO89a] GTCO Corporation "CalComp 9500 Digitizer Series: User's Guide", GTCO Corporation, Copyright 1989 CalComp/GTCO 9500 digitizer with Pressure Pen stylus: senses tip force, not pressure: electromagnetic / magnetic coupling, stylus is transmitter. Includes software to use tip sensor as switch, with adjustable force threshold. [GaverWW89a] Gaver, W.W. "The SonicFinder: An interface that uses auditory icons", Human-Computer Interaction, Vol 4 No 1, 1989, pp 67-94 Cited in Nielsen90 Add sound effects / autitory feedback to events in graphical user interface: trash sound when putting something into trash can, etc. Symbolic == beep, methaphorical == music, iconic == trash sound. Mentions utility for visually impaired, as well as other users. [GeiserG89a] Geiser, Georg "Handprinted Data Entry with a Touch-Sensitive Numeric Keypad", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 305-308 Handwriting character recognition using a calculator's 5x5 (3x3 plus spaces between) keypad (similar to some Japanese calculator products): instead of double keying for alphabetics on a small calculator. Compare with Lucent/Microsoft touchpad calculator. Claims "very low" error rate: but unknown, does not say what the rate was. Shows number of variants/variations considered for handwriting recognition of single characters is NOT combinatorial. [GiordanoAG89a] Giordano, Albert G. and Disney, Christine "Desktop Dictionary of Information Systems Technology", Longman Group, 1989 (hardcopy book) Technical dictionary of computer terms 1989



[Go90] Go Corporation, Foster City, California "Go Corp. Gets IBM's Backing For 'Tablet PC'", PC Week, April 9, 1990, page 1 Go Corp. principals: Robert Carr (Aston-Tate), Jerry Kaplan (Lotus), Daniel Lewin (Next). Portable handwriting recognition product: PenPoint [GonzalezRC89a] Gonzalez, Rafael C. and Jerrera, Juan A. "Apparatus for Reading a License Plate", United States Patent 4,817,166, March 28, 1989, assigned to Perceptics Corporation, Knoxville, Tennessee. Features on license plates are edges, contours, height, width, topline and baseline, structural analysis, templates once the state (and therefor font) are known. [Goolsbey89] Goolsbey, Michael A., Jones, Eugene R., Micka, William F., and Robinson, Thomas, S. "Locating Individual Images in a Field for Recognition or the Like", United States Patent 4,817,186, March 28, 1989 Hardware to segment separate typed (handwritten?) characters for OCR recognition [Graphonomics89] International Graphonomics Society "Membership directory, Graphonomics", April 1989, c/o Department of Experimental Psychology, University of Nijmegen, PO Box 9104, 6500 HE Nijmegen, The Netherlands. Index of researchers in handwriting analysis, signature verification, and character recognition [GratkeNG89a] Gratke, Norman G. "Capacitance membrane switchcore with intertrace capacitive coupling and-or intratrace capacitive coupling", United States Patent 4,857,684 Capacitive (non-switch-contact) keyboard with driving and sensing rows and columns, multiple keys may be depressed simultaneously. multi-touch capable? [GreenM90] Green, Mark, and Jacob, Robert "SIGGRAPH '90 Workshop Report: Software Architectures and Metaphors for Non-WIMP User Interfaces", Computer Graphics, Vol 25 No 3, July 1991 UI/GUI design beyond pointing devices such as mouse and/or stylus digitizer: using gaze, sound, body motion: also gesture? [Greenstein89] Greenstetin, Joel S. and Baijal, Anish "An Investigation of Techniques for Occasional Numeric Data Entry", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 310-314 User-interface review of numeric keypad with mouse, multiple menus, sliders, clickers, simultaneous menus, used as substitute for a touchscreen: compare with Lucent/Microsoft materials. Reference to Quest Automation in 1970's as earlier handwriting recognition product. [GregoryWD89a] Gregory, William D. "Frequency dependent identification of materials", United States Patent 4,881,024, November 14, 1989 Capacitive sensing of material by measuring capacitance change at different frequencies, also change in phase. Electrode is segmented in 64-segment grid to measure different regions of object under test: allows profiles of classes of objects to check for internal structure / location of internal sub-objects. Also check Zimmerman 1995, Matias 2012 [GridPad89a] Grid Systems "Getting Started with GRiDForm", Grid Systems Corporation, September 1989 (hardcopy manual) Grid OS (on DOS) for tablet computer with handwriting recognition. Same binder as Program Development Manual [GridPad89b] Grid Systems "GRiDPAD Program Development Manual", Grid Systems Corporation, September 1989 (hardcopy manual) DOS-based tablet computer: Form definition graphical user interface. Allows for varying sizes of elements (radio buttons, etc.) Handwriting recognition context by subset / range/set of characters via HwrEnableChars. Returns list of possible characters with metric of certainty. P. 3-38 input from Pencept Penpad (but Grid has own handwriting recognition TSR?). Static buffers for tablet data, estimate 25 points/character (HwPInitialize) Same binder as Getting Started with GRiDForm [GridPad89c] Grid Systems "GRiDPAD A breakthrough in portable computing", Grid Systems Corporation, September 1989 Sales brochure with hardware specs (not software) for GridPad slate-style pen computer. [GridPad89d] Computergram International "Grid's GridPad handheld takes handprint data entry", Computergram International, October 2, 1989, no 1275 BYTE magazine gives award to GridPad product: compare 1985 award to PenPad. Additional GridPad information in file. GridPad 1910 ruggedized tablet computer, etc. One source cites GRiD Compass 1101 as "The first Clamshell Laptop", but it was predated by the DG/1 from Data General running MS/DOS. [GridPad89e] Computer Chronicles "Computer Chronicles: Portable Computers (1989)", Computer Chronicles video 1989 News presentation on portable computers, including touchscreen/pen computers: GridPad, Poqet PC, Dynabook 286, etc. Pre-Newton. Compare with GO / PenPoint. [GridPad90a] BYTE "The BYTE awards (important products and technologies)", Byte magazine, January 1990, vol 15 no1, p. 285. -- GRiD System's GRiDPad, Lotus Magellan, Zenith MinisPort, Poqet PC, etc.



[GridPad90b] Grid Corporation "GridPAD PadBase+ Training Course", course description, April 11, 1990, provided by Robert Artusy, R2Z Incorporated, 39120 Argonaut Way, Suite 319, Fremont, California 94538 GridPAD user-interface with stylus, training Correspondence includes list of features to be used for punctuation marks, handwriting prototype base forms and shapes [GrossbergS89a] Grossberg, Steven and Mingolla, Ennio "Neural Networks for Machine Vision", United States Patent 4,803,736, February 7, 1989, assigned to The Trustees of Boston University, Boston, Massachusetts. Boundary/edge/curve contour, feature contour, and object recognition in machine vision [GuiseDA89a] Guise, Dario A. "Efficient Frame Systems", Proc. EPIA 89 4th Portuguese Conference on Artificial Intelligence, pp. 39-50 Description of Garnet UIMS user interface management system, with color management. [GuoZ89a] Guo, Z. and Hall, R.W. "Parallel Thinning with Two-Subiteration Algorithms", CACM, Vol 32 No 3, March, 1989, pp 359-373 Skeletonization/line-thinning using four compass directions, which preserves line connectedness [HaarslevV90a] Haarslev, Volker and Moeller, Ralf "A Framework for Visualizing Object-Oriented Systems", Proc. ECOOP/OOPSLA '90, October 21-25 1990, pp. 237-244 Separating application and visualization in user-interface UIMS for automatic visualization of class hierarchies and graphs. Lisp/X-Windows, cites to visual programming [HaberS90a] Haber, Stuart and Stornetta, W. Scott "How to Time-Stamp a Digital Document", Proc. CRYPTO '90 10th Ann. Int'l. Cryptology Conf. on Advances in Cryptology, pp. 437-455 Digital time-stamping (certificate of date signed/existence of document) by time service signing hash of document and time value. Also forward-chaining of timestamps (hash of previous document) to prevent collusion of timestamp service to falsify record. Also in Journal of Cryptology Vol 3 no 2 pp 99-111, 1991 [HallRW89a] Hall, R.W. "Fast Parallel Thinning Algorithms: Parallel Speed and Connectivity Preservation", CACM, Vol 32 No 1, January 1989, pp 123-131 Robert M Harlick is editor for many CA.C.M. OCR line-thinning articles OCR handwriting line thinning algorithm which does NOT eliminate diagonal lines [HarnadS90a] Harnad, S. "The symbol grounding problem", CNLS Conference on Emergent Computation, Los Alamos, May 1989. Also Physica D 42: pp. 335-346 Cited in MartinGL90 Relating abstract symbols to actual objects: relating symbols to symbols -- learning Chinese from a Chinese language dictionary alone. symbol system vs. semantics. [HauptmannAG89a] Hauptmann, Alexander G. "Speech and Gesture for Graphic Image Manipulation", Proc. CHI '89, May 1989, pp. 241-245 Wizard-of-Oz simulation of gesture/speech interface to computer, using three-dimensional 3D multiple hand and finger gestures. Mentions Rolky and Lee multi-touch input devices for prototypes. [HayashiY89a] Hayashi, Y., Sakata, M., Nakao, T., Ohno, T. and Ohhashi, S. "Alphanumeric character recognition using a connectionist model with the pocket algorithm", (abstract only), Proceedings of International Joint Conference on Neural Networks, Washington, D.C., June 18-22, 1989, Vol II p. 606 OCR for a single font using neural networks [HenryTR90] Henry, Tyson R.; Hudson, Scott E.; and Newell, Gary L. "Integrating Gesture and Snapping into a User Interface Toolkit", in in UIST'90 (1990), ACM, pp 112-122 "X" cross-out gesture to delete; alphabetic gestures in a special "gesture area" on the tablet -- compare with Pencept PenPad? [HigginsCA90a] Higgins, Colin L. and Duckworth, R. James "The PAD (Ped And Display) - A Demonstrator for the Electronic Paper Project", Computer Processing of Handwriting, World Scientific Publishing, 1990, pp. 111-131 Electronic Paper Project EPP prototype of integrated tablet and display. Compare with IBM PLI Paper-Like Interface. Refers to parallax problems of touchscreen digitizer. Touchscreen (Scriptel: some parallax problems) used with Apple MacIntosh and MacWrite instead of mouse. [HildebrandTK89a] Hildebrand, Thoms K. and Stein, Norbert J. "Electronic book for reading textual information from an electronic medium / Elektronisches Buch zum Lesen literaturartiger Informationen aus einem elektronischen Medium", European Patent EP 0337401 A2, published October 18, 1989 (in German) Electronic book system with flat display, touch-display-screen interface: touching a word or other element in text displays additional relevant information, boolean search expressions. [HorowitzSL89] Horowitz, Steven L. "Method and Apparatus for Generating Size and Orientation Invariant Shape Features", United States Patent 4,803,230, January 31, 1989, assigned to GTX Corporation, Phoenix, Arizona. Character recognition, OCR, statistical decision tree. OCR using features of six bounding rectangles at different angles, translated to direction (chain?) codes OCR using Fourier transforms, autocorrelations, moments, centroid of largest hole and center of character [HsiehAJ90a] Hsieh, A.-J., Kunk, S.-J., Shiau, S.-L., Kao, M.-C., and Chen, J.-W. "An Experimental System for Stroke-number Free On-line Chinese Character Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal



Canada, April 2-3, 1990, pp 73-86 Taiwanese efforts on both OCR and on-line Chinese character recognition: 5401 characters x 23 people x 10 samples each OCR, 5401 x 23 people (mixed) on-line data base collection. Later came by to visit Wang: part of five-year project, 9 people OCR, 5 on-line, first year is "find out everything there is to know" [Hubby89] Hubby, Laurence M. "Flat Panel Display with Integrated Digitizing Tablet", United States Patent 4,873,398, October 10, 1989 Optical digitizing tablet integrated with a flat (flat front surface CRT?) display. Light is emitted by the stylus, projected back through the same optical path used for projecting the image on the front surface (? ). Cites Scriptel. [HullJJ90a] Hull, Jonathan J., Commike, A., and Ho, T.-K. "Multiple Algorithms for Handwritten Character Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 117-130 OCR for handwritten Zip codes, address blocks. Description of IBM signature verification system at Federal Reserve [HurstJ89a] Hurst, Jan (chair); Mahoney, Michael S.; Taylor, Norman H.; Ross, Douglas T.; and Fano, Robert M. "Retrospectives: The Early Years in Computer Graphics at MIT, Lincoln Lab and Harvard", SIGGRAPH '90 Panel Proceedings, pp. 19-38, 39-73 (two parts) Light gun (light cannon) invented 1949 on Project Whirlwind. Origin of term "kludge". [IBM89a] Federal Computer Week "Smart Signatures at Fed", Federal Computer Week, May 22, 1989, Page 1 [IBM89b] IBM Corp. Research Division, press release "IBM Writing Interface May Replace Typing", spokesperson Gerald Present, September 1989, 914-945-3884 Description of IBM's "paperlike interface" user interface [IBM90a] IBM Corp. "Function-independent Approach to Driving Soft Keyboard", IBM Technical Disclosure Bulletin, Vol 33 No 4, September 1990 pp. 159-ff (partial copy) Soft/virtual keyboard using touchscreen on separate display: compare with Kaplow, Knowlton. Keyboard images and configuration can be changed for application, driver implementation. [IBM90b] IBM Corp. "Method of Providing On-Screen Keyboard for Touch-Sensitive Systems Without Affecting Application Programs", IBM Technical Disclosure Bulletin, Vol 33 No 5, October 1990 pp. 320-ff Pen-computing driver generates keyboard input from handwriting and gestures, work with existing applications. Compare with PenCept ca. 1986, CIC. [IBM90c] IBM Corp. "Pressure-Sensitive Icons", IBM Technical Disclosure Bulletin, Vol 33 No 13, June 1990 pp. 222-ff (partial copy) Pressure-sensitive touchscreen implements buttons by pressing harder. [IMCS90] International Machine Control Systems "MousePen product literature", January 1990. 1332 Vendels Circle, Paso Robles, California 93446. 1-805-239-8976 Stylus-shaped mouse with mechanical trackball: substitute for a tablet? [ITRI89] ITRI "Chinese Character Recognition", Wang Laboratories internal memorandum on discussions with Industrial Technology Research Institute (ITRI) in Taiwan, June 20, 1989 [ImpedovoS90a] Impedovo, S., Castellano, M., Pirlo, G., and Dimauro, G. "An Off-line Writer Identification System Based on a Syntactic Approach", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 53-64 Like signature verification, but using regular hand printing (a nothing paper, looks like they had to change horses from adaptive handwriting recognition in mid-stream) [IndustrialEE89] Industrial Electronic Engineers, Inc. "Pixie Graphic LCD Switch keyboard product literature", 1989, 7740 Lemona Ave, Van Nuys, CA 91409, 818-787-0311 User-interface item: keys with individual graphic LCD displays in the key top for relabeling the keys [InfoWorld90a] Krohn, Nico "Scenario Demonstrates Pen-Based Input System", InfoWorld, April 16, 1990 p. 8 Scenario Dynawriter tablet/display for PC with handwriting recognition: compare with CIC, PenCept, PaperLike Interface of IBM, Sklarew. Mentions GridPad. [InfoWorld90b] Krohn, Nico "IBM Will License Go's Pen-Based OS for Laptops", InfoWorld, July 23, 1990 p. 1 IBM to license Penpoint operating system from GO, plans to test in 150 user sites. Portable to different architectures. Response on Pen Windows from Microsoft. [IshikawaY90a] Ishikawa, Y., and Kakajima, K. "A Real Time Connected Word Recognition System", 10th International Conference on Pattern Recognition, Atlantic City, New Jersey, June 16-21, 1990 Note on Chinese character recognition product internal to Wang Laboratories, but shown publicly. [JackelLD89a] Jackel, L.D. "VLSI Technology and Neural Network Chips", Tutorial Notes No 8, International Joint Conference on Neural Networks, Washington D.C., June 18-22, 1989 Contains examples of neural nets for handwritten numerals recognition in O.C.R. [JellinekHD90a] Jellinek, Herbert D. and Card, Stuart K. "Powermice and User Performance", Proc. CHI '90, April



1990, pp. 213..220 Study of powermice (mouse with velocity gain depending on speed of user input) for scrolling etc. Powermouse gain has no improvement on accuracy or pointing, more likely to be addressing problem of larger displays and smaller desk working area . [JohnsonJ89] Johnson, J., Roberts, T.L., Verplank, W., Smith, D.C., Irby, C.H., Beard, M., and Mackey, K. "The Xerox Star: A Retrospective", IEEE Computer, Vol 22 No 9, September, 1989, pp 11-28 User-interface description of the development of the Xerox Star desktop paradigm, now called ViewPoint (originally on Alto computer) Xerox Star original desktop direct-manipulation user-interface intended only for casual users Desktop metaphor for user interface is intended for office applications an publishing, not software developments (tools based) Direct-manipulation desktop user interface (no processing on files per se) is orthogonal to tools-based user interface (UNIX/SunView Windows: any file can be operated on by any tool, even if not sensical) Clean user-interface lies more in Xerox Star details, not in features Mouse pointing input must be handled at low-level by system to prevent jerkiness, jumps, dead time as the system degrades: these (mouse or tablet or whatever) are eye-hand co-ordination devices One-button mouse (or tablet stylus) vs. two-button: fewer button confusion errors, but more increased selection errors Overlapped-windows cause users to spend time adjusting windows so they don't overlap: just use tiled windows to begin with (if the display is 17" large) Xerox Star: cites user-interface work by many others using graphical/sketch input, Xerox View Point [JohnstonJP89a] Johnston, James P. and Russell, Donald L. "Information Display and Entry Device", United States Patent 4,814,760, March 21, 1989, assigned to Wang Laboratories, Lowell, Massachusetts. Patent on integrated tablet and display (touchscreen) for a dynabook/electronic-ink. Tablet is capacitive, flexible/resilient layer surface on top can be pressed closer by stylus to capacitive sensor pad below, increasing capacitance. X/Y matrix grid of sensors. Also mentions detecting a change in resistance (impedance). Appears to measure each grid point separately [JonesD89a] Jones, D., Hapeshi, K., and Frankish, C. "Design guidelines for speech recognition interfaces", Applied Ergonomics, Vol 20 No 1, March 1989, pp 47-52 Syntax (context) has profound effects on speech variability, so design user-interface to minimize effects Very important to collect samples (speech) in context for training: phonetic, semantic, and personal stress context affect variability Issues of user-interface design for task-specific, limited speech recognition [Jourjine89] Jourjine, A.N. "Microprocessor Assemblies Forming Adaptive Neural Networks", United States Patent 4,809,193, February 28, 1989 Cellular automaton for neural nets, using radiated energy coupling between separate microprocessors [Jourjine90] Jourjine, Alexander "Variable capacity transmission medium", United States Patent 4,967,369 Neural net establishing correlation between signals by exchange of random codes [JoyceR90a] Joyce, Rick and Gupta, Gopal "Identity authentication based on keystroke latencies", CACM Vol 33 No 2, February 1990, pp. 168-176 Keyboard latency (biometric) to authenticate (verify identity of) user, handwritten signature shapes for signature verification. Cites to Rand 1980, but not to Saltzer 1975. [Katsurada89] Katsurada, Morihiro "Method of a System for Analyzing Characters", United States Patent 4,811,412, March 7, 1989, assigned to Shapr Kabushiki Kaisha, Osaka, Japan. Segment/parse character from right and left, see if they match, if they don't, try templates [Katt90] Katt, Spencer F. (pseudonym) "Microsoft ueber alles?", PC Week, April 16, 1990, page 182 "Rumor Central": Microsoft conference in Canada, microsoft starting development of Windows-based operating system for handwriting recognition, to compete with GRiD and Go Corporation, estimated release is 12 to 18 months [Kawakami90] Kawakami, Yasushi; Obata, Fukue; Makihara, Kayoko; Shiro, Yamada; Morimoto, Yoshinari et al "Data processing device with screen separated into line format area and ata display area", United States Patent 4,897,804, January 30, 1990 Text-input recognition area on tablet, data can be moved into regular display area [KellyW90] Kelly, William "Sparking Stylus for Acoustic Digitizer", United States Patent 4,891,474, January 2, 1990 SAC Science Accessories Corporation sparking stylus for acoustic/sonic tablet digitizer [Kensington89] Kensington Microware, Turbo "Mouse trackball product literature", Kensingon Microware, 1989. 251 Park Avenue South, New York, New York 10010 Mouse/digitizer trackball replacement products [Khotanzad90] Khotanzad, Alireza and Hong, Yaw Hua "Rotation invariant image recognition using features selected via a systematic method", Pattern Recognition, Vol 23 No 10, October 1990, pp 1089-1102 Character recognition using rotation-invariant features - Zernike moments (what about "E" and "W"?) Rotation-invariant OCR: 84% on numeric handprinted characters, 16,550 samples, 99% on "clean" alphabetics Numeric handprinting OCR samples from Recognition Equipment, Inc. Claims 84% correct recognition rate is very effective



[KinnucanP89a] Kinnucan, P. "The Versatility of Graphics Tablets", Computer Graphics Review, Vol 4 No 7, July 1989, pp 44-52 Simple-minded review article on digitizer technologies [KnightLW89a] Knight, Leland, W. and Retter, Dale "Datahand: Design, Potential Performance, and Improvements in the Computer Keyboard and Mouse", Proceedings of Human Factors Society 33rd Annual Meeting, October 1620, 1989, Denver Colorado, pp 450-454 Two handed mouse/keyboard user-interface combination, using the "Datahand" product [Koezuka89] Koezuka, T.; Tsukahara, H.; Nakashima, M. "Pattern Matching Method and Apparatus", United States Patent 4,805,224, February 14, 1989, assigned to Fujitsu, Limited, Kawasaki, Japan. Video pattern matching recognition using two-dimensional extracted image from frame buffer [KreifeldtJG89a] Kreifeldt, J.G., Levine, S.L., and Iyengar, C. "Reduced Keyboard Designs using Disambiguation", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 441-444 Stenotype-type user interface keyboard design: one key per finger, multiple characters per key [KreinPT90a] Krein, Philip T. and Meadows, R. David "The Electroquasistatics of the Capacitive Touch Panel", IEEE Trans. on Industry Application, Vol. 26, No 3, May/June 1990, pp. 529-534 Seems to be same as KreinPT88a: review of wavy-line distortion and inaccuracies in resistive/capacitive touch panels and tablets using resistive sheets. [KrzyzakA90a] Krzyzak, A., Dai, W., and Suen, C.Y. "Unconstrained Handwritten Character Classification Using Modified Backpropagation Model", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 155-166 [KuanCL89a] Kuan, C-C.L. and Srihari, S.N. "Analysis of handwritten text images using run-length based algorithms", IEEE Trans. Pattern Analysis and Machine Recognition, to appear, 1989 Cited in Taxt90 [KulpJ90a] Kulp, James and John "Personal communication: telephone conversation", April 1990. SuperScript Incorporated, Cambridge, Massachusetts, 617-876-0787 Working on a portable handwriting recognition product (with forms) John Kulp says they are a Symbolics spin-off: CD-ROM portable box for manuals. Did their own elastic matching: not super happy with it (using lots of context helps) [KunduA89a] Kundu, Amlan; He, Yang; and Bahl, Paramvir "Recognition of Handwritten Words: First and Second Order Hidden Markov Model Based Approach", Pattern Recognition, vol. 22, no. 3, p. 283, 1989 Cited by Marlin Eller, Microsoft Pen Computing group [KuoWC90a] Kuo, Wen-Cheh; Lin, Chia-Hui; Chou, Te-Chang; Lian, Yuan-Hsiung "Touch-control Computer Mouse", United States Patent 4,977,397, December 11, 1900 Small touchpad mounted in a mouse-like housing, to plug into a mouse port printed patent title miss-spells "mouse" as "house". [KurlanderD89a] Kurlander, David and Feiner, Steven "A Visual Language for Browsing, Undoing, and Redoing Graphical Interface Commands", Reprint from Visual Languages and Visual Programming, S.K. Chang ed.; Plenum Press, NY 1990, pp. 257-275 Graphical/visual programming language, but really a graphical editor user interface for "undo" showing command history in small graphical snapshots (panels) of what the image looked like [KurosawaY89a] Kurosawa, Y. "Pattern recognition apparatus with two-part reference storage", European Patent 335696, Assigned to Toshiba Two-level OCR recognition using contours for one level, then detailed features for final resolution Performs choice of matching pattern from among number of candidate standards preselected using partial characteristic data. Describes tablet template, side-menu touch buttons for tablet (user-interface) Kurta's cordless stylus/pen for a digitizing tablet [Kurta89a] Kurta Corporation "Kurta IS/ONE, IS/ADB, and IS/TWO product description", Kurta Corporation, 3007 East Chambers Road, Phoenix, Arizona 85040 shows static templates GUI for picking commands in various applications [Kurta89b] Kurta Corporation "PageMaker, FreeHand, and Persuasion just got easier to use", Kurta Corporation, 3007 East Chambers Road, Phoenix, Arizona 85040 on-tablet templates with command menus, QuickKeys scripting language for keyboard shortcuts [Kuzunuki89] Kuzunuki, Soshiro, Shojima, Hiroshi, Miura, Masaki, Mori, Junko, and Mifune, Toshimi "Real Time Handwritten Character Input System", United States Patent 4,860,372, August 22, 1989, assigned to Hitachi, Limited, Tokyo, Japan. User interface on keyboard emulation with handwriting input: Ward85 was cited as prior art (check US Patent category 382,13). Handwriting user interface: use a Times-square circular buffer for writing in, because written character are bigger than typed font [LaLondeWR90a] LaLonde, Wilf R. and Pugh, John R. "Inside Smalltalk, Volume I", Prentice-Hall, 1990



Smalltalk object-based programming system, object-based programming language (prototype-based?), graphical user interface with windows, and IDE. Multiple classes of number data types: float, integer, and pure fraction. Transparent windows and background using a two-bit pixel map -- black/white, transparent/opaque, with transparent/white combination declared value undefined: compare to bit/mask layers in V25 hardware on Wang Freestyle The Guide. BitBlt support. View==Window. [LacyME89a] Lacy, Mark. E. "Artificial Laboratories", AI Magazine, Vol 10 No 2, 1989, pp. 43-48 Artificial realty environment, mentions virtual switches and virtual slide switches in graphical user-interface UI: input is by a DataGlove in three dimensions: refers to hand movements instead of gestures -- pinch gesture shown? [LandmeierWL89a] Landmeier, Waldo L. "Ink-on-Glass Digitizer Tablet and Method of Construction", United States Patent 4,873,399, October 10, 1989 Transparent touchscreen with spaced vertical conductive ink conductors on top surface of glass substrate, second set of horizontal conductors on bottom surface ITO? Transparent? Cites Rockwall patent on digitizer [LeCunY90a] Le Cun, Y.; Boxer, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D., and Baird, H.S. "Constrained Neural Network for Unconstrained Handwritten Digit Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 145-154 In his notes, claimed neural nets can learn feature on their own: actually, his net structure, pixel resolution, etc. determined the class of features. OCR of handwritten zip codes: segmentation, "bad" samples already done. [LeeS89a] Lee, Seongwhan, and Kim, Jin H. "Unconstrained Seal Imprint Verification using Attributed Stroke Graph Matching", Pattern Recognition, Vol 22 No 6, 1989, pp 653-664 Like signature verification, but for the stamps/seals used in oriental/Chinese countries Spatial distributions of stroke skeletons: recognition independent of size/scale/magnification [LegaultR90a] Legault, R., Suen, C.Y., and Nadal, C. "Classification of Confusing Handwritten Numerals by Human Subjects", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 181-194 Similar to Shillman functional attribute experiments, using old zip-code samples Contains some hypothesis on distinguishing features for handwritten numerals and human recognition [LiHF89a] Li, H.F., Pao, Derek, and Jayakumar, R. "Improvements and Systolic Implementation of the Hough Transformation for Straight Line Segments", Pattern Recognition, Vol 22 No 6, 1989, pp 697-706 Add contiguity check to Hough transform to find straight line features in a binary image: handles overlapping lines, short lines. [LiHF89b] Li, H.F., Jayakumar, R., and Youssef, M. "Parallel algorithms for recognizing handwritten characters using shape features", Pattern Recognition, Vol 22 No 6, 1989, pp 641-652 Static/OCR handwriting recognition: features are local shape, start and end points, edge types, and join relations in contours (digits only!) OCR for handwritten digits: 120 samples per digit Handwriting OCR: twelve relations for contours: compare to combinatorial explosion in Ward88? chain codes via systolic array [Lien89] Lien, Yih-Liang L. "Apparatus and Method for Vectorization of Incoming Scanned Image Data", United States Patent 4,817,187, March 28, 1989, assigned to GTX Corporation, Phoenix, Arizona. Curve-tracing, curve-follower, scanner algorithm for detecting lines in OCR or image analysis. [LinWC90a] Lin, Wen C. "Handbook of digital System Design, 2nd Edition", CRC Press, 1990 (hardcopy book) Upper-level textbook on digital circuits and systems. More focus on physical device properties (analog properties / switching frequency / etc.) [LinnJ90a] Linn, John "Practical Authentication for Distributed Computing", Proc. 1990 IEEE Symp. on Research in Security and Privacy, Los Alamitos, California IEEE Computer Society, 1990 pp. 31..40 Overview of DSSA Distributed System Security Architecture (DEC Digital Equipment Corporation): systems are able to trust each other because of PKI authentication of identity and integrety of messages. Identity certificates from a CA, also dynamic binding certificates between two parties for delegation of authority: compare with SSL protocol. Certificates used to trust of software programs, especially during boot/loading. [LintonMA89a] Linton, Mark A.; Vlissides, John M.; and Calder, Paul R. "Composing User Interfaces with InterViews", IEEE Computer, February 1989, pp. 8-22 UIMS (Garnet/Interactors) in C++, colors and appearance of UI defined in run-time calls to C++ classes, including color management, model is based on X toolkit. Multiple UIs for same application, UIs need not be entirely graphical. [LoganJD89a] Logan, Jame D. and Litvin, Yury "Touch Screen Computer-Operated Video Display Process and Apparatus", United States Patent 4,821,029, April 11, 1989 Touch screen UI: select a button or area, information is copied to a different window or area on the screen.



[Logitech89] Logitech Corporation "LogiMouse C7 Technical Reference Manual", Logitech Corporation, 1989 Electronic digitizer tablet sold as a mouse replacement: emulates Microsoft Mouse data format, other tablets [LuS90a] Lu, S.W. and Suen, C.Y. "Hierarchical Attributed Graph Representation and Recognition of Handwritten Chinese Characters", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 87-100 (abstract only) Chinese/Kanji OCR [MCC90] MCC Microelectronics and Computer Technology Corporation "Interactive Tablets", description of work in progress, CHI '90 Conference on Human Factors in Computing Systems, Seattle Washington, April 2-3, 1990. Available from Tom Parish, MCC, PO Box 200195, Austin Texas 78720, 512-343-0978 General description of neural net character recognition, user-interface for handwriting at MCC MCC staff on handwriting/gesture user interface: Tom Parish (general), Jim Pittman and David Chapman (hardware), Gale Martin (recognition), Kent Wittenburg (interpretation), Bill Gooch (Prototypes) Working on mathematical expression user-interface stroke segmentation [MacTutor90a] MacTutor "MacTutor: The Macintosh Programming Journal", MacTutor: The Macintosh Programming Journal, Vol 6. No. 3, March 1990 (hardcopy book) General issue on programming development and environments for MacIntosh, commercial IDEs, Hypercard. Build environments similar to PenApps/ Visual Basic: Prototyper 2.0 from SmethersBarnes draws and generates code: HyperBasic from Teknosys, compiler linker and resource mover for XCMDs and XFCNs (HyperCard and Hypertalk extension); MacInterface from Holder Egand and Company: ToolKit HyperCard script editor; Classy Object Factor with Application Generator; Prograph 1.2 OOP programming CASE tool; Serius Desktop programming Serius79 for graphical custom applications. [MacTutor90b] MacTutor "MacTutor: The Macintosh Programming Journal", MacTutor: The Macintosh Programming Journal, Vol 6. No. 4, April 1990 (hardcopy book) p. 84, TooLKit MenuMaker by Shared Intelligence for Hypercard development, p. 50 HyperBasic for HyperCard, p. 33 Prograph by TGS Systems, p. 13 ScriptEdit for HyperCard. Build environments similar to PenApps/ Visual Basic: Prototyper 2.0 from SmethersBarnes draws and generates code: HyperBasic from Teknosys, compiler linker and resource mover for XCMDs and XFCNs (HyperCard and Hypertalk extension); MacInterface from Holder Egand and Company: ToolKit HyperCard script editor; Classy Object Factor with Application Generator; Prograph 1.2 OOP programming CASE tool; Serius Desktop programming Serius79 for graphical custom applications. [MackR89a] Mack, Robert, and Lang, Kathy "A Benchmark Comparison of Mouse and Touch Interface Techniques for an Intelligent Workstation Windowing Environment", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 325-329 Replace mouse with touchscreen digitizer tablet on screen to Windows 2.0 by using new driver. Refers to double-click/double-tap gesture problems of mouse versus keyboard, but overall positive. Suggests delaybefore-select as alternative to select-on-lift when using touchscreen with stylus, so that user has some time to "fine-tune" click/tap/select position, also refers to maintain-contact to select followed by drag. Dwell/press-and-hold -- maybe combinable: click-hold-drag / maintain-contact-and-drag is ordinary drag. Stylus faster than keyboard for pointing. [MahachKR89a] Mahach, Karen Renee "A Comparison of Computer Input Devices: Linus Pen, Mouse, Cursor Keys and Keyboard", "Linus Pen, Mouse, Cursor Keys, and Keyboard", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 330-334.. Linus electronic ink tablet/display had low accuracy: typing faster than writing: points out that a handwriting user-interface is not really just handwriting, but a lot of menu-picking for data entry, which needs to be studied Looks like a good researcher to contact on electronic ink studies [MaltzIJ89a] Maltz, Ivan J. "Video Compositing Using aSoftware Linear Layer", United States Patent 4,827,253, May 2, 1989 Overlay foreground image (on a touch display, etc.) by having a separate blending-factor palette look-up table givin blending factor (alpha-blending) for particular pixels, separate from the images being overlaid. [Marshall89] Marshall, William W. and McWhortor, William F. "Method and Apparatus for Pattern Recognition", United States Patent 4,817,176, March 28, 1989, assigned to William F. McWhortor, Fountain Hills, Arizona. Character recognition from OCR using Fourier transforms, returning confidence percentage goodness based on second-best match. [MartinGL89a] Martin, Gale L. "The utility of speech input in user-computer interfaces", International Journal of Man-Machine Studies, Vol 30 No 4, April 1989, pp 355-376 Points out that user-interface questions for speech recognition are hard to answer, because so many interaction modalities are involved Points out indirectly (for Votan speech recognition) that the "perfect" recognition of 100% went down over time after training. Grand review of speech recognition in user-



interfaces Speech recognition is faster than typing, and it has the advantage of being an additional channel of input besides the hands (haptic?) For a simplified set of commands, speech recognition and mouse clicks equally efficient Report by group at MCC Microelectronics and Computer Technology Corporation on their handwriting/speech recognition system (MAGIC) Cites a large number of studies that show speech recognition is not as good as keyboard input (!!), although this paper says speech is better [MartinGL90a] Martin, Gale L., Leow, Wee Kheng, and Pittman, James A. "Function Complexity Effects on Backpropagation Learning", MCC Technical Report ACT-HI-062-90, Microelectronics and Computer Technology Corporation, Human Interface Laboratory, 3500 West Balcones Center Drive, Austin, Texas 78758, January 1990 Neural network report on on-line handwriting recognition Data collection of 8,600 upper-case letters from (from NCR for digits) 110 people on an integrated tablet/display, 200 points/second 1000 points/inch Training sets of 500,2500,6261 sample from different people States one needs tens of thousands of samples of handwriting data base to train a system, more than an adaptable neural net Neural nets need to be designed to detect human features, otherwise not much good [MartinGL90b] Martin, Gale L., and Pittman, James A. "Recognizing Hand-Printed Letters and Digits Using Backpropagation Learning", MCC Non-confidential report, to appear in Advances in Neural Information Processing, D.S. Touretzky, editor, Morgan Kaufman, 1990 Cites NCR corporation data-base of hand-printed on-line character shape collection 8,600 on-line handwriting samples from 110 people, 200 points/second at 1000 points/inch: images converted to blurred 15x24 grayscale image for static character recognition Training set of 500 to 6,300 samples, 18 people, separate from test data: human handwriting recognition error rate was 3.4% (compare Neisser60) Gives table of exact trade-offs between reject rate and error rate on handwriting recognition, for digits and letters separately Estimate of number of training samples needed for adaptive recognizer / neural network Credits at MCC/NCR to Dave Rumelhart, Joyce Conner, Janet Kilgore, and Kay Bauer for collection [MartinGL90c] Martin, Gale; Pittman, James; Wittenburg, Kent; Cohen, Richard; and Parish, Tom "Sign Here, Please", BYTE Magazine, July 1990, pp. 243-252 Overview article on handwriting/gesture and voice recogition, especially parsing/chunking problems, neural net trainable handwriting recognizer at MCC: mentions DragonDictate, Freestyle, VoiceBox from Voice Recognition Technologies, Interactive Worksurface Project (tabletop? / blackboard metaphor?) at MCC, other voice recognition companies and products. [MarzinkewitschR90a] Marzinkewitsch, Reiner "Ein Arbeitsplatz zum rechnerunterstützten handschriftlichen Rechnen mit mathematischen Formeln", Dr. Nat. Dissertation, Uni. des Saarlandes, Saarbrücken, 1990 Ph.D. thesis on pen-computing/tablet input of mathematicla formulas, editing, on graphics tablet (complaints about the stylus/Griffel). (in German). Sections on 2D parsing and segmentation. Discussion of context-free 2D grammars. Cites to Anderson 70, not to Martin/Kaplow. [MathewsMV90a] Mathews, Max V. "Three Dimensional Baton and Gesture Sensor", United States Patent 4,980,519, December 25, 1990 See Radio Drum: Electrostatic grid of sensor pads, sense two or more styli (drumsticks) in X, Y and Z by capacitive coupling. Pads have segmented shape so that shielding of one layer by another is minimized. Tapered conductors: charge ratio? Baton striking surface (touching) is inferred/predicted from Z motion. [MatthewsHG89a] Matthews, Henry G.; Zalenski, Thomas; Barbetti, Jamie L.; and Mletzko, Al "Method and apparatus for calibration of a charge ratio digitizer", United States Patent 4,831,566, May 16, 1989 Self-Calibration for a charge ratio (or really any capacitive) digitizer: phantom/simulated cursor/stylus electrodes at known positions, check for reported position of the phantom cursors and calculate correction factors. [MaulsbyDL89a] Maulsby, David L.; Witten, Ian H; and Kittlitz, Kenneth A. "Metamouse: Specifying Graphical Procedures by Example", ACM Computer Graphics, Vol 23 No 3, July 1989, pp. 127-136 Programming by example: deduce macro operations for graphical actions by applying constraints to observed operations in GUI. Must infer variable (such as row indices) [McArthurD90a] McArthur, David; and Stasz, Cathleen "An Intelligent Tutor for Basic Algebra", RAND Report R3811-NSF Prepared for The National Science Foundation, August 1990 Two-dimensional mathematical input: not clear on the input means for user interface, perhaps on-line handwriting recognition [McAvinneyP90a] McAvinney, Paul and Rubine, Dean H. "Videoharp", United States Patent 4.968,877, November 6, 1990 Electronic VideoHarp musical instrument using optical sensing of multiple fingers (multi-touch) and gestures: either harp-like (strum) or piano-like (projected keyboard). Optical sensing on two sides (either side) of transparent instrument, such that instrument can be played with both hands like a harp, or mounted horizontally like a piano. MIDI output. to a synthesizer. Position of finger along a ray detected by shadowing. See also RubineD88a [McDonaldJE89a] McDonald, James E. and Stone, Jimmy D. "Parallax induced pointing error avoidance method



and means for systems using touch screen overlays", United States Statutory Invention Registration H716, December 5, 1989 Put margin around active touch areas (button) for touchscreen on CRT display for safer targetting / pointing error. Margin sized to match parallax error / accuracy. [MeadowsRD90a] Meadows, Robert D. "Integrated Liquid Crystal Panel and Optical Touch Panel", United States Patent 4,916,308, April 10, 1990 Optical touch screen with high resolution equal to LCD pixels, small angled mirrors reflect light from outermost row/column of LCD display to be the light beams across the display. Stylus for high resolution. [MeeksML90a] Meeks, M.L., and Kuklinski, T.T. "Measurement of Dynamic Digitizer Performance", to be published in Computer Processing of Handwriting, Leedham and Plamandon, editors, World Scientific, Singapore, 1990 Report on tablet characteristics and instrumentation for measuring handwriting Describes test bed for calibrating tablet digitizers with an X/Y table to correct for *fixed* positional errors -- not tilt or velocity related accuracy problems? [Microslate89] Microslate "press release", Microslate, 1989 Rumored touchscreen portable product: handwriting recognition? [Microsoft90a] Microsoft Corporation, announcement of "Windows/H", Product development, PC Week "Spencer F. Katt" column, April 16 1990, p. 182 Microsoft (Lloyd Frink and Marlin Eller) on Windows/H PenWindows user-interface for handwriting recognition: using "Anfa" or "Infa" company [Microsoft90b] Microsoft Corporation "Windows of Opportunity: Microsoft OEM Briefing 1990", Microsoft, Bellevue WA, October 1-3, 1990 Included information "Information at your Fingertips". Windows/H Windows for Pen Computing: rotation for Pen Windows computer screen for vertical or horizontal pad of paper orientation. Pen Windows will allow the user to rotate the screen. Stylus senses rotation of stylus barrel and pressure (tip force), proximity above screen. Side button to distinguish between handwriting, gestures, mouse-like pointing actions. Virtual keyboard / synthetic keyboard with writing window. Gestures, writing input with position. Drag, select, bracket gestures. fetched from http://www.anonymous-insider.net/windows, 2011 [Microtouch89a] Microtouch "Microtouch unMouse product file", (various) Microtouch.com File of references regarding PC UnMouse from microtouch: resistive-film small touchpad/touchscreen panel on a mechanical switch, user can drag finger across surface of touchscreen to simulate proximity information or mouse motion, user presses on entire surface to close switch to for mouse click / touch-lift. Usually operated in relative-motion mode, not absolute digitizer tablet mode. Available for DOS/Windows PC and for MacIntosh. [Microtouch90a] (no author) "ScreenWrite Touch Screen Performance Specifications", Available from Microtouch Corporation, Microtouch.com Digitizer touch-screen/tablet, shows calibration/linearization template for adjusting for regional position errors -- rectangular grid with alternating rows/columns offset. [MillerBP90a] Miller, Barton P.; Fredriksen, Louis; and So, Bryan "An empirical study of the reliability of UNIX utilities", Comm. ACM, Vol 33 No 12, Dec. 1990, pp. 32-44 Fuzz-testing: random keyboard input to Unix utilities (very mature programs) causes most to fail, because few programmers check plausibility of user input, and thus fail on unanticipated input. [MingJ90] Ming, Ji "The Statistical Information Formulation for Noisy Speech Recognition", 10th International Conference on Pattern Recognition, Atlantic City, New Jersey, June 16-21, 1990 [MinskyM90a] Minsky, Margaret; Ming, Ouh-young; Steele, Oliver; Brooks, Frederick P. Jr.; and Behensky, Max "Feeling and seeing: issues in force display", Proc. 1990 Symp. on Interactive 3D Graphics: SIGGRAPH, Vol 24 No 2, mar. 1990, pp. 235-241 Haptic feedback to give the illusion of a surface requires surface, texture, temperature, softness, etc. Software synthesizes, also viscosity. Special note on stability: analog integration required with digital control, raising joystick sampling from 30Hz to 1000Hz was not enough, wanted to achieve similar stability to human arm (which has 200msec response time, but is still stable) [Mira89] Mira Imaging, Inc. "Hyperspace (tm) product literature", 1989. 969 Logan Avenue, Salt Lake City, Utah, 84105 3-D digitizer software: works with Science Accessories, IsoTrak (Polhemus), Cyberware laser scanner, Topcon. [MoellerR90a] Moeller, Ralf "Entwicklung von Visualiserungswerkzeugen in objektorientierten Systemen unter Verwendung von KI-Programmiermethoden", Universität Hamburg, Technical Report B-149, 1990 (partial copy) 1989 Doctoral thesis: describes LabView and other visual programming languages, page 13 shows virtual slide switch element [Momenta89] Momenta Inc "press release", Milpitas California, October 19, 1989 Rumored handwriting recognition portable product from Momenta



[Momenta90] Display Momenta Corporation "What's New in Laptops and Pen Computing", Bill Lempesis of Dataquest, Flat Panel Display News, Vol 1 No 3, May/June 1990 Rumors of Go Corporation, Grid, Momenta: contains a P.R. blurb from Momenta on "four waves of personal computing user-interfaces", leading up to verbal/voice recognition. [MoogRA90a] Moog, Robert A. and Rhea, Thomas L. "Evolution of the Keyboard Interface: The Boesendorfer 290 SE Recording Piano and the Moog Multiply-Touch-Sensitive Keyboards", Computer Music Journal, Vol 14 NO 2, Summer 1990, pp. 52-60 Moog Multiply Touch-Sensitive keyboard (MTS keyboard) with resistive-sheet X/Y sensing, conductive rubber capacitive sensing of Z/force/pressure, velocity (photographs). Similar Boesendorfer 290 (modern) piano recorder. [MoriR90a] Mori, Ryoichi and Kawahara, Masaji "Superdistribution: The Concept and the Architecture", Trans of IEICE, Vol E73, No 7 July 1990 -- Special Issue on Cryptography and Information Security DRM document distribution system, software usage fees and royalties, permission. Uses trusted software secure hardware module (S-box) for real-time-clock for timestamps and storing private keys, hardware detection of hardware tampering. Electronic file contains additional information. [MoriY89a] Mori, Y. and Yokosawa, K. "Neural networks that learn to discriminate similar kanji characters", in Advances in neural information processing systems I, D.S. Touretzky, editor, Morgan Kaufman, 1989 cited in MartinGL90 [MyersBA89a] Myers, Brad A. "User-Interface Tools: Introduction and Survey", IEEE Software, Jan 1989, Vol 6 No 1, pp. 15-23 In Garnet system papers: refers to need for one application to have many user interfaces, separation (as a design question?) of user-interface an application. [MyersBA89b] Myers, Brad A.; Guise, Roger B; Zanden, Brad Vander; Kosbie, David; Marchal, Philippe; Pervin, Ed; Kolojejchick, John A. "The Garnet Toolkit Reference Manuals: Support for Highly-Interactive, Graphical User Interfaces in Lisp", Carnegie-Mellon University report CMU-CS-89-196, November 1989 In Garnet system papers: KR object model, updates to prototypes (using Garnet interface builder) automatically reflected in all instances on screen. Circular gestures for rotating objects. Dynamic themes? "A high-level tool, such as the Garnet interface builder, can display a prototype on the screen, and allow the user to edit it. These edits are then automatically reflected in all instances of that prototype. For example, the designer might be changing the standard look-and-feel of the menu prototype, and immediately all menus in the system will change accordingly." [MyersBA89c] Myers, Brad A.; Zanden, Brad Vander; and Dannenberg, Roger B. "Creating Graphical Interactive Application Objects by Demonstration", Proc. SIGGRAPH 89, pp 95-104 Same title as Myer's '86 paper? In Garnet system papers: Lapidary user interface tool. Supports different styles, examples are MacIntosh-like and OpenLook-like. Also shapes of objects can be changed: square to circle, etc. Dynamic themes. Page 97: graphical replace mechanism allow wholescale replacement or modification of graphical objects at run/design time: e.g. round buttons to checkboxes, etc. [MyersBA89d] Myers, Brad A. "Encapsulating Interactive Behaviors", Proc. CHI '89, May 1989, pp. 319-324 In Garnet system papers: Encapsulate interactive behaviors into "Interactor" class objects, separate from GUI-element objects. New interactors can be defined for new kinds of input, such as gesture and handwriting recognition. Interactors can handle two-handed (multi-touch) input, e.g. mouse and touch tablet at same time. [MyersBA90a] Myers, B.A., Guise, D., Dannenberg, R.B., Zanden, B.V., Kosbie, D., Pervin, E., Mickish, A. and Marchal, P. "Garnet: Comprehensive support for graphical, highly-interactive user interfaces: The Garnet user interface development environment", IEEE Computer, Vol 23 No 11, Nov. 1990, pp 71..85; also in Baecker95a Toward the Year 2000 Garnet system: Constraint-based graphical editor for constructing GUIs: visual programming language for rapid UI prototypes. Look and feel independent GUI gadgets/objects include floating buttons, number sliders, gauges, text fields, and scroll bars. When designer changes standard look and feel, all menus in system immediately change accordingly: example is changes to shapes, font, color of buttons, slide bars. Uses Opal (language) as graphical object system. Defines "slots" as fields or instance variables, using prototype inheritance. Pop-up virtual keyboard? SIP soft input panel? dynamic themes? [MyersBA90b] Myers, Brad A. "A New Model for Handling Input", ACM Trans. on Inf. Systems, Nol 8 No 3, July 1990, pp. 289-320 Garnet system: UIMS/IDE supporting arbitrary looks and feels, programmer can control both appearance and feedback behavior of UI elements such as menus. UI defined independent of the graphics and application program via Interactors. Interactors can be attached to graphics on the fly, even while interface is running. Includes definition of "slots" as accessible dynamic parameters to an object. Shows dynamically-changeable themes? [MyersBA90c] Myers, Brad A. "Demonstrational interfaces: A step beyond direct manipulation", Carnegie Mellon



University Research Showcase, Computer Science Department, 1998 User interface: example values or example operations displayed while user is using user interface. If includes inferencing of user intent, called intelligent user interface. Example-based programming, direct manipulation. [NCR90] NCR Corporation, press release "Workstation Sales Systems to be Bought from NCR", May 15, 1990 Press-release on NCR signature capture / handwriting electronic ink system [NCSC89a] National Computer Security Center, Trusted Unix Working Group (TRUSIX) Fort Meade, Md. "Rationale for Selection Access Control List Features for the Unix (R) System", NCSC-TG-020-A, Library No. S-232,508 Study/design report on ACL access control list features for Unix: multiple permissions beyond read/write/execute access mode, multiple entries for users and for groups. Additional modes/permissions/rights can be defined. Specifies syntax of ACL entries. [Nagasawa89] Nagasawa, H., Ito, K., Horii, S., and Tanoshima, K. "Optical Character Reader Apparatus", United States Patent 4,799,271, January 17, 1989, assigned to Oki Electric Company, Limited, Tokyo, Japan. OCR character recognition matching entire words against a dictionary [Nakamura89] Nakamura, Y., Ooi, K., and Yura, K. "Hand-held data input apparatus", United States Patent 4,866,646, September 12, 1989, assigned to Toshiba. Patent on external handwriting recognition product with integrated tablet/display to attach to keyboard port, and voice output User-interface for electronic ink, editing documents Recognizes handwritten character data input through force/pressure sensitive tablet and sequentially stores result of character data [NakaneK90a] Nakane, Neiichi; Kuwabara, Naoya Ikeda; Koreeda, Hiroyuki; Aotsu, Hiroaki; Kawase Masaki; Tatsuno, Yujiro; Nonaka, Naomichi; and Suzuki, Kazunarai "Multi-window display control system", United States Patent 4,954,818, September 4, 1990 Displaying data in multiple windows [NakataniM89a] Nakatani, Munehiro "Image Processing Apparatus", United States Patent 4,817,174, March 28, 1989, assigned to Minolta Camera Kabushiki Kaisha, Osaka, Japan. Edge contour/curve detection for machine vision: for focusing cameras? [NegroponteN89a] Negroponte, Nicholas "A Personal Perspective: An Iconoclastic View Beyond the Desktop Metaphor", International Journal of Human-Computer Interaction, Vol 1 No 1, 1989, pp 109-113 User-interface: a desktop is NOT complete human communication: hand gestures, winks (body gestures), tone of voice, context of the situation, etc. [Neurogen89] CAP International, Inc. "Neurogen, Inc.: Reading Numbers with Neural Networks", Market Report, October 2, 1989. Principals: Kupfersmith. One Longwater Circle, Norwell, Massachusetts, 02061, 617-982-9500 Neurogen claims 99% correct OCR recognition on handwritten numerals, but with 30% rejection rate Report on Neurogen, other handwriting OCR recognition companies for numerals : Nestor, Hect-Nielson Neurocomputer, Siemens, REI/ Recognition Equipment Incorporated, Scan-Optics, Elsag (Italy), Recognitive Sciences, Calera, Datacopy Accutext, Caere OmniPage, Calera TrueScan [NewellA90a] Newell, Allen and Fahlman, Scott E. "Information Processing Research report WL-TR-91-1121", Information Processing Research report WL-TR-91-1121, CMU, Published 1992, Final Report for Period July 1987 - July 1990 Section Research in Uniform Workstation Interfaces: Garnet Project replaced Dante project. LISP/X11 UIMS system for allowing users to construct widget GUIs using constraints. [NichollsHR89a] Nicholls, Howard R. and Lee, Mark H. "A Survey of Robot Tactile Sensing Technology", International Journal of Robotics Research, Vol 8, No 3, June 1989, pp. 3-30 Tactile-array touchpad sensors for robotics: 2D sensors include through thickness of resistive rubber, laterally across rubber between adjacent electrodes; matrix mats of carbon fibers (nanotubes?); capacitive grid; compliant elastomeric membrane pressed on surface affecting internal reflections; Separate section on recognizing shapes from static patterns on surface array (multitouch) [NielsenJ89a] Nielsen, Jakob "CHI '89 Trip Report", http://www.useit.com/papers/tripreports/chi89.html Comments on demonstration of the Paper-Like Interface with handwriting recognition by Cathy Wolf et al from IBM, with a "formula editor" (spreadsheet/mathematical notation) application. Also a mention of gestural interaction in an application toolkit for prototyping user interfaces from Mark Rosenstein of MCC Human Interface Tool Set and "silicon paper" (electronic paper). Several paragraphs on Wang Freestyle, described as an email application with the ability to integrate graphics, spoken comments, and animated writing (miss-called "gestures"). [NielsenJ90a] Nielsen, Jakob "Traditional Dialogue Design Applied to Modern User Interfaces", Communications of the ACM, Vol 33 No 10, October 1990, pp 109-118 Review of Wang Freestyle, other pen and non-pen UIs: shows postage stamp feature of Freestyle for documents. Says scrolling of text is bad in UI design. [NolanPR89a] Nolan, Peter R. "Designing Screen Icons: Ranking and Matching Studies", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 380-384



Suggested icons for user-interface: symmetric confusions, "confusion matrix" of preferred selections [NormileD90a] Normile, Dennis and Johnson, J.T. "Computers Without Keys", Popular Science, August 1990, pp. 66-69 Soni Ai Note PalmTop with handwriting recognition and electronic ink drawing, GridPad, SuperScript, GO Corp./PenPoint (not yet released). Typing plate with boxes for handwriting recognition, with send/transfer button at end. Refers to template-matching recognizer. [Numonics89a] Numonics Corporation "Numonics Series 2300 Microprocessor-Based Digitizer Tablet", Numonics Corporation, 1989 Electromagnetic digitizer, includes commands for setting scaling, offset, origin and rotation/tilt of digitizer coordinate system. [NunezLM89a] Nunez, Linda Mensinger "professional resume", for work in on-line handwriting recognition, 1989 Resume for former employee of Nestor, Incorporated, worked on Japanese on-line handwriting recognition in 1987 [ON90] ON Technology Inc. "ON Technology, press release", April 24, 1990, Peter B. Miller to start handwriting recognition-base systems software company for Slate computers. [OoiK89] Ooi, K.; Sasaki, H.; and Ariyoshi, S. "Character recognition system", European Patent 335233 (number incorrect?) July 26, 1989, assigned to Toshiba (abstract only) Includes device for obtaining coincidence between recognition character candidates and character patterns stored in dictionary memory [OssnerJ90a] Ossner, J. "Transnational symbols: The rule of pictograms and models in the learning process", in "Designing User Interfaces for International Use", J. Nielsen, Ed., Elsevier Science Publishing, Amsterdam, 1990, pp 11-38 Cited in Nielsen90 [PCLaptop89a] Patrick, Mark and Sachs, George "Richman Maximizer PIM Contact Manager", PC Laptop magazine, October 1989, page 41 Announcement for PIM/Contact manager: MS-DOS. [PCLaptop89b] Patchett, Craig "Touchscreen and CD-ROM: Is the Scenario Dynabook too good to be true?", PC Laptop magazine, October 1989, pp. 14-22 Touchscreen DOS PC with no keyboard: main application is as electronic book with built-in data CD-ROM drive. SAW surface acoustic wave technology, pressure/force sensitive. Three-point calibration (X + Y, no tilt/rotation?) [PastorJ89a] Pastor, Jose "Reliable Document Authentication System", United States Patent 4,853,961, August 1, 1989 Authenticate documents (or, authenticate postage meter stamp) using public-key cryptography by third party. Authenticating document with public/private key shows source authentication of document, also discloses key for decrypting document. Documents can be authenticated without being read. (Seem a bit pre-certificate.) [PatrickM89a] Patrick, Mark and Sachs, George "X11 Input Extension Protocol Specification", M.I.T. X Consortium - Technical Review - X11R4 Public Release, December 1989 Extension to X graphical system standard to support devices beyond system keyboard and system pointer (mouse): event input includes Key, Button, Valuator data in motion events, proximity events, focus, feedbacks [PenPoint90a] GO Corporation "Using the GO Computer", GO Corporation internal document: September 26, 1990 Description of user interface of PenPoint operating system, 1990. Selectable screen orientation landscape/portrait. Orientation/rotation PDF page 187 Choosing a screen layout. [PentlandA90a] Pentland, A. and Kuo, J. "The Artist at the Interface", 10th International Conference on Pattern Recognition, Atlantic City, New Jersey, June 16-21, 1990 [PeppersNA89a] Peppers, N.A., Young, J.R., Pierce, G.A., Nishi, H., and Katsuki, K. "Apparatus for Preprocessing of Character Recognition", United States Patent 4,809,344, February 28, 1989, assigned to Nippon Sheet Glass Company, Limited, Osaka, Japan. Segmentation in OCR scanner input for paragraphs, words picture, and individual letters, also doing character normalization. [PeppersNA89b] Peppers, N.A., Young, J.R., Nishi, H., and Katsuki, K. "Page Segmenter", United States Patent 4,817,169, March 28, 1989, assigned to Nippon Sheet Glass Company, Limited, Osaka, Japan (abstract only). Segmenting a document image into parts "with a single feature" as pre-processing before pattern recognition [PerkinsR89a] Perkins, Ronald; Blatt, Louis A.; Workman, Daniel; Ehrlich, Susan F. "Iterative Tutorial Design in the Product Development Cycle", Proc. Human Factors and Ergonomics Society Annual Meeting, 1989, pp. 268272 (partial copy) Report on tutorial for Wang Freestyle, but indirectly a human factors report on usability of Freestyle



voice/electronic ink annotation system, pen computing system without use handwriting recognition, instead using of gestures and direct manipulation. Notes that subjects found most useful: Synchronized voice/sketching annotation in recorded whiteboard system. [PerryS89a] Perry, Tkla S. and Voelcker, John "Of mice and menus; Designing the user-friendly interface", IEEE Spectrum, September 1989, pp 46-51 Historical view (circa 1989) of graphical user interfaces: Sketchpad in 1962, lightpen in 1960's, Graphical curve-tracer with a potentiometer, mouse in 1964 by Douglas Engelbart at SRI, Trackball. Sketchpad split screen into two windows. No mention of handwriting or gesture recognition, or of Rand tablet. [PersonalWriter89] Personal Writer, Inc. "Personal Writer 10SL/15SL product literature", 1900 Avenue of the Stars, Suite 2870, Los Angeles, California 90067 (see also Anatex). Copyright 1987/1988/1989 Previously called Anatex (which see) Adaptive handwriting recognition product for MacIntosh, trainable, using a spelling dictionary [PersonalWriter90a] Personal Writer, Inc. "Personal Writer. Pen and paper has come of age", 1900 Avenue of the Stars, Suite 2870, Los Angeles, California 90067 (see also Anatex). Copyright 1987/1988/1989 Previously called Anatex (which see) Slide presentation on Personal Writer: handwriting recognition product PW10SL Version 1.0 on IBM PC, other for Apple MacIntosh. Refers to tablet as "absolute mouse" (equivalent). [PetersonS90] Peterson, Scott "Draft patent application for "The Guide"", for Simulated Input Device User Interface, Wang Laboratories, as drafted by patent counsel. Patent description for Wang/Guide SID simulated device user-interface, application independent [PfleegerCP89a] Pfleeger, Charles P. "Security in Computing", Prentice-Hall, 1989 Tutorial and reference on encryption and PKI, intrusion detection, user authentication, secure/trusted computer systems, copy protection, DRM, legal issues of (excessive) computer security [PhillipsM90] Phillips, Mark J; Philbin, Patrick T. and Blesser, Barry A. "Coordinated Determining Device using Spatial Filters", United States Patent 4,963,703, October 16, 1990 Electromagnetic digitizing tablet/pen with tilt measurement and stylus tilt (mis-named as parallax) correction: Cited later for prior art on acoustic/ultrasonics tablets [PlaisantC90a] Plaisant, Catherine and Wallace, Daniel "Touchscreen Toggle Switches: Push or Slide? Design Issues and Usability Study", Report HCIL-90-08, CS-TR-2557, CAR-TR-521, November 1990, U. Maryland Institute for Advanced Computer Studies Touchscreen virtual toggle and slider switches: users spontaneously associate slider switch with a control function. Compare to Apple slide-to-unlock. [PlamondonR89a] Plamondon, R. and Lorette, G. "Automatic signature verification and writer identification -- the state of the art", Pattern Recognition, Vol 22 No 2, March 1989, pp 107-132 Review of all current commercial and research signature verification systems -- specificallyexcludes patents [PlamondonR90a] Plamondon, R. and Yergeau, P. "A System for the Analysis and Synthesis of Handwriting", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 167-180 [Poqet89] Poqet Computer Corporation "Poqet Computer Corporation press release", October 2, 1989 Rumored portable touchscreen product: no character recognition [Quam89] Quam, David L., Williams, George B., Agnew, Jeffery R., and Browne, Patricia C. "An Experimental Determination of Human Hand Accuracy with a DataGlove", Proceedings of Human Factors Society 33rd Annual Meeting, October 16-20, 1989, Denver Colorado, pp 315-319 Pointing accuracy with human hand, but open loop (no coupling to a tablet digitizer/display tracking crosshair, for example). [QuerciaV89] Quercia, Valerie and Ro'Reilly, Tim "X Window System User's Guide for X11 R3 and R4 of the X Window System", Three volumes: O'Reilly & Associates, 1989 (hardcopy book) X-Windows documentation: mouse/pointer based windows interface, can be used with a tablet digitizer with a puck with at least three functional buttons, for mouse-button menus. No use with a stylus. Mousebutton menus can be re-defined. electronic copy also. [QuerciaV90a] Quercia, Valerie and Ro'Reilly, Tim "X Window System User's Guide OSF/Motif Edition", Three volumes: O'Reilly & Associates, 1990 (hardcopy book) X-Windows documentation for OSF/Motif windows manager system: mouse/pointer based windows interface, can be used with a tablet digitizer with a puck with at least three functional buttons, for mousebutton menus. No use with a stylus. Mouse-button menus can be re-defined. electronic copy also. [RecognitiveSciences89] Litvin, Y. and Kuklinski, T. "Recognitive Sciences Provides Cost Effective OCR Solutions for OEMs", company profile, 1989. Principals: Ted Kuklinski and Yuri Litvin. 323 Andover Street, Wilmington, Massachusetts, 01887 Kuklinski's and Yuri's O.C.R for handwritten and embossed characters



[RegtienPPL89a] Regtien, P.P.L. "Silicon Tactile Image Sensors", in "Traditional and Non-Traditional Robotic Sensors", Proc. NATO Advanced Research Workshop on Traditional and Non-Traditional Robotic Sensors, Maratea, Italy, August 28-September 2, 1989. pp. 117-126 Review of 2D tactile/touch/touchpad sensor technologies: displacement-based / force include elastomeric deformable plastic sheets, piezoresistive, piezoelectric. [RevelleGL90a] Revelle, Glenda L. and Strommen, Erik F. "The effects of practice and input device used on young children's computer control", Collegiate Microcomputer, Vol 8 No 4, Nov. 1990, pp 33-41 Joystick, mouse, trackball pointing devices: cites to earlier studies using touchscreen digitizer tablets. Child users always press and hold button for long time when intending to select, click on button unintentionally while thinking what to do next. [RobertG90a] Robert, Gregory; Chase, David; and Schaefer, Ronald "Software Licensing Management System", United States Patent 4,937,863, June 26, 1990 Software licensing: program checks in with licensing service, that keeps track of usage. If denied, check group license table. License released by program when done. [RobertsonGG89] Robertson, Geroge G., Card, Stuart K., and Mackinlay, Jock D. "The Cognitive Coprocessor Architecture for Interactive User Interfaces", Proc. 2nd Annual ACM SIGGRAPH symposium, Williamsburg Virginia, 1989, pp 10..18 User-programmable buttons, RoomsButtons experimental system extended (with other objects) to support gestures. Visual feedback on gestures? Describes a 3D and animated user interface system involving rooms that uses interactive objects (buttons) which are responsive to multiple gestures (press, rubout, check, and throw). [RosenbergJ90a] Rosenberg, Jarret (moderator): Asente, Paul; Linton, Mark; Palay, Andrew "X Toolkits: the Lessons Learned", Proc. UIST '90, pp. 108-111 X Windows needs a toolkit. Constraint and error checking in toolkit means windows implementation can be simpler, since it finds bugs and reduces need for run-time checking or complications. Like a good compiler does for an instruction set. [RubineD90a] Rubine, Dean and McAvinney, Paul "Programmable Finger-Tracking Instrument Controllers", Computer Music Journal, Vol 14 No 1, Spring 1990, pp. 26-41 (partial copy) multi-touch musical instrument touchscreen interface: see also VideoHarp, SensorFrame 92. [RyderJH90a] Ryder, John Hoyt Sr. and Smith, Susanna Ross "A method of verifying receipt and acceptance of electronically delivered data objects", European Patent Specification EP 0367700 B1, June 9, 1990 DRM: electronic verification that user accepts software/use license, entering acceptance record into object/file/program. [SAC89a] Science Accessories Corporation "Model GP-7 Grafbar Mark II Sonic Digitizer", Science Accessories Corporation, 1989 Acoustic digitizer, two microphones in instrumentation bar at top of digitizing area up to 20x20 inches. 1..4 button cursor or stylus. 70 KHz acoustic signal generated in stylus/puck. Inking stylus, support circuit diagrams. [SabourinR89a] Sabourin, Robert and Plamondon, Re'jean "Segmentation of Handwritten Signature Images: A Structural Approach", unpublished manuscript, July 26, 1989. Laboratoire Scribens, Ecole de technologie Superieure, Departement de Production Automatisee, Montreal, Quebec, Canada. Reviewed for PAMI and Henry Baird: extraction of signatures from scanned documents prior to verification [SabourinR90a] Sabourin, Robert and Plamondon, Rejean "Progress in the Field of Automatic Handwritten Signature Verification Systems Using Gray-Level Images", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 1-12 Signature verification with OCR: can tell photocopies from originals [SandigeRS90a] Sandiga, Richard S. "Modern Digital Design", McGraw-Hill, 1990 (hardcopy book) Upper-level textbook on digital design. Focuses on standard commercial components (e.g. no description of multiply) [SantoriM90a] Santori, Michael "An instrument that isn't really: LabVIEW's software approach to instrumentation systems yields an intuitive programming tool for test needs", IEEE Spectrum, August 1990, pp 36-39 LabView graphical programming language for software simulation and control of virtual laboratory instruments, describing dataflow programming. Introduction gives history of LabVIEW to 1986. Quoted in JohnsonGW94a. [SatoK89] Sato, Koji, Kurakake, Masahiro, and Fujiwaki, Kazuo "Optical Character Reader", United States Patent 4,797,940, January 10, 1989,assigned to Sumitomo Electric Industries, Osaka, Japan (abstract only). Hand-held OCR scanner which can read one line at a time, with hardware to segment into separate characters [SatsutaM90a] Satsuta, Mayasuki and Nakamura, Shoji "Multi-Window Display Control System", Japanese laid-



open patent application JP 2-155029, June 14, 1990 Semi-transparent overlapping windows using stippling / half-tone using XOR to flip pixel bits: offset if multiple windows so can see each. Double-click gesture to bring lower window to top. [ScanOptics90] Scan-Optics, Inc. "Easyreader 1720 OCR product", press release, February 1, 1990 OCR scanner product: programming via format control sheets, filled out by hand to define fields, field lengths, numeric range context. [Scenario89a] Scenario, Inc. and Visible Systems Corp. "Easy Touch/Touch Translator", structured UI design tool, PC Week, Dec 4, 1989, vol 6 no 48, p 73 Touch-screen user-interface design tool: Pen-based UI SDK. [Scenario89b] Scenario Inc "Dynabook product announcement file", October 1989, Boston, Massachusetts. Press kit for Touch-screen portable (sort of) CD-ROM product: no handwriting recognition yet. File contains additional references on Dynawriter (change of product name) with handwriting recognition. detachable resistive-film (SAW) touchscreen -- most likely EloGraphics? Name taken from Alan Kay's Dynabook? Scenario, Inc. Technical Products Summary: include consulting, Touch API/SDK utilities, Touchscreen Designer GUI developer, ELODEV TSR driver supports keyboard buffer stuffing, access to speaker, pressure data (??), user-specific clibration (left-handed writers?). Touch Translator / Dynavision for converting touchscreen menus to keyboard commands -- compare to PenCept PenPad utilities. Graphics Touch Zone Generator TouchUp, TouchBack [SchlosserPA89a] Schlosser, Philip A. and Kable, Robert G. "Electrographic apparatus", United States Patent 4,853,493, August 1, 1989 Single-resistive sheet capacitive digitizer: using resistors at contact points at edges to correct mostly for pincushion/reverse pen-cushion distortion in position. Single on one side, ground on other. [SchneidawindJ90a] Schneidawind, John "Software pioneers are reunited at hot new Arizona company", Press report, 1990 (publication not known) Bob Frankston and Dan Bricklin join Slate Corporation: pen computing applications (PenPoint, W4PC). Investment by Mitch Kapor [Scriptel90a] Scriptel "Press Information on PenWriter", Press reports circa 1990 Press reports on Scriptel: includes Ohio Gov. Richard Celeste using Scriptel transparent tablet as telautograph to sign bill concerning electronic signatures. [SearsA90a] Sear, Andrew; Plaisant, Catherine; Shneiderman, Ben "A new era for touchscreen application: High precision, dragging icons, and refined feedback", U. Maryland HCI Lab., Dep.t of Comp.Sci, CAR-TR-506 / CS-TR2487, June 1990. Also in Advances in Human-Computer Interaction, Hartson and Hix, 1993 Review of touchscreen user interface considerations, including hardware trade-offs. Mentions first-touch versus lift-off for selection and pointing. Touchscreen keyboards somewhat slower typing than regular keyboards, but fast than people thought. Accuracy of rotation gestures improved by longer lever distance by user. Microsoft UnMouse. See: multi-touch "has been explored", looking for applications when it becomes commercially available: touchscreen manufacturers will be important. SAW may be able to detect as least two touches. On-screen touchscreen keyboards more flexible. [SeymourJ90a] Seymour, Jim "The "Slate PC" Wave Is About To Break", PC Magazine, October 16, 1990, page 85..86 Editorial that pen-computing PCs (like Palm?), as small portable devices making only minor use of handwriting character recognition, would be successful. Mentions GRiD, Go/PenPoint, Momenta. No mention of Microsoft Pen-Windows. [ShearVH90a] Shear, Victor H. "Database Usage Metering and Protection System and Method", United States Patent 4,977,594, December 11, 1990 Metered database usage: database is encrypted, user's charged for amount of data fetched. Database "self destructs" if tampered with: delete decrypting key [Shu89] Shu, Joseph Shou-Pyng "One-pixel-wide Edge Detection", Pattern Recognition, Vol 22 No 6, 1989, pp 665-673 Edge detection/skeletonization/line-thinning using a 3x3 pixel filter, similar to Sobel edge detector filters, Robert 2x2 edge detectors, combined with heuristic (minimum-cost) search Edge-streak encoding, similar to chain codes for edges [Signaware89] Signaware Corp. "Software Description", 1989, 300 South Duncan Avenue, Suite 273, Clearwater, Florida 34615, 813-461-4211 Internal Wang memo on signature verification vendors: one in Florida, one in Ireland [SiliconGraphics90a] Silicon Graphics "Silicon Graphics Workstation - Desktop", ACM CHI 1990 Issue 56, Video Silicon Graphics Workstation. Available at open-video.org, 2010 Shows highlighting on selection and motion during drag in GUI for Silicon Graphics Workstation Unix, for file explorer window. Drag and drop, drop-available highlighting when dragging over a target. [SimonJC90a] Simon, J.C. and Baret, O. "Handwriting Recognition as an Application of Regularities and



Singularities in Line Pictures", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 23-38 [SinghG89a] Singh, Gurminder and Green, Mark "Chisel: A System for Creating Highly Interactive Screen Layouts", Proc. UIST '89, 2nd Annual ACM SIGGRAPH symposium on User Interface Software and Technology, pp. 86-94 Automatic screen layout in UIMS user interface definition: color and window resources management [SinghG89b] Singh, Gurminder "Automating the Lexical and Syntactic Design of Graphical User Interfaces", Ph.D. Thesis, U. of Alberta, Canada, 1989 UIMS development system. UI defined declaratively using a graphic editor / layout editor not for the user interface appearance, but for the interaction structure and positional constraints of the UI. [SinghG90a] Singh, Gurminder; Kok, Chun Hong; and Ngan, Teng Ye "Druid: A System For Demonstrational Rapid User Interface Development", Proc. UIST '90, 3rd ann. ACM SIGGRAPH symposium on User Interface Software and Technology, New York, 1990, pp. 167-177 UIMS development system for prototyping. User defines UI elements graphically (direct manipulation), animates them manually, system "learns" the UI behavior to simulate. Prototype implemented using Motif GUI. [Sklarew89a] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", European Patent Application 0 254 561 A2 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Title and abstract read almost identically to other Sklarew patents [Sklarew90a] Sklarew, Ralph "Handwritten Keyboardless Entry Computer system", United States Patent 4,972,496, November 20, 1990 U.S. Patent application on handwriting recognition user-interface, arm-held "slate" type pen computer. Patent description of prior art for Linus product, Grid patent on user interfaces: transparent digitizer over display for handwriting input, with character recognition Abstract refers to transparent sensing surface over display, electronic ink, character recognition, and learning mode/ trainable recognition. Cited for Unistroke character set. Transparent front-mounted digitizer tablet/touchscreen: cites advantages (stylus tilt, etc.) over rear-mounted tablets. Refers to trainable recognition for handwritten characters and symbols, but not to macros or gestures. [SklarewR90b] Sklarew, Ralph "Correspondence file: Ralph Sklarew", Personal correspondence, Ralph Sklarew, 1990 Technical/hardware descriptions of keyboardless touchscreen portable computer with virtual on-screen keyboard [SmallCH90a] Small, Charles H. "Inexpensive software reads printed material", Electronic Design News, January 18, 1990, pp 77-86 Review of OCR scanner companies: Caere Corporation, Inovatic, Olduvai Corporation, Xerox Imaging Systems. [Sony90] Sony Corporation "Sony hand-held computer press release", March, 1990 Hand-held slate computer product from Sony Corporation. [SpitzG89a] Spitz, Gabriel "The Preferred Pen Activation Force for the Low Resolution Freestyle Tablet", Research report 14 July, 1989, Wang Laboratories, 1 Industrial Avenue, Lowell, Massachusetts 01851 Internal Wang report on stylus pen force/pressure for handwriting force [SpitzG90a] Spitz, Gabriel and Akagi, Kenichi "Stylus Button Location", research report 3 January, 1990, Wang Laboratories, 1 Industrial Avenue, Lowell, Massachusetts 01851 Internal Wang report on barrel-switch/side-button on digitizer tablet stylus [SrihariSN90a] Srihari, S.N. "Reading Unconstrained Handwriting with Bounded Context", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 109-116 Overview of some recognition context problems: project was OCR for handwritten Zip codes, address blocks [Stentiford89] Stentiford, Frederick W.M. "Pattern Recognition System", United States Patent 4,817,171, March 28, 1989, assigned to British Telecommunications public limited company, Great Britain. OCR [StringaL90a] Stringa, L. "A Structural Approach to Automatic Primitive Extraction in Hand-printed Character Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 65-72 (abstract only) [SuenCY90a] Suen, C.Y., Nadal, C., Mai, T.A., Legault, R. and Lam, L. "Recognition of Totally Unconstrained Handwritten Numerals Based on the Concept of Multiple Experts", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University,



Montreal Canada, April 2-3, 1990, pp 131-144 [Summagraphics89a] Summagraphics Corporation "Summagraphics The Summasketch II ADB / The Microgrid II Series MacIntosh SE and II Version", Summagraphics, 1988, 1989 Electromagnetic digitizers with stylus or puck: 16 button cursor/puck, Laser Interferometer System (for production testing?), Compatible (mouse replacement) with Apple computers, menu picking. Full Multiple Monitor Support tablet/digitizer can be mapped to any contiguous group of monitors (on Apple Desktop Bus). Independent (of application) control panel configuration of scaling, offset, button mapping, and tablet orientation (90-degrees?). File includes data sheet on SummaSketchII for IBM PC. [Summagraphics89b] Summagraphics Corporation "Decisions: Summagraphics goes Hollywood", Summagraphics 1989 Summagraphics tablets in movies/films as technology prop. Mentions Dr. Halo III graphics application software. [Summagraphics89c] Summagraphics Corporation "Draw on our experience -- Summagraphics tablet modes", Summagraphics 1989 Summagraphics tablet product information: SummaSketch, Bit Pad Plus for IBM/MacIntosh: Windows driver (mouse), Autodesck Device Interface (driver). Optical mouse, Microgrid II drafting-table size. Most are electromagnetic, but one CR Charge Ratio capacitive tablet. Stylus and cursor (puck). [Summagraphics90a] Summagraphics Corporation "Summagraphics MM II 1201 Graphics Tablet Technical Reference", Summagraphics January 1990 Summagraphics electromangetic tablet: modes Point Mode, Remote Request Mode (polled input only), Stream Mode, Switch Streem Mode, Report Rate, Increment Mode, Axis Update Mode. Device driver guidelines. Stylus, 4-button and 16-button cursor (puck). Compare increment mode with press-and-hold distance. Manual setting of origin, [SunMicrosystems90a] Sun Microsystems "Open Windows Developer's Guide 1.1, User's manual", Sun Microsystems, 1990 X-Windows-like windows management system with graphical objects. pop-up menus? drag gesture? Popup virtual keyboard? Cited in Pier92 [SuzukiT90a] Suzuki, T. and Mori, S. "A Thinning Method Based on Cell Structure", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 39-52 Line-thinning / skeletonization [Taguchi89] Taguchi, Yoshinori; Yamanami, Tsuguya "Coordinate Input Device with Display", United States Patent 4,845,478, July 4, 1989 Patent on Wacom electromagnetic / magnetostrictive digitizer using cordless stylus. Does not describe sensor or switch for detecting stylus down or contact with surface. [TakahashiH90a] Takahashi, Hiroshi "Information Processing Apparatus", United States Patent 4,954,967, September 4, 1990 Scrolling bar in hardware using a linear array of microswitches to detect where finger is. Microswitches implemented as X and Y conductors, a hold in a cover sheet lets finger press the layers together at a defined location. [TangJC90a] Tang, John C. and Minneman, Scott L. "Videodraw: A Video Interface for Collaborative Drawning", Proc. CHI '90, April 1990, pp. 313-320 Video whiteboard system: actual video image of other user (or of user's hands) shown ion display aligned with whiteboard touchscreen, so that writing surface and video not separate. Actual drawing ink used: first user cannot use second user's ink, but this was advantageous for users to draw separate layers of drawing. [TaniishiS89a] Taniishi, Shinnosuke "Acoustic coordinate input device using a roughened surface to attenuate the surface wave component", United States Patent 4,853,496, August 1, 1989 Acoustic digitizer with roughened surface to reduce the surface acoustic wave (SAW) component: detection uses internal waves in the glass. Acoustic signal is generated by vibrating stylus. [TappertCC89] Tappert, Charles C. and Jeanty, H.H. "A Study of Several Accuracy-Improvement Methods for a Handwriting Recognition System", IBM Research Report, IBM T.J. Watson Research Center, P.O. Box 704, Yorktown Heights, NY 10598, November 1989 Most promising feature algorithm for improving recognition over elastic matching is pairwise comparison (but NOT tuned by hand!) Features (beyond elastic matching) are heights of endpoints, relative stroke positions (min and max extents), line extension, relative width, etc. Most handwriting recognition error for other systems was transducer/stylus % error, user error (user wrote "wrong")! Cites on study that handwriting on integrated tablet/display (electronic ink) sloppier than on separate tablet, much sloppier than on paper [TappertCC90a] Tappert, C.C., Suen, C.Y., and Wakahara, T. "On-line Handwriting Recognition - a Survey",



manuscript in preparation, to appear in IEEE Trans. Pattern Analysis and Machine Intelligence: C.C. Tappert, IBM T.J. Watson Research Center, Yorktown Heights, N.Y 10598 Historical development of elastic matching for script, discrete handwriting character recognition Commercial handwriting input systems listed [TappertCC90b] Tappert, Charles C. "Rationale for Adaptive Online Handwriting Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 13-22 [TappertCC90c] Tapper, Charles C., Suen, Ching Y,. and Wakahara, Toru "The State of the Art in On-line Handwriting Recognition", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 12 No 8, August 1990, pp 787-ff Comprehensive review of handwriting/gesture recognition technologies as of 1989. Gives historical remarks pen-computing in the 1950s, 1960's, 1970's, 1980's. Cited in Agulnick 295 patent. [TaxtT89a] Taxt, Torfinn; Flynn, Patrick J.; and Jain, Anil K. "Segmentation of Document Images", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 11 No 12, December 1989, pp 1322-1329 Statistical classifier to separate text and background images on documents Document segmentation using contextual Bayes, Besag's, relaxation, Owen and Switzer, versus Haslett's classifiers: which is best varies [TaxtT90a] Taxt, Torfinn; Olafsdottir, Jorunn B.; and Daehlen, Morten "Recognition of Handwritten Symbols", accepted for Journal of Pattern Recognition, 1990 O.C.R. on handwriting with 98.6% and 97.7% accuracy O.C.R. of handwritten map symbols, using several edge detection, spline curve fits, Fourier expansion, and statistical classification (lower-case alphanumerics) O.C.R. of handwritten lower-case alphanumeric using circular graphs [Teh89] Teh, Cho-Huak and Chin, Roland T. "On the Detection of Dominant Points on Digital Curves", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 11 No 8, August 1989, pp 859-872 Scale-space filtering (size-dependent) of small features in machine vision. Keywords: chain code, curve tracing, polygonal approximation, tangential deflection. Points of high curvature are dominant features for pattern recognition. Lots of references on chain codes and curve tracing for machine vision. [Termiflex89] Termiflex Corporation "TouchGraf Keyless Keypad: the Next Generation of Flexible Terminals", 316 Daniel Webster Highway, Merrimack, NH 03054, March 1989 Handwriting capture terminal product, separate digitizer built into a hand-held terminal: digitizer/touchpad and display separate [TeulingsHL90a] Teulings, H.-L. "Handwriting movement control - Research into different levels of the motor system", Doctoral Thesis, Nigmeegs Instituut voor Cognitive-onderzoek en Informatietechnologie, Nijmegen, 1990 [TeulingsHL90b] Teulings, H.-L. and Schomaker, L.R.B. "A Handwriting Recognition System Based on the Properties and Architectures of the Human Motor System", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 195-ff. On-line recognition with a tablet give better results than off-line OCR grayscale recognition, because motor information available. [TogawaF90a] Togawa, Fumio and Hirose, Hitoshi "Handwritten Character-Recognizing Apparatus for Automatically Generating and Displaying Character Frames", United States Patent 4,953,225, August 28, 1990 boxed/boxless recognition with boxes generated to right of current/previous handwriting character as user writes: automatic spacing of characters, stroke parsing / segmentation. [Toshiba89] Toshiba America Information Systems, Inc. "Toshiba PenPC product information", Toshiba America press release November 3, 1989, 714-455-1707 Pre-announced portable handwriting recognition product for ComDex 1989: kludged together from Linus, other parts? (Includes separate announcement about Linus Technologies (Sklarew). DOS 3.3 OS. Mentions mouse emulation with touchscreen digitizer. Does not mention PenPoint or Pen Windows tablet computers. [TsuchiyaH90a] Tsuchiya, Huraki "Electronic Book", European Patent EP 0390611 A2, October 3, 1990 Electronic book with update/loading by wireless communications or floppy-disk/laser card, etc. Described as a personal computer deprived of functionality such as full keyboard and printing. Page-turning user interface is small set of dedicated keys/buttons. Annotation? Shows fold-out device with two-part display; compare with Refalo? [Uncapher89] Uncapher, Kieth "Interview with Kieth Uncapher", Arthur L. Norberg, 10 July, 1989. Charles Babbage Institute, Center for the History of Information Processing, University of Minnesota, Minneapolis. Available at http://www.cs.utexas.edu/users/umair/cs370/interviews/oh174ku.rtf Interview with Kieth Uncapher of RAND Corporation: mentions GRAIL system with electronic ink on the Rand tablet. Describes use of tablet for flow-chart and programming text entry. Describes invention of the disc drive (to replace the drum drive) as a result of this project. Mentions a video film showing the Rand tablet and handwriting recognition in operation in the 1960's. Rand tablet invented by Tom Ellis around 1966: also an earlier tablet by Herb Teager at Boston College (Graphacon?). Other work by Gabe Groner,



and Bill Sibley. [Unser89] Unser, Michael and Eden, Murray "Multiresolution Feature Extraction and Selection for Texture Segmentation", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 11 No 7, July 1989, pp 717-728 Segmentation of textured images for machine vision: refers to Fisher's multiple linear discriminants and Karhunen-Loe`ve transform [VanLeeuwen89] Van Leeuwen, H.C. "A development tool for linguistic rules", Computer Speech and Language, Vol 3 No 1, January 1989, pp 83-104 Phonology, grapheme-to-phoneme conversion rules for text-to-speech [Vosburgh89] Vosburgh, A.M. and Brown, N.R. "Following the bouncing ball: An evaluation of a self-pacing enrollment interface for a large-vocabulary speech recognition system", Research Report RC-64372, IBM Thomas J. Watson Research Center, Yorktown Heights, New York, 1989 Brown,NR89 on user interface / enrollment for adaptive recognition [Wacom89a] Wacom, Inc. "Wacom PC Installation and Utilities Manual", Park 80 West Plaza II, Saddle Brook, NJ 07662, November 1989 Describes emulation and multiple command sets for digitizer controller/driver Digitizer tablet with absolute and relative (i.e. mouse) modes [Wacom89b] Wacom, Inc. "Super Digitizer product information", West 115 Century Road, Paramus NJ 07562, August 1989 Product literature on Wacom force/pressure-sensitive pen stylus and cordless pen stylus tablet. Product literature describing Wacom cordless pen stylus using resonating coil in tablet digitizer. 1986 CIC memo lists a katakana and Roman and kanji system under development. Digitizer tablet with absolute and relative (i.e. mouse) modes. [Wacom89c] Wacom, Inc. "Wacom SD-420 12x12 Cordless Digitizer / Super Digitizer, SD-320 15x15, SD-210 18x25", Wacom, 1989 Digitizer tablet with cordless, battery-free stylus and or cursor (puck). SP-300 pressure-sensitive stylus. SD-421L has eletrostatic adhesion surface to hold papers. Proximity == cursor reading height. Correspondence on WT-460M digitizer for handwriting character recognition -- other clients? [Wacom90a] Wacom, Inc. "Wacom Sapphire tablet and mouse", Wacom, 1990 (date approximate) (physical device) Electromagnetic tablet (Wacom), small format, pressure/force-sensing cordless stylus, also includes "mouse" -- mouse-styled wireless puck. [WakimotoK90a] Wakimoto, K., Shima, M., Tanaka, S., and Maeda, A. "An Intelligent User Interface to an Image Database Using a Figure Interpretation Method", 10th International Conference on Pattern Recognition, Atlantic City, New Jersey, June 16-21, 1990 [WalkerJT90a] Walter, Jame T. and Kausch, Marvin L. "Video Overlay Image Processing Apparatus", United States Patent 4,954,970, September 4, 1990 Overlay image on screen, using separate table *degree* of mixing (0 transparent to 1 opaque) for each pixel.: 1 is complete masking or protection. [Wang89a] Wang Laboratories "The Wang Freestyle System", Wang Laboratories product literature Wang Freestyle: pen-computing electronic-ink and voice annotation, no character recognition. Used customized Hitachi tablet, with first (?) double-ended stylus with an eraser on the back end. Pure electronic ink and selection application system, with synchronized voice: interesting case of human communication. Standalone Demo diskette (5.25"). [Wang89b] Wang Laboratories "Freestyle: A Multimedia Conferencing System", Video: ACM CHI 1989 Issue 45 Video Presentations: Available (2009) from http://www.open-video.org/details.php?videoid=8067&surrogate=clip Wang Freestyle: pen-computing electronic-ink and voice annotation, no character recognition. [Wang90a] Wang Laboratories "Freestyle: Wang Freestyle/Light User's Guide, Release 1.21", Wang Laboratories, Lowell Massachusetts Wang Freestyle: pen electronic-ink and voice annotation, no character recognition. Used customized Hitachi tablet, with first (?) double-ended stylus with an eraser on the back end. Describes image-only document (voice/electronic ink/imaging) data base using no handwriting recognition. No use of gestures. Cited by Bill Buxton. [Wang90b] Wang Laboratories "Freestyle: Take a look at the personal communication system you already know", Wang Laboratories, Lowell Massachusetts 5.25" floppy demo of Wang Freestyle, with brochure explaining the Freestyle UI: pen electronic-ink and voice annotation, no character recognition. [Wang90c] Wang Laboratories "Equipment/Device: Wang Freestyle miniature keyboard ca. 1990", Wang Laboratories (physical device) Small physical keyboard with pressure switches, suitable for activation with a stylus. Full 102-key keyboard, including function and shift keys. Intended to be used in conjunction with Freestyle tablet digitizer for



Freestyle UI. Likely a prototype. [Wang90d] Wang Laboratories "Wang Freestyle Installation Guide, Release 1.2", Wang Laboratories 1990 (Partial copy) Installation guide for PC version of Wang Freestyle, handwriting/direct-manipulation office software with whiteboard and audio recording. [WangJS89] Wang, J.S. "Electronic text-graphic editing system", United States Patent 4,829,470, May 9, 1989, assigned to IBM Corporation. Text-graphic editing system includes mixed object editor which flows text around into irregularly shaped objects on page (for a mark-up language, like runoff for text composition in documents) [WangPSP89a] Wang, P.S.P. and Zhang, Y.Y. "A Fast and Flexible Thinning Algorithm", IEEE Trans. Computers, Vol 38 No 5, May 1989, pp 741-745 Compares and shows results of different skeletonization/thinning algorithms for character and arbitrary shapes [WangPSP89b] Wang, P.S.P. "A New Approach for Chinese Handwriting Recognition", Proc. Int. Conf. on Electronic Imaging, Pasadena, pp. 373-379, 1989 On-line (D.C.R.) recognition with very little system training/learning for Chinese. Chinese recognition using stroke angles in 22.5 degree ranges (chain codes?) [WardJR89a] Ward, Jean Renard "Testing tablet characteristics", Internal Report, Wang Laboratories, May 1989 Technical instructions on testing tablets to reveal bad behaviors for electronic ink, handwriting character recognition Describes differential vs. absolute error, spirograph tests, etc. [WardJR89b] Ward, Jean Renard "Tablet error mechanisms and performance analysis for Freestyle", Internal Report, Wang Laboratories, May 1989 Technical instructions on testing tablets to reveal bad behaviors for electronic ink, handwriting character recognition Describes differential vs. absolute error, spirograph tests, etc. [WardJR90a] Ward, Jean Renard and Kabel, Robert "The "How to" of Electronic Ink: Tablet Error Mechanisms and Performance Analysis for Handwriting Capture", Research report, Wang Laboratories, July 1990 Descriptions of "bad behaviors" in digitizing tablets, engineering review of the effects, test procedures for showing the bad behaviors [WardJR90b] Ward, Jean Renard "One View of On-going Problems in Handwriting Character Recognition", in Frontiers in Handwriting Recognition, C.Y. Suen, editor, Proceedings of the Conference on Frontiers in Handwriting Recognition, Concordia University, Montreal Canada, April 2-3, 1990, pp 101-108 Biggest obstacle to on-line handwriting recognition/gesture technology is lack of developed applications. [WareC89a] Ware, Colin and Baxter, Curtis "BAT Brushes: On the Uses of Six Position and Orientation Parameters in a Paint Program", Proceedings of 1989 CHI conference, 1989, pp 155-160 User-interface on simulating a brush with a stylus using stylus rotation angle, x/y/z digitizer tablet [WatsonJS89a] Watson, James S. "Ground Switching Technique for Silkscreened Digitizer Grids", United States Patent 4,853,499, August 1, 1989 Ground switching in transparent electromagnetic digitizer using ink-on-glass, "passive stylus". Two-layer grid. [Weideman89] Weideman, W.E., Manry, M.T. and Yau, H.C. "A comparison of nearest neighbor classifier and a neural network for numeric handprint character recognition", IEEE International Conference on Neural Networks, Washington, D.S., 1989 Martin,GL90 [WenzelEM90a] Wenzel, Elizabeth M.; Fisher, Scott S.; Stone, Philip K; and Foster, Scott H. "Virtual Acoustics Displays", Proc. of Visualization '90, IEEE, San Francisco, October 23-26, 1990, pp. 10-20 3D audible/acoustic display coordinated with real-time gesture, three-dimensional visual objects. Auditory cues/icons using Auditory Cue Editor. Artificial/virtual reality in general user interface. Three-dimensional psychophysical/psychoacoustic audio synthesized. Numerous references to Buxton, Foley, Egan for audio cues. [WhiteSR90a] White, Steve R. and Comerford, Liam "ABYSS: An Architecture for Software Protection", IEEE Trans. on Software Engineering, Vol 16 No 6, June 1990, pp. 619-629 Software copy protection / rights authorization based on passing rights-to-execute vi a token. Enforcement by protected/secure (logically, physically, procedurally secure) processors, encrypted/secure communications between secure processors. Rights-to-execute can be time-limited, dependent on callers, etc. Use count enforced by limited number of use-once rights-to-execute. Can be extended to enforce additional terms of use (contract terms). [WiechaC89a] Weicha, Charles (chair); Bois, Sstephen; Green, Mark; Hudson, Scott; and Myers, Brad "Panel: Direct manipulation or programming: how should we design interfaces?", Proc. 2nd ACM SIGGRAPH Symp. on User Interface Software and Technology, UIST '89, pp. 124-126 Direct manipulation to create user interface vs. text specification: real question is how to combine these



two. [WolfCG89a] Wolf, Catherine G., Rhyne, J.R., and Ellozy, H.A. "The paper-like interface", in "Designing and Using Human-Computer Interfaces and Knowledge-Based Systems", G. Salvendy and M.J. Smith, Eds., Elsevier Science Publishers, Amsterdam, 1989, pp 494-501 Description of paper-like interface PLI projects at IBM: electrostatic/capacitive digitizer (Scriptel) over LCD display, exploration of gesture input. Video demonstration shown in IBM lobby. Gestures described as handmarkings for user-interface in spreadsheet (Lotus) and drawing applications. Music editing demonstration modeled after Buxton. Two-dimensional mathematics editing. Note: video demonstration filed under Ellozy88a. [WolfCL90] Wolf, Chris L; Maikowski, Michael F.; Martin, Winston N.; and Lindsley, Walter E "System for selectively modifying code generated by a touch type keyboard upon detecting of predetermined sequence of make codes and break codes", United States Patent 4,937,778, June 26, 1990 Make use of roll-over on keyboard to use space bar as a shift key, by sequential decoding of make/break codes on keyboard keys [WooA90a] Woo, Andrew; Poulin, Pierre; and Fournier, Alain "A Survey of Shadow Algorithms", IEEE CGA November 1990, pp. 13-32 Rendering techniques for photorealistic shadows in virtual images. Some attempt to account for specular reflection, reflective properties of bumpy surfaces. Bump-mapping, particle shadows, polygonization [YamadaK89a] Yamada, K., Kami, H., Tsukumo, J., and Temma, T. "Handwritten Numeral Recognition by Multilayered Neural Network with Improved Learning Algorithm", Proceedings of International Joint Conference on Neural Networks, Washington, D.C., June 18-22, 1989, Vol II pp 259-266 OCR for handwritten numerals using neural network. Gives samples of low-quality, but real-world handwriting test data for numerics for OCR. [Yamaguchi89] Yamaguchi, Mikio and Sato, Koji "Optical Character Reader", United States Patent 4,817,185, March 28, 1989, assigned to Samitomo Electric Industries, Limited, Osaka, Japan. Hand-held scanner for reading price tags [YoungDA90a] Young, Douglas A. "The X Window System: Programming and Applications with Xt: OSF/Motif Edition", Prentice-Hall, 1990 (hardcopy book) X-Windows programming: describes setting resource values (providing values) to affect the way a widget appears or behaves by specifying values for resources: XtSetValue, XtGetValues [YuSS90a] Yu, Shiaw-Shian and Tsai, Wen-Hsiang "A new thinning algorithm for gray-scale images by the relaxation technique", Pattern Recognition, Vol 23 No 10, October 1990 pp 1067-1077 Uses chain codes context to preserve line straightness in line thinning: for sketch recognition? Straightness of lines very important in Chinese/Kanji character recognition [ZandenBV90a] Zanden, Brad Vander and Myers, Brad A. "Automatic, Look-and-Feel Independent Dialog Creation for Graphical User Interfaces", Proc. CHI '90, April 1990, pp. 27-34 Garnet system: Jade UI definition UIMS: UIMS supporting arbitrary looks and feels. Dialog box ore menu look and feel changed on the fly by switching look and feel databases. Jade obtains graphic information from (multiple) style files, example changing Garnet style dialog to OpenLook style, style can include changes to layout or graphics or adding elements, position of objects in dialog. Shows dynamicallychangeable themes? [ZhuangX89a] Zhuang, X., Haralick, R.M., and Joo, H. "A Simplex-Like Algorithm for the Relaxation Labeling Process", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 11 No 12, December, 1989, pp 1316-1321 biblio88 (Prev page)

Main Page







Main Page


[ATTHobbit93a] A.T. & T. "ATT Hobbit pen computer running PenPoint (functional unit)", Personal collection of Jean Renard Ward, [email protected] (May not be functional) PenPoint "Hobbit" pen computer from AT&T, includes manuals. Appears to be functional as of April 2011, PenPoint Operating System. [Abandonware93a] winworldpc.com "Microsoft Windows for Pen Computing 1.0.7z", wdl2.winworldpc.com, fetched 2012 Archive of installation file for Windows for Pen Computing 1.0 / Pen Windows -- self-install zip file is in electronic video files. [Aha92a] Aha! Software Corporation "aha! InkWriter with SmartInk reference card / Basics", Aha! Software corporation, 1992 Manual for InkWriter for PenPoint: electronic-ink scribble editing software, later acquired by Microsoft. Supports delays/deferred handwriting recognition, indexes ink by the recognition results. Editing gestures generally similar to PenPoint: X to delete, caret gesture to insert, B to make bold, press-and-hold to start selection. Compare with van Raamsdonk. [Aha93a] Aha! Software Corporation "aha! InkWriter Handbook", Aha! Software corporation, 1993 Manual for InkWriter for PenPoint: electronic-ink scribble editing software, later acquired by Microsoft. Supports delays/deferred handwriting recognition, indexes ink by the recognition results. Compare with van Raamsdonk. [Aha93b] Aha! Software Corporation "aha! InkWriter Summary", Aha! Software corporation, 1993 Summary of commands (not gestures_ for InkWriter for PenPoint: electronic-ink scribble editing software, later acquired by Microsoft. Supports delays/deferred handwriting recognition, indexes ink by the recognition results. Compare with van Raamsdonk. [AkeleyK93a] Akeley, Kurt "RealityEngine Graphics", Proc. SIGGRAPH 93, Computer Graphics vol 27, pp. 109-115 Subpixel image blending as alternative to simple alpha blending. Discussion of hardware accelleration of different commercial systems. [AllenD92a] Allen, Dennis "Pen-Input Systems", BYTE Magazine, April 1992, P. 10



Part of a series on pen-input systems and PDAs [AllenG93] Allen, Gregory "Data Input Grid for Computer", United States Patent 5,214,428, May 25, 1993 mechanical grooves restrict the motion of a digitizer stylus to octant chain-code directions: shows a set of single-stroke characters for recognition [AlsonS91a] Alson, Stewart "NCR takes Expo by storm with pen-based computer", InfoWorld, July 1, 1991 NCR 3125 pen computer runs either PenPoint for Windows for Pen-based Computing (sic). [Amstrad93] Amstrad93 "Amstrad PenPad PDA 600", Reference available at www.oldcomputers.com/museum/doc.aps?c=876 Z-80 handheld pen-computer/touchscreen, 1993, styled as fold-out notebook (Refalo?). Contains quotes from project manager Cliff Lawson, Mark Stevenson, Wulf Broening: software by German company Wiedmann Additional materials in file -- shows touch buttons on outside edge of display area? [ApikiS93a] Apiki, Steve "A Unified Ink Standard", BYTE Magazine, September 1993, p. 26 JOT standard for electronic ink storage? [AppleNewton93a] Apple "Apple Newton product", (various) File of references to Apple Newton: August 1993 introduction? Hand-held PDA, resistive touchscreen, scribble/rubout gesture. Includes comparison of Palm PDA and Apple Newton for physical size (Newton too big for hand) [ApplePenLite93a] Apple "Apple PenLite Tablet Computer", (Various) Electronic file of references to Apple PenLite, a pen-computer based on Apple PowerBook Duo from 1992/1993. Project cancelled 1994 in favor of Apple Newton PDA. Most of material in German. See also video "Watch Apple PenLite Video". References obtained October 2012. [ApplePenLite93b] Apple "Watch Apple PenLite Video", YouTube.com Video of Apple PenLite pen-computer prototype, circa 1993. Mac PowerBook Duo with touchscreen, same UI: touchscreen/stylus replaces mouse as locator device. [Apte93] Apte, A. and Kimura, T.D. "A Comparison Study of the Pen and the Mouse in Editing Graphic Diagrams", Proc IEEE 1993 Symposium on Visual Languages, 1993, Bergen, Norway, pp 352-357 Citrin96 cites for single-stroke graphical editing gestures: comparison of NoteTaker on PenWindows / Windows for Pen Computing without gesture (menu picks only) and the GDE gesture-based graphical editor in GO/Penpoint. Concludes that for drawing, gesture-based editor with a pen/stylus is preferred by users because modes do not have to be changed to a menu. [ArmbrusterJL93a] Armbruster, Jeffrey L. "PenRight! Pro SDK 3.1", Windows/DOS Developer's Journal, Vol 4. No. 3, March 1993, pp. 45-48 (Abstract only) PenRight. Pen Computing development environments: PenRight! for Windows review article, 1993 [AscensionTechnology91a] ascension-tech.com "Ascension Technology: The Flock of Birds - Position and Orientation Measurement System, Installation and Operation Guide for Standalone and Multiple Transmitter/Multiple Sensor Configurations", Ascension Technology Corporation, 2004 (first version 1991) 3D/6DOF "tablet" position sensors, used in a number of hand-gesture projects, virtual reality inputs (i.e. sense position and orientation of toy "gun". Multiple small electromagnetic sensors with cables to match one or more multiple base transmitters. Original version of manual 1991. [AvitzurR92a] Avitzur, R. "Your own handprinting recognition engine", Dr. Dobb's Journal, April 1992, Vol 17 No 4, pp 32-37 combination Chain-code/zone-based handprinting/handwriting recognizer. Open-source. Csspne.tk blog posting says Avitzur also author of Milo mathematical equation editor for Apple Mach. [BadlerNI91a] Badler, Norman I. "Human Body Modeling and Simulation", US Army Research Office Report ARO 28131.1-MA, December 1991 Mentions Ascension Technology 6DOF/3D digitizer sensors used in robotic input for model of human body. [BaeckerR91a] Baecker, Ron; Small, Ian; and Mander, Richard "Bringing icons to life", CHI '91 Proc. SIGCHI Conf. on Human Factors in Computing Systems pp. 1-6; also in Baecker95a Toward the Year 2000 Animated icons in GUI: animation can show behavior of tool/operation selected by icon. Example is a pile of documents, top document can be removed from pile. Compare with Freestyle? [BalasubramanianPS93a] Balasubramanian, P.S.; Levy, L.; and Yarm, J. "Image Orientation Sensing and Correction for Notepads", IBM Research Disclosure n347 03-93, March 01, 1993 Notepad/portable computer with touchscreen, sensors detect rotational orientation of display, adjust user interface for landscape/portrait layout. [BallBR91] Ball, Brian R. "Readings in human-computer interaction: Toward the Year 2000", Morgan Kaufmann Publishers, 1995, ISBN 155860-246-1 (partial copy) Chapter "Touch, Gesture, and Marking": device independence and virtual devices, equivalence of various pointing/locator devices, mouse/joystick, button/blow-suck tube, touchscreen, etc. [BarlowDC93a] Barlow, Douglas C. "Computer Network Operating with Multilevel Heirarchical Security with Selectable Common Trust Realms and Corresponding Security Protocols", United States Patent 5,204,961, April



20, 1993 Strick heirarchy of trusted systems, systems enforce specific security policies for systems in particular security/trust realms. All inter-system messages sealed (signed/encrypted) and authenticated PKI. [BarrettDM93a] Barrett, David M., McNally, J. Michael, Martin, Patricia A., Huntington, Monathan T. II and Douthart, Robeert M. "Computer with separate display plane and user interface processor", United States Patent 5,260,697, November 9, 1993 Divisional application of UI work by Ward, this one is for the hardware. UI include handwriting recognition bar and simulated devices. Handwriting bar on The Guide at Wang. [BartholomewJ92] Bartholomew, Jim "Microsoft Visual Tools Suite for Windows Ver 1.5", www.accu.org, 1992 Review of Microsoft development tool suite. Mentions VBX controls added to Control Palette in Visual C++ App Studio or Visual Basic Toolbox, four controls for Windows for Pen Computing Windows for Pen Computing development environments: Visual C++ App Studio graphical environment support VBX controls for Windows for Pen Computing. [BartlettJF92a] Bartlett, Joel F. "Transparent Controls for Interactive graphics", WRL Technical Note TN-30, Digital Western Research Laboratory, July 1992 Shows transparent control panel / menu window with buttons over another image. Uses stippling to get the see-through effect. [BaudelT93a] Baudel, Thomas and Beaudouin-Lafon, Michel "Charade: Remote Control of Objects Using FreeHand Gestures", CACM, Vol 36, No 7, July 1993, pp. 29-35 Hand-gesture control of presentation graphics using DataGlove: next-chaptere gesture is open hand (start) wiping movement of hand (flick gesture to right), close hand. Syntax is Start/Dynamics/End. Gesture recognized by angle/orientation of hand, open and closed fingers and thumb, bendings of fingers. [BelgeM93a] Belge, Matt; Lokuge, Ishantha; and Rivers, David "Back to the Future: A Graphical Layering System Inspired by Transparent Paper", Proc. CHI '93 INTERACT '93 and CHI '93 Conf. Companion on Human Factors in Computing Systems pp. 129-130 Human factors study of transparent/translucent/opaque layers in graphical user interfaces (e.g. CAD GUIs). Users would determine what layer something was on from optical cues, or by moving one layer and seeing which objects moved (because they were in that layer). Transparency/blending done with Gain Momentum GUI system. [BennettWE91] Bennett, William E., Boies, Stephen J., Davies, Anthony R., Etzold, Karl-Friedrich, Rodgers, Todd K. "Optical Stylus and Passive Digitizing Tablet Data Input System", United States Patent 5,051,736, September 24, 1991 Optical stylus digitizer utilizing extremely small patterns within a cell, some similarity to Anoto [BierEA91a] Bier, Eric A. "EmbeddedButtons: Documents as User Interfaces", Proc. UIST '91, Nov 11-13, 1991, pp 45..53 Arbitrary document elements may behave as buttons, including graphics and text, and document may be control panel for an application. Describes multi-state button. double-click gesture on a button? Button also shows state? Has pop-up menus. [BierEA91b] Bier, Eric A. and Freeman, Steve "MMM: a user interface architecture for shared editors on a single screen", Proc. UIST '91, Nov 11-13, 1991, pp 45..53 MMM: Multi-Device Multi-User Multi-Editor. Conference editor, everyone uses the same screen: whiteboard with multiple remote pointers, etc. Users can work simultaneously on different parts/attributes of document, can also collaborate: e.g. two-mouse (multi-touch) zoom/pinch gestures. Double select highlighted (overlapping) if two users select same thing or have overlapping selections. Multiple input devices on one machine for multiple users, or remote machines connecting to central server. [BierEA91c] Bier, Eric A. and Pier, Ken "Documents as user interfaces", Proc. CHI '91 SIGCHI Conf. on Human Factors in Computer Systems, pp 443..444 Arbitrary document elements may behave as buttons, including graphics and text, Has pop-up menus. Button action is user-specifiable script. Small targets (buttons) targettable using "gravity" snap-to feature of Gargoyle. [BierEA93a] Bier, Eric A.; Stone, Maureen C; Pier, Ken; Fishkin, Ken; Baudel, Thomas; Conway, Matt; Buxton, William; and DeRose, Tony "Toolglass and Magic Lenses: The See-Through Interface", Proc. SIGGRAPH '93, pp. 73-80 Two-handed input (touchscreen, stylus/tablet, trackball and mouse cursor: multitouch?) trackball moves a virtual transparent layer, for transparent objects, such as a transparent selection/framing tool and a mouse click to make an action happen, for example a virtually-transparent color selector or shape selector. dropshadow selector [Blaszczak93] Blaszczak, Mike "Presentations delivered: Mike Blaszczak", www.nwlink.com/~mikeblas/speaks.htm Presentations on Windows for Pen Computing in Visual Basic, 1993. Windows for Pen Computing development environments: mentions whitepaper presented in 1993/1992 concerning development in



Visual Basic and Windows for Pen Computing [BlonderGE92a] Blonder, Greg E. and Boie, Robert A. "Information Processing / Capacitive Moments Sensing for Electronic Paper", United States patent 5,113,041, May 12, 1992 X/Y electrostatic/capacitive grid tablet, uses analog interpolation instead of digital/calculated: analog circuitry in stylus looks for zero-crossing of induced signals. Force-dependent capacitor for measuring tip force/pressure (film/sheet on tablet pressed towards lower grid). No reference to Bell Labs 1989/1991 "capacitive moments" report cited in electronic instrument references e.g. Jaffe 93 essay. (Note: Different citations give different title for same authors and patent number.) [BloombergDS91a] Bloomberg, Dan S. "Multiresolution Morphological Approach to Document Image Analysis", ICDAR, Saint-Malo, France, October 1991 Image analysis for extracting text areas from scanned document images. Uses bitblt/raster operations for optimation to word/byte memory boundaries in computer bitmaps. [BloombergDS92a] Bloomberg, Dan S. and Tow, Robert F. "Adaptive Scaling for Decoding Spatially Periodic Selfclocking Glyph Shape Codes", United States Patent 5,091,966, February 25, 1992 DataGlyphs: binary patterns for encoding information on printed paper, glyphs formed by angle of small marks, darkness of marks used for pixels of image instead of dots. Compare with Anoto, other references. [BlySA93a] Bly, Sara A.; Hodges, Jeffrey D.; Kupfer, Michael D.; Lewis, Brian T.; Tallan, Michael L.; and Tom, Stephen B. "Updating local copy of shared data in a collaborative system", United States Patent 5,220,657, June 15, 1993 Project and resource management system [BoltRA92a] Bolt, Richard A. and Herranz, Edward "Two-handed Gesture in Multi-Modal Natural Dialog", Proc. ACM UIST '92, Nov 15-18, 1992, Monterey, CA, pp. 7-14 Discussion of multi-hand gestures in free space by human user subjects: includes rotate gestures for rotating in 3D. Experimental hardware used DataGloves by VPL Research (finger positions) and Polhemus Navation Sciences 6DOF position/orientation sensor. [BonadioAR93a] Bonadio, Allan R. "Apparatus and method for interactively manipulating mathematical equations", United States Patent 5,189,633, February 23, 1993 GUI graphical user interface for mathematical expression parsing and editing. Expressions are typed in, then can be manipulated/evaluated using direct manipulation editor. [BourlandJD91a] Bourland, Joe D; Barbs, Charles F.; Geddes, Leslie, Tacker, Willis A. Jr.; and Graber, George P.; "Pressure Mapping System with Capacitive Measuring pad", United States Patent 5,010,772, April 30, 1991 Multi-point pressure measuring pad for patients on a bed, using array of capacitive sensor nodes formed by grid of conductors with elastomeric rubber material between. Compare with multi-touch [BoyceJ92a] Boyce, Jim "Maximizing Windows 3.1", New Riders Publishing, 1992 (hardcopy book) Multiple color schemes defined in control.ini file of Windows 3. Dynamic themes/scheme management in Windows for color schemes. [BricklinD93a] Bricklin, Dan "JOT defines electronic ink", BYTE Magazine, Vol. 18 No 11., October 1993, page 110 (Abstract only) Short description of JOT standard for electronic ink data for GO/PenPoint and Microsoft Windows for Pen Computing. [BrocklehurstER91a] Brocklehurst, E.R. "The NPL electronic paper project", International Journal of Man-Machine Studies, Vol 34 No 1, January 1991, pp 69-95 (Abstract only) cross-out gesture [BrocklehurstER91b] Brocklehurst, E.R. and Kenward, P.D. "Preprocessing for Cursive Script Recognition", NPL Report DITC 132/88, 1988 Cited in FordDM91a [BruggeJ92a] Brugge, John "Introduction to PenPoint", MacTECH,The Journal of Macintosh Technology: available at http://www.mactech.com/articles/frameworks/6_2/PenPoint_Brugge.html Overview article on PenPoint from about one year after PenPoint OS announced in Apple journal. Detachable (intermittent) networking, gestural support, notes handwriting recognition is limited but not described as a problem in context of operating system. [BryanM93a] Bryan, Marvin "The MacWrite Pro Book", Sybex MacIntosh Library, 1993 (hardcopy book) MacWrite word processing / electronic publishing [BucklandMK92a] Buckland, Michael K. "Emanuel Goldberg, Electronic Document Retrieval, and Vannevar Bush's Memex", J. Am. Soc. for Information Science, Vol 43 No 4, May 1992, pp 284-294 Vannevar Bush's Memex 1945 likely anticipated as a technical development by 1931 Patent by Emanual Goldberg, and reference would/should have been known. Also mentions 1934 work by Paul Otlet, 1935 by Walter Schuermeyer and electronic document retrieval, others. [BurdeaG92a] Burdea, Grigore; Roskos, Edward; Silver, Deborah; Thibaud, Francois; and Wolpov, Robert "A Distributed Virtual Environment with Dextrous Force Feedback", Proc. Informatique '92, International Conference



Interface to Real and Virtual Worlds. pp. 255..265, Nanterre, France: EC2 Hand controller with two fingers + thumb force feedback. No haptic/tactile feedback, intended to be used with Dataglove for input. [BurskyD92a] Bursky, Dave "Chip Set Simplifies the Design of Pen-Based PCs", Electronic Design, November 2, 1992 PICK (Pen Interface Controller Kit) digitizer interface chip set from TriTech Electronics, to support Scriptel electrostatic digitizer in pen-computing. Refers to changes in pen-position frequency (from 100KHz?) to deal with environmental noise from backlight, etc. [Buxton91b] Buxton, Bill "GEDIT - The Use of Single-Stroke Marks (Marking Menus) in Graphical Interaction", www.billbuxton,com, http://www.youtube.com/watch?v=0A21YfqEa1c Video posted by Bill Buxton in 2010 on GEDIT, marking menus with single strokes in 1991. Menus pop up either on a stationary hold or on a radial stroke ending with a stationary hold. Dwell/press-and-hold: 0:20 stroke, 1:05 hold at end of stroke, 1:27, hold, 1:55 hold and drag. Click/tap gesture known in the art. [Buxton92a] Buxton, W. "The Active Desk", www.billbuxton.com, fetched 2011 The Active Desk: posting by Bill Buxton on Xerox research project on surface computing, display and digitizer (Scriptel) in physical desktop, rear projection display. Used in Bricks graspable user interface, collaborative drawing/whiteboard, prototype for VisionMaker Digital Desk. [BuxtonWAS92b] Buxton, William A.S. "Telepresence: Integrating Shared Task and Person Spaces", Proc. Graphics Interface '92, pp. 123-129 Telepresence for videoconferencing, shared drawing space (whiteboard touchscreen). Cites to liveboard, 3D stereo whiteboard, VideoDraw, Videowhiteboard. Importance of human gestures in human communication. In Videowhiteboard (compare with ClearBoard), other participant shown as "shadow" on wall touchscreen. [BuxtonWB93a] Buxton, William "MATE: A Graphical Text Annotation and Editing System Using Digital Ink", University of Toronto, 1993: available at billbuxton.com Video presentation of pen-based text annotation, editing, correction application using tablet, marking menus, and electronic ink. Companion to "A Marking Based Interface for Collaborative Writing". Compare with van Raamsdonk, Slate PenScheduler? [BuxtonWB93b] Buxton, William "The Chameleon: Spatially-Aware Palmtop Computers", University of Toronto, 1993: available at billbuxton.com Video presentation of palm-sized (touchscreen?) device that senses 3D/6D position in a room, user can move device to see 3D presentations of data. Companion to "Situated information spaces" and "Virtual reality for palmtop computers"". Compare with "StarFire" video by Tognazzini? [Byte92a] Baran, Nicholas "Rough Gems: First Pen Systems Show Promise, Lack Refinement", BYTE Magazine, Vol 7 No 4, 1992, pp. 212-217 Reviews of features of 1992 pen computers GridPad, TraveLite from DFM Systems, and Momenta. Compass menu (marking menu) on Momenta. PenDOS, PenWindows, PenPoint. Contemporaneous with release of Windows 3.1 (page 34) which had screen rotation. [Byte92b] Byte Magazine "The Palmtop 386 Is Getting Closer", BYTE Magazine, Vol 7 No 1, 1992, p. 37 C&T announces PC/Chip architecture with 386SX prossor. Palmtop / hand-held product plans for PenPoint, GeoWorks, CIC, Microsoft pen computing. PenPoint to run on 386SX-compatible processor. [Byte92c] Byte Magazine "The 1992 Byte Awards of Excellence", BYTE Magazine, Vol 7 No 1, 1992, p. 124 PenPoint operating system given award of excellence by Byte Magazine, true object-oriented operating system. See also page 126, NCR 3125 pen computer, wireless stylus. [Byte93a] Byte Magazine "The 1992 BYTE awards", BYTE Magazine, Vol 18 No 1, 1993 Dynapad pen/tablet computer with PenWindows or PenPoint: two pen apps get awards, PenMagic Numero (spread-sheet) for PenPoint, PenSoft Perspective PIM manager for PenPoint. Same article has tutorial article "Digitizer Renaissance". Mentions PenPoint awards in previous year. [CHI92a] Plaisant, Catherine and Wallace, Daniel "Touchscreen Toggle Design", Proc. CHI '92, May 3-7, 1992, pp. 667-668 Video presentation at CHI 92: "Touchscreen Toggle Design" showing virtual toggle and slider switches: See also -- See also Plaisant92a. Compare to Apple slide switch, Fig 2 bottom left? Also teaches equivalents of slider toggle and lever toggle (traditional toggle switch). [CIC92] CIC Communications Intelligence Corporation "CIC Press Release: CIC PenDOS Handwriter Recognition Systems licensed exclusively to IBM", Software Industry Report: October 19, 1992 CIC licenses PenDOS to IBM as exclusive: apparently does not include handwriting recognition technology per se (later sold as Graffiti 2 after Unistroke case) [CJOT93a] Tam, Cynthia "Technology Update: Visual Keyboards", Canadian Journal of Occupational Therapy CJOT, Vol 60 No 2, June 1993, pp. 102-105 Review of visual keyboard / virtual keyboard / on-screen keyboard offerings for Occupational Therapists. ScreenDoors by Madenta Communications, word-predictor dictionary based on user's word-usage history,



automatic punctuation and capitalization. WiViK 2.0 (Windows Visual Keyboard), uses dwell for headpointers to click key, dictionary for word prediction and/or abbreviation expansion: successive expanding quadrant keyboard, similar to marking menus: user points to one quadrant of keyboard, gets expansion of that quadrant divided into four successive quadrants, etc. [CaldwellDW92a] Caldwell, David W and Medendorp, Nicholas W. "Touch-sensitive Control Circuit", United States Patent 5,153,572, October 6, 1992 Multiple-touch-pad touch sensor, if there are multiple touches on more than one pad (detected by impedance change) output is inhibited to reject multi-touch input. [CallowayJD92a] Calloway, Jack D.; Holzer, Raymond F;, Matheny, Stephen E.; and Orwig, Gary W. "Electronic Maintenance Support Work Station", United States Patent 5,146,404, September 8, 1992 Compare to Kaplow; Allow user to pick through exploded views of a parts assembly diagram at multiple levels. An item picked in a text list is automatically shown in a graphical representation. [CampWO91a] Camp, William O. Jr. "Tablet Digitizer with untethered stylus", United States Patent 5,072,076, December 10, 1991 piezoelectric/resistive film touch-sensitive digitizer with passive single stylus: two resistive sheets with piezoelelectric film (PVDF polyvinyidene diflouride) between. Force on deformable/flexible films causes piezoelectric voltage between two resistive sheets, voltage sensors at four corner of the sheet. Cites to resistive film tablet digitizers, electrostatic/capacitive grid with battery-powered stylus. [CarrR91a] Carr, Robert and Shafer, Dan "The Power of Penpoint", Addison Wesley, 1991, ISBN: 0-2015-7763-1 Technical background and information and rationale for design of PenPoint operating system by GO corporation for pen computing, incorporating gestures and handwriting recognition, integrated digitizer tablet and display. Includes sections on applications that do not rely on handwriting recognition. Option sheets, writing pads in UI. User interface of display can changed between landscape and portrait orientation. Page 121 "landscape mode" for screen rotation. [CarrR92a] Carr, Robert; Fraisl, Danile J.; Isaac, Steve; and Ouye, Michael "Computer documents as compound documents in a notebook metaphor", PCT International Patent Application WO 92/08199, 14 May 1992 (Later EP0555403 A1?) PenPoint/GO User interface using notebook metaphor instead of desktop metaphor: notebook, virtual bookshelf of applications, indexing, tabs, pagenumber. Documents may be embedded in other documents. [Casio93a] Casio, Inc. "Casio Z-7000 Zoomer product information", Casio, Inc. Casio Z-7000 Zoomer PDA: GEOS Operating system, Handwriting recognition (Graffiti?). File contains additional information on other pen/GEOS products, comparison to Apple Newton. [CawkellAE93a] Cawkell, A.E. "Encyclopedic Dictionary of Information Technology and Systems", Bawker-Saur, 1993 (hardcopy book) Technical dictionary circa 1993 [Chefalas92] Chefalas, Thomas E.; Fujisake, Tetsunosuke; Kim, Joonki; and Tappert, Charles C. "Robust Prototype Establishment in an On-line Handwriting Recognition System", United States Patent 5,121,441, June 9, 1992 Averaging out the training sets in a trainable recognizer to get a better prototype for elastic matching [ChungCY92a] Chung, Connie Y.; Keen, Robert G.; Raymond, Ozella; Sayder, Jan W.; and Tohlen, Dennis A. "Methods of remapping a keyboard within a computer system", United States Patent 5,086,503, February 4, 1992 Re-map keyboard, show new key assignments on screen. Cites to Kaplow for reconfigurable keyboard. [ClarkMR92a] Clark, Michael R. and Mustafa, Musa "Six Degree of Freedom Graphic Object Controller", United States Patent 5,095,303, March 10, 1992 Mouse 3-D/6-D with finger ring for Z axis, three mutually perpendicular side wheels on mouse for inputting orientation. [ClarkPC93a] Clark, Paul C. and Hoffman, Jance J. "BITS: A Smartcard Protected Operating System", CACM, Vol 37 No 11, November 1994, pp. 66-94 Secure embedded operating system for smartcards: Boot code stored on host, smartcard verifies checksums (signatures?) on load. PKI and certificates on card. Compare with Yee. [CloughWA91] Clough, William A.; Ouelette, Daneil; De La Sablonniere, Serge "Portable Computer with Touch Screen and Computer System Employing Same", PCT International Patent Application @0 91/12578, 22 August 1991 Portable slate pen-computer using touch screen (not proximity digitizer): Microslate, Inc. of Canada. [CohenO93a] Cohen, Oryx; Meyer, Shawna; and Nilsen, Erik "Studying the Movement of High Tech Rodentia: Pointing and Dragging", InterCHI '93 Conference on Human Factors in Computing Systems, Amsterdam, 1993, pp 135-136 Comparison of MousePen, Microtouch UnMouse, Touch Pad, Mouse, trackballs, joystick: UnMouse lower accuracy than touchpad for human use (dragging, pointing): not clear that UnMouse was used in absolute or relative mode; refers to poor placement of barrel buttons on PenMouse stylus [ColbyG91a] Colby, Grace and Scholl, Laura "Transparency and Blur as Selective Cues for Complex Visual



Information", SPIE Vol. 1460 Image Handling and Reproduction System Integration, 1991, pp 114-125 Use depth cues -- blurring and semi-transparency of overlay plane -- to give user cues about importance of information in complex representation. e.g. In multi-layered map, very important information (name of city) is in sharp focus, relevant but less important information blurred or semi-transparent to varying degree (roads, landsat data, etc.) [ComerfordLD93a] Comberford, Liam D., Ellozy, Hamed A., Jelenek, Frederick, Levy, Stephen E., and Nahamoo, David "Method and Apparatus for Improving the Paper Interface to Computing Systems", United States Patent 5,243,149, September 7, 1993 Electronic clipboard tablet computer with audio recording, optical scanner, digitizing tablet [ComputerBusinessReview92a] Computer Business Review "Pen Announcements", Computer Business Review, Feb 25, 1992, Issule No. 1867 Slate announces vendors for Day-timer Pen Scheduler PIM personal information management software: NCR, Poqet, Momenta, MicroSlate. Kurta cordless pen/stylus for digitizer, Kurta Lite-Touch pen, PenPoint, PenWindows, Pen-DOS (CIC). SunDisk Flash memory for pen-computing market. [ComputerChronicles93a] The Computer Chronicles "PDA Review, 1993", The Computer Chronicles, 1993 Review of PDA devices, including Apples Newton: Sharp Expert Pad (Newon), Psion 3A, Amstrad pen computer with touchscreen, HP 100LX, Casio Zoomer, AT&T EO 440 Personal Computer. Brief segment with CIC PenDOS at start of video. [ComputerChronicles93b] The Computer Chronicles "DOS 6.2 (PenDOS), 1993", The Computer Chronicles, 1993 Review of DOS 6.2, starts with demonstration of CIC PenDOS pen extensions to DOS on touchscreen. [Computerwoche92a] Computerwoche "Modem und Funktelefon schon integriert: Personal Communicator von EO und AT&T stiehlt Apples Newton die Schau", Computerwoche April 12, 1992 At announcement at at Comdex, PenPoint / EO (manufactured by Matsushita) from GO corporation attracts attention away from Apple Newton: cellular data phone already included in device. [Computerwoche93b] Computerwoche "Pen-computing-Anbieter setzen nicht alle technischen Trends um", Computerwoche V. 22, 1993: available at http://www.computerwoche.de/heftarchive Comparison of pen-enabled products Windows for Pen-Computing by Microsoft and Pen-DOS from CIC with more radical approach of PenPoint by GO Corporation, with operating system designed from the ground up, but none have met market estimates for success. Mentions that Windows for Pen Computing allows the definition of gestures. CIC Handwriting recognition licensed by IBM as best performing. Also Pen-Right from Grid computing. Microsoft includes support for Visual Basic and C 7.0 MFC. [Computerwoche93c] Computerwoche "Ein kleiner Überblick über die Betriebssysteme für Pen-Computer", Computerwoche 26 Feb 1993: available at http://www.computerwoche.de/heftarchive Listing of pen-computing operating systems and development environments: PenRight from Grid, PenDOS from CIC, penPoint from GO, and Pen-Windows. Mentions one problem of using mouse-type drivers with pen computing is low resolution, less than that of digitizer. Mentions Pen Pal scripting language. [Cornett91] Cornett, Johnny A. and Corbett, James D. "Touchscreen Control Panel with Sliding Touch Control", United States Patent 5,053,758, October 1, 1991 Touchscreen GUI technique, where user slides finger to target, and releases on target: intended to get past inaccuracies/mis-registration of touchscreen film digitizer with display. Compare to Litvin techniques, which are not cited. Contains reference to Minsky gesture article, 1984 [CraneHD93] Crane, Hewitt D. and Rtischev, Dimitry "Pen and voice unite: Adding Pen and Voice Input to Today's User Interfaces Opens the Door for More Natural Communication with Your Computer", BYTE Magazine, Vol 18 No 11, October 1993, pp. 98-102 (Abstract only) compare with Anoto-technology products incorporating voice recording for electronic ink notes? "The strengths of pen and voice technologies complement each other to create a powerful, natural user interface." "Speech to text systems must deal with a wide range of homophones words that sound the same but have different meanings." [Csspne.tk92a] www.csspne.tk "Interacting with computers through pen gestures", www.csspne.tk/newton, March 10 1992 (?) Spreadsheet with gestures on Apple Newton: mentions Numero by PenMagic, and AtHand @Hand by Slate / Dan Bricklin. Refers also to pen-computing pen-based extension to Apple MacIntosh, Milo, supporting editing of mathematical expressions. Slate PenApps application builder IDE, PenMagic Numero spreadsheet and database, Pensoft Perspective personal information manager PIM. [CushmanWH91a] Cushman, William H. and Rosenberg, Daniel J. "Human Factors in Product Design", Elsevier Science Publishing Company Inc., 1991, ISBN 0-444-89031-9 (hardcopy book) Chapters on Keyboards and Input Devices for Computers, Theory and Design Strategies for User Interface Software. Practical Considerations for the Design of User Interface Software. Brief section on auditory displays (speech output interfaces) [CypherA91a] Cypher, Allen "EAGER: Programming Repetitive Tasks by Example", Proc. CHI 91, New Orleans,



April 28-May 2, pp. 33-39 UI detects pattern of repeated action sequences (command input) by user (e.g. moving mail messages to different folders), displays "EAGER" icon in pop-up list, user can select that icon or another. Gestures on buttons? [CypherA91b] Cypher, Allen "EAGER Demo Movie", acypher.com/Eager, fetched 2010 Web page describes video (on file) from CHI 91 of Eager, programming by demonstration/example. UI learns repeated action sequences by user (e.g. moving mail messages to different folders), displays "EAGER" icon in pop-up list, user can select that icon or another. Gestures on buttons? [CypherA91c] Cypher, Allen "Video presentation: Eager: programming repetitive tasks by example", Proc. CHI '91, pp. 445-446 Video demonstration of EAGER, programming by demonstration/example: UI learns repeated action sequences by user (e.g. moving mail messages to different folders), displays "EAGER" icon in pop-up list, user can select that icon or another. Gestures on buttons? [CypherA93a] Cypher, Allen, ed. "Watch What I Do: Programming by Demonstration", M.I.T. Press, 1993 (partial copy) Collection of papers on programming by demonstration/programming by example. Co-editors: Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, Alan Turransky [DamoreK93a] Damore, Kelley "Borland C++ 4 adds visual controls", InfoWorld, November 8, 1993, page 19 IDE with object browser, class expert, floating palettes, events. Pen Computing development environment: includes VBX Visual Basic controls: does not mention pen computing, but that was supported already in Borland C++ 3.1 [DaoJ91] Dao, James; Foyt, David C.; and Allen, Kenneth R. "Keyless Flat Panel Portable Computer - Computer Aided Notebook", United States Patent 5,049,862, September 17, 1991 Notebook style (fold-out) pen computer, similar in appearance to Refalo. Corded stylus. Two flat panel displays in the two sides of the notebook, each a touchscreen (two digitizers), provision for ring binder at the hinge to hold paper. Rotatable change of orientation to acommodate landscape format, or for lefthanded writers. Rotatable notebook computer. [Dauphin92a] Dauphin "The Dauphin 5000 Pen-Based Notebook", Press release, Dauphin Computer, Lombard Illinois Pen-computer slate format using Scriptel electrostatic digitizer. File contains additional materials: The Notebook that Points to the Future [DeBaarDJMJ92a] deBaar, Dennis J.M.J.; Foley, James D; and Mullet, Kevin E. "Coupling Application Design and User Interface Design", Proc. CHI '92, May 3-7, 1992, pp. 259-266 UIMS/UIDE with automatic GUI layout: data model is not separate from user interface model, user interface is generated automatically from (enhanced) data model for different UIs. Examples include Open Look, OSF/Motif, Apple, and CUA. [DeWidRD93a] DeWid, R.D. "Scroll control Box", IBM Technical Disclosure Bulletin TDB v36 n4 04-93, April 1, 1993, pp. 399-404 (partial copy) Rather than tie up user interface display space with scroll bars, have a single scroll widget on screen. [DiffieW92a] Diffie, Whitfield; van Oorschot, Paul C.; Wiener, Michael J. "Authentication and Authenticated Key Exchanges", Designs, Codes and Cryptography, Vol 2, 1992, pp 107..125 Early description of Diffie-Hellman key exchange used in SSL: station-to-station protocol uses certificates [DigitizerTechnology91] Digitizer Technology Company "User's Guide to the Virtual Tablet Editor", Digitizer Technology Company, Redmond WA, 1991 Graphical editor for defining macros, regions, virtual/simulated devices on digitizing tablet: compare with Kaplow? Related to WinTAB specification? [Discover93a] Discover Magazine "1993 Discover Awards: Computer Hardware and Electronics: The Great Communicator", Discover Magazine, October 1, 1993 GO/PenPoint (EO Personal Communicator) innovation award: cellular phone (digital), fax machine, peninput personal computer. Voice annotations on documents (compare Freestyle?). Instant-on: power up on less than five seconds. [DonoghueK93a] Donoghue, Karen "An Intelligent Sketchpad: A Gestural Language for Denoting Temporal Relations in Dynamic Design", Master's Thesis, Media Arts and Sciences, M.I.T., June 1993 Interactive sketching system using gestures on a touch tablet. Examples are taken from football play diagrams, handmarkings: e.g. draw circle around object and and long arrow line to show how the object (e.g. football player) should move. Pen pressure (tip force) used for visual weight/width/shading, stylus tilt for transparency. Marks/gestures for serif and sans-serif fonts for text editing. Double-ended arrow for zoom/scale. Wacom/ Calcomp Drawingboard, Kurta XGT tilt/pressure digitizer tablets. Karen Donoghue was on Freestyle project at Wang with Ken Knowlton. Bill Buxton as reader for thesis. [DrDobbs93] Dr. Dobb's Journal "New Products: Lattice Introduces PenBase Database management Library", Dr.



Dobb's Journal, April 1, 1993 Database library for use with PenRight for pen computers: compare with PenDBase for PenApps [DuwaerAL93a] Duwaer, ARne L. "Data Processing System with a Touch Screen and a Digitizing Tablet, Both Integrated in an Input Device", United States Paten 5,231,381, July 27, 1993 Stylus detector digitizer and finger detector touchscreen on top of each other: refers to ITO for sensors, grid for stylus detector sheet for touch detector, but not specific to any particular digitizer technology, [DysonE91a] Dyson, Esther "GO Corporation: our friend the pen - promoting pen-based input system", Release 1.0: Jan 22, 1991 PenPoint application with intermittent network connections, multiple networks supported, full-peer-level file sharing over network (via TOPS). Longer discussion of PenApps. Penpoint MIL machine interface layer, wireless data modem support, deferred network communications. PenWares: comparison of GO Corporation developer's conference to introduce PenPoint operating system (Bill Campbell), and Microsoft similar conference for Pen Windows the following February 21. (Note: The style, pacing, and features of the Microsoft PenWindows presentation closely paralleled the GO/PenPoint presentation.) [DysonE91b] Dyson, Esther "Component software", Release 1.0: 25 February 1991 Page 22: discussion of component software and component reuse, as in GO/Penpoint and Component WorkShop, ParcPlace systems [DysonE91c] Dyson, Esther "Act III -- Notepads: A New Environment, or a Pen off the old block?", Release 1.0: 25 February 1991 Page 24: PenPoint easier to program for more advanced features than PenWindows. Active Book: PDA based on Smalltalk -- compare with Self and with Newton PDA / NewtonScript? Also mentions PenPal software from PenSoft, and Eden Group VPi386 notepad with PenWindows and touch-screen-enabled AutoCad. [DysonE91d] Dyson, Esther "PenBook: Just the Pages Please", Release 1.0: 30 September 1991 Page 14: PenBook, simple (no editing) e-Book / e-publishing application for PenPoint OS, similar to Kindle. Limited text mark-up (white-out, highlighting). [DysonE91e] Dyson, Esther "Smart Recognition: The Brain Behind the Screen", Release 1.0: 31 October 1991 Page 1: PenSieve context recognition by SuperScript: handwriting recognition context word list compared to pick list user interface -- for Pen Windows. Page 4: Paragraph's Parascript/Calligraph: ParaGraph handwriting recogition Page 5: Strategic Position -- Faint Heart Never Won Fair Market (Momenta pen computer), Grid PenRight publicly open APIP [DysonE91f] Dyson, Esther "Power through Scripting", Release 1.0: 31 May 1991, pp. 1-3 Scripting of applications for users to use existing applications/components for additional applications: HP NewWave, Apple Events (compared to Objects), Open Scripting. Mentions PenPoint, no details. Dates direct manipulation to 1980s: word-processor direct move of chunks of text, calendar by typing data into right place. [DysonE92a] Dyson, Esther "The Electronic Frontier: Where Personal Computing Goes from Here", Proc. IFIP 12th World Computer Congress, Madrid, September 1992, pp. 13-22 Pens and GroupWare: Critieus GO PenPoint and Microsoft Windows for Pen Computing (Pen Windows) as too expensive ($2000 in 1992), Cites Pen/GEOS (Graphical Environment Operating Systems) and Palm Computing. Quotes Jeff Hawkings regarging Slate for being cross-platform. Sidekick 2 from Borland. [DysonE92b] Michalski, Jerry "Unified Messaging, Part 1", Release 1.0: 31 December 1992, pp. 1-3 Unified messaging (pre-internet): data, voice, fax, video, e-mail, real time or store-and-forward. Discusses difference in nature of interaction in shifting of time, and change of media (phone/video, email/voice). Unified messaging means easier to use appropriate medium and form (e.g. send the fax you need to, rather than voice mail because do not know fax number). Discusion of PenPoint/GO "GOnna do it all" GO Message Center. Notable Technologies doing PenPoint messaging apps. Apple Newton messaging. PenPoint as replacement/alternative for desktop, Newton as adjunct to desktop. [DysonE92c] Dyson, Esther "Apple's Newton: Handywidget in Sight", Release 1.0: 15 June 1992 comparison of Apple Newton with speculative features of HandyWidget. Features wireless communications and handwriting recognition for Newton, promise of large third-party applications market, system perpetually upgradeable: comparison with PenPoint. Separate from desktop market. Also reports on internal development history at Apple. [DysonE92d] Dyson, Esther "Context in Context", Release 1.0: 30 November 1992 Niche markets (long end of the tail) too small to be taken over by biggest companies ("Microsoft"): large apps market. Breakdown chart of cellular data markets, GO/PenPoint is only specific example of cellular data device. [DysonE93a] Michalski, Jerry "Directory Services", Release 1.0: 23 April 1993 Directory services, basically unified address book for communications to avoid rampant duplication in diverse systems, Newtons/Zoomers interface to existing directories. PenPoint and Super PIM personal



information manager. Overview of Internet and X.400 addresseing. [EO93a] EO "The World of Messaging: An Introduction to Personal Communications", EO, Inc., 1993 User documentation provided for EO personal communicator, running PenPoint operating system. Describes E-mail, FAX, AT\&T Mail services, Applelink. See also ATTHobbit file (functional device) [EO93b] EO "Lookup Guide to the EO Personal Communicator", EO, Inc., 1993 User documentation provided for EO personal communicator, running PenPoint operating system. Describes basic function of PenPoint and EO "Hobbit" hardware: caret, check/tick, circle, cross out, undo, flick up gestures. Press (hold) gesture for move, tap+press two stroke gesture for copy. Notebook app can be rotated on screen, independently of "desktop". See also ATTHobbit file (functional device), Agulnick file. Marquee animated highlighting for object drag selections. [EO93c] EO "EO Personal Communicator: Applications", EO, Inc., 1993 Brochure on third-party applications for PenPoint/EO: InkWare Notetaker, Longhand by Lexicus, MHS link for GO mail by GO: PenASCII, penHost, Slate PenApps Application Builder, Pensoft Perspective, PenMagic Numero, PenCross [EO93d] EO "Getting Started with your EO Personal Communicator", EO, Inc., 1993 User documentation provided for EO personal communicator, running PenPoint operating system. Marquee animated highlighting for object drag selections. [EO93e] EO/ATT "How to Appease a fat cat FROM a gondola", AT\&T / EO, Inc., 1993 Sales brochure for ATT/EO Hobbit personal communicator pen computer tablet. Shows functional unit (actually display) running PenPoint operating system. Applications include Slate Corporation PenApps (similar to Visual Basic?), Pensoft Corporation Perspective forms/reports/planner, PenMagic Software Numero spreadsheet and LetterExpress letters/fax editing, Ink Development Corporation NoteTaker, fax software. Deskin Research Group Gnosis database and document storage software with remote server storage. [EO93f] EO/ATT "EO Personal Communicator: Video", AT\&T / EO, Inc., 1993 Video marketing presentation for EO personal communicator running PenPoint: shows wireless data, notebook metaphor, and cellphone communications: tablet-sized smartphone with handwriting pencomputing user interface [EO93g] EO/ATT "AT\&T EO Personal Communicator: Third Party Product and Services Catalog", AT\&T / EO, Inc., 1993 PenPoint applications: include aha! InkWriter, Gnosis Mobile Information Manager, InkWare NoteTaker 1.2, Longhand cursive handwriting recognition, wireless/network communications applications, PenApps (Slate application builder), external memory flash [EdwardsMJ93a] Edwards, Martin J. "Touch sensor array systems and display systems incorporating such", United States Patent 5,194,862, March 16, 1993 Array/matrix of touch sensors (touch switches) for multi-touch input in a graphics tablet: sensing elements integrated with LCD. Addressing technique for dealing with large number of touch sensors: touch sensors are bi-stable (maintain state after touched), so that touched locations in the array can be determined. Mentions capacitive touch sensor grids relying on physical displacement of conductors under force. Active matrix (independent) addressing fabricated using thin-film-transistors (TFT). Cited in Ure Patent as technology for multi-touch. [ElbraT92a] Elbra, Tony "Computer Security Handbook", NCC Blackwell Limited, 1992 (hardcopy book) UK handbook on computer security: physical security, software auditing, simple explanation of networks, rudiments of physical storage of back-ups, etc. [EllozyHA91a] Ellozy, Hamed A; Jeanty, Jenry H; and Tappert, Charles C. "Handwriting recognition employing pairwise discriminant features", United States Patent 5,005,205, April 2 1991 Pairwise analysis of characters against prototypes, then voting, then highest score of the winning pair: appears similar to the Pencept technology (not patented) [ElrodS92a] Elrod, Scott; Richard, Bruce; Gold, Rich; Goldberg, David; Halasz, Frank; Jannsen, William; Lee, David, McCall, Kim; Pedersen, Elin; Pier, Ken; Tang, John; and Welch, Brent "LiveBoard: A Large Interactive Display Supporting Group Meetings, Presentations, and Remove Collaboration", Proc. CHI '92, SIGCHI conf. on Human Factors in Computing Systems, 1992, pp. 599-607 Multi-user remote whiteboard using rear-projection screen and wireless digitizer pen with four additional buttons (for menus, etc.) Cordless pen digitizer projects light onto screen, detected optically. Sweeping gestures for scrolling and panning, also for turning pages in a slide-show application: tap as gesture? Multitouch: up to three users with three pens at same time, therefore can share the tools. Other references state as multi-user / multi-touch: up to three simultaneous pens/users. [Endo91] Endo, Yasuo; Akimichi, Shinji; Hirose, Tadayoshi; Furukawa, Motonori; Yoshida, Kyoko "Context-Base Input/Output System", United States Patent 5,012,521, April 30, 1991 Gesture recognition user interface: refers to recognition of patterns (sketch recognition?) of elements of



on-line drawing (curves, arcs, lines), but emphasis is on gestures: X for erase, clash for cut (of two segments), closed wrap (circle or lasso) to cause recognition of handwriting, wrap/lasso plus arrow for magnify, copy, reduce. No modal input, gestures can be input at any time, compare with Tablet PC gestures. Integrated tablet digitizer and display. [FeinerS91a] Feiner, Steven "Hybrid User Interfaces: Breeding Virtually Bigger Interfaces for Physically Smaller Computers", Proc. UIST '91, Nov 11-13, 1991, pp 9..17 Combine two head-mounted displays (low resolution and high resolution) and two pointing inputs (low resolution and high resolution: use Polhemus 3-D electromagnetic digitizer to track head). Cites to Sutherland 1965 for head-mounted display. Mentions combination of dataglove and stylus for pointing. [FischerAM91a] Fischer, Addison M. "Public key-signature cryptosystem with enhanced digital signature certification", United States Patent 5,005,200, April 2, 1991 CA generates certificates include authority/authorization/permission/rights attributes, along with public key: examples include dollar signing limit for business, requirement for specific co-signature(s). Single certificate may be signed by multiple CAs. Also mentions signing of other objects and combinations of objects. Storing signature separate from document: allows paper documents to be verified by re-entering data from paper document. [FitzmauriceGW93a] Fitzmaurice, George W. "Situated Information Spaces and Spatially Aware Palmtop Computers", CACM Vol 36 No 7, July 1993, pp. 39-49 Portable/palmtop computers (with touchscreen) instrumented to know nearby objects in physical environment: calendar, office setting, phone, etc. Does not talk about display rotation or orientation landscape/portrait [FitzmauriceGW93b] Fitzmaurice, George W.; Zhai, Shumin; and Chignell, Mark H. "Virtual Reality for Palmtop Computers", ACM Trans. on Information Systems, Vol 11, No 3, July 1993, pp. 197-218 Portable/palmtop computers (with touchscreen) with 3D/6D position sensing, used to show a "virtual environment" as moved around. Alternatively, pie menus for navigation. Chameleon prototype uses video camera feed from camera in front of a regular display to a small hand-held display, with Ascension Technology 3D/6D digitizer. Virtual environment showing floating documents in 3D space, old documents to rear. [FitzpatrickGP93a] Fitzpatrick, G.P.; Haynes, T.R.; and Sterrett, W.R. "Translucent Window Attribute", IBM Technical Disclosure Bulletin, Vol 36 No 06A, June 1993 Translucent windows in windowing system, so that when a window is over another you can still see what is underneath. Translucency /semi-transparent done by muting the color of the underlying window (add grey teint) and/or by blurring slightly. [FlowerDR91a] Flower, Dale R. Flower; Gallarelli, pat; Goffinet, Kevin P.; Miles, Anthony W., Rudisill, Charles A.; and Vorhees, Kevin H. "Touch sensing display screen apparatus", United States Patent 5,038,142, August 6, 1991 Touchscreen or tablet measuring force at four corners of a monitor with strain gauges, monitor is mounted so that it can only move in the "Z" direction, and that not by much. [Forcier93] Forcier, Mitchell D. "Script/Binary-encodedcharacter Processing Method and System with Moving Space Insertion Mode", United States Patent 5,220,649, June 15, 1993 Editing of handwriting/electronic ink without recognition of characters, use of editing gestures distinguished from handwritten character/glyphs based on context: word-wrap, inserting spaces, editing of handwritten lines of text: Compare with van Raamsdonk. Press reports from 1999 of Forcier accusing Microsoft of infringement: see correspondence with Forcier [Forcier93a] Forcier, Mitchell D. "Script/Binary-encoded Character Processing Method and System", United States Patent 5,231,698, July 27, 1993 Editing GUI of handwriting/electronic ink without recognition of characters, use of editing gestures distinguished from handwritten character/glyphs based on context. Editing of electronic ink: word-wrap, inserting spaces, editing of handwritten lines of text Distinguishes handwritten electronic ink from other non-handwriting input during editing Compare with van Raamsdonk. Press reports from 1999 of Forcier accusing Microsoft of infringement [FordDM91a] Ford, David Malcom "On-Line Recognition of Connected Handwriting", Ph.D Thesis, University of Nottingham, U.K., May 1991 Contains review of handwriting recognition techniques for discrete, connected, and cursive writing: primitive sub-strokes, elastic matching, feature matching, Freeman chain codes. Digitizer used was a Pencept PenPad, specifically engineered for use in a handwriting-input system. [Fox92] Fox, Abijah Shawhan; Greanias, Evon Constantine; Kim, Joonki; and Tappert, Charles Carson "A system for automatic adjustment and editing of handwritten text images", European Patent Publication Number 0176715B1, Proprietor International Business Machines, Armonk, NY. Stroke segmentation and alignment to a base writing line [FrancikE91a] Franci, Ellen; Rudman, susan Ehrlich; Cooper, Donna; Levine, Stephen "Putting Innovation to Work:



Adoption Strategies for Multimedia Communication systems", CACM, Dec. 1991, Vol 34 No 12, pp. 53-63; also Baecker95a Towards the Year 2000 Report on adoption plan for Wang Freestyle as pen-based system for cooperative/collaborative work, synchronized voice/electronic-ink annotations. Refers to conversational turns enabled in whiteboarding system. Cites to early resistance to telephone as an electric toy (citations) [FrederickV93a] Frederick, Vivian and Yasuda, Phyllis, with Yamada, Judy and Koda, Ann "Using Windows 3.0/3.1", McGraw-Hill, 1993 (hardcopy books) Color and pattern Scheme GUI in Windows 3.0/3.1 [FrickeJ93a] Fricke, Jeorg "Tactile graphic computer screen and input tablet for blind persons using an electrorheological fluid", United States Patent 5,222,895, June 29, 1993 Tactile display for the blind, uses a fluid that changes state in response to a voltage. Intended to display braille. [FriedlandM91] Friedland, Mary "They Made It Happen (Momenta)", Pentop Magazine, Nov. 1 1991 Kamran Elahian, founder of Momenta pen-computing company: Portia Isaacson compares MADE (Momenta Application Development Environment) with PenPoint and Pen-Windows. [FritznerC91a] Fritzner, Claus; Nilsen, Leif; and Skomedal, Asmund "Protecting Security Information in Distributed Systems", Proc. IEEE Computer Society Symposium on Research in Security and Privacy, 1991, May 20..22, 1991, pp. 245..254 Overview of public key security techniques in computer systems: X.509 certificate, Privilege Attribute Certificate for access control. [FrohlichD93a] Frohlich, David "The History and Future of Direct Manipulation", Hewlett-Packard internal publication, 1991 Broad psychological comparison of direct manipulation GUIs (a la Shneiderman) with virtual partners, virtual reality systems. Cites to Shneiderman for three-point definition of "direct manipulation" [Fujitsu91a] Fujitsu "Fujitsu tablet PC History: Poqet Pad", www.shopfujitsu.com, 2010 PoqetPad, PoqetPad Plus: Early pen computer / touchscreen PDA using NEC V20/V25 (8086 clone) processor. MS-DOS with handwriting recognition, electronic ink capture, signature verification, mouse emulation, pop-up virtual keyboard. NestorWriter recognition software and Nestor PenShell API, PenPal toolkit forms software, HyperPAD scripting language (based on HyperTalk). Half-sized display (640x200), resistive-film touchscreen. Landscape and portrait orientation. File contains additional materials [GemmellM92a] Gemmell, Mark; Robson, Christopher "Input device with tactile feedback", PCT Patent Application PWO9200559A1, January 9, 1992 Stylus and digitizer with haptic feedback, conveying texture by vibrational/noise [GlassnerAS93a] Glassner, Andrew S. ed. "Graphics Gems I", AP Professional Press, 1993 First volume in "Graphic Gems", collection of papers: Useful 2D Geometry (alpha blending to de-jaggy lines); Circles of Integral Radius on Integer Lattices; Convenient Anti-Aliasing Filters that minimize "Bumpy" sampling (alpha blending); Frame Buffers and Color Maps (alpha transparency factor); Useful 3D Geometry; 3D Rendering; Ray Tracing; Numerical and Programming Techniques; Matrix Techniques; Modeling and Transformations; Curves and Surfaces [GlyphicTechnology93a] Lentczner, Mark "Glyphic Script / Codeworks for EO/PenPoint", Glyphic Technology, fetchd from www.ozonehouse.com/mark/codeworks.html 2015 Codeworks/Glyphic Script: prototype-based scripting language for EO (and Windows), with IDE, drag-anddrop-building tool. [Go92a] GO Corporation "GO Corporation Announces PenPoint Custom Solutions Alliance", Press Release, GO Corporation, april 16, 1992 PenPoint application by PenMagic Software, Pensoft Corp., Slate Corp., Ingram/Micro, Merisel. Application can be cross-developed on Windows PC. Slate PenBook e-book; PenMagic Numero spreadsheet / electronic "work paper"; Pensoft Perspectvie PIM personal information manager. Also PenPoint MIL Machine Interface Layer hardware abstraction layer HAL. [GoldbergD91] Goldberg, D. and Goodman, A. "Stylus User Interfaces for Manipulating Text", Proceedings of the Fourth ACM SIGGRAPH Symposium on User Interface Technology (UIST'91), 127 - 135; also Baecker95a Towards the Year 2000< Cited by Bill Buxton: Shows "undo" graphically, using electronic ink: Scriptel electrostatic digitizer. Shows markup editing as a good application for handwriting recognition and input: cites Suenaga and Nagura. Note: did not cite van Raamsdonk. Cites to PenPoint, but specifically mentioned before PenPoint: penpoint recognizer can make errors in gesture recognition. Cites parallax problem with Scriptel digitizer above display. Discusses U-V recognition, boxed versus boxless, but no reference to PenCept. Segmenting cursor: initially one writing box, box is moved when you start writing outside the current box. Tap gesture to replace (incorrect) single-character recognition with recognizers second alternative. ("tap-correction"). Tap-twice to correct manually.. Mentions two differences between stylus and mouse pointing: precision,



and multiple-buttons on mouse, does not mention lift-vs-down, relative-vs-absolute position, etc. [GoldbergD93a] Goldberg, D. and Richardson, C. "Touch-typing with a stylus", INTERCHI '93 Conference Proceedings, Amsterdam, April 1993, pp 80-87 See also SIGGRAPH Video Review 88, New York: ACM., 1993. Unistroke handwriting character recognition [Goodisman91] Goodisman, Aaron "A Stylus-Based User Interface for Text: Entry and Editing", S.M. Thesis, Massachusetts Institute of Technology, Dept. of EE and CS, May 1991 General study and overview of stylus-based user interface for text input and text editing, including the use of gestures. Makes comparisons to PenPoint, but is not specifically about PenPoint. Mentions differences between Stylus and Mouse GUI user interfaces, such as activate-button-on-touch versus activate-buttonon-lift, scroll-up flick versus pull-scroll-bar-down. Supervisor was David Goldberg of Xerox PARC, who later patented similar work on mark-up editing using gestures, boxes, with handwriting recognition (Unistroke). Describes state of the art in stylus-based user interactions including text entry and text editing; includes section on buttons (pages 67-71); Describes that complex buttons exist in mouse-based systems that allow multiple actions (tap and tap and hold); describes advantages of highlighting buttons when stylus positioned above it to provide feedback to user; Describes using buttons with active region larger than the button image to increase ease of selecting buttons. "hold" refers to PenPoint press-and-hold [GookinD92a] Gookin, Dan "Oh, to Find a Better Mouse", Also Available at www.wambooli.com/fun/dang/mouse electronically published manuscript for PC Computing Usability Labs on mouse, trackball, and similar devices: includes low-cost tablets and touchpads such as AceCat, UnMouse, PortaPoint. Also MousePen (note on writing with a mousepen). Unmouse: default (only?) behavior is non-absolute relative motion, mentions problem of double-click with a touch pad. [GoreA93a] Gore, Andrew and Ratcliffe, Mitch "Newton's Law: A Digital Nomad's Guide", Random House Press, 1993, ISBN 0-679-74647-1 (hardcopy book) Handwriting recognition on page 157 ff. Mentions JOT electronic ink standard with Slate, but not supported on Newton. Handwriting recognition "not well explained" by Apple. Appears to show a scrub-out gesture for erasing text: text is selected (alternatively) before the scrubbing gesture, or by the scrub gesture itself. Gestures: caret gesture. Due to problems with recognizer, recommends deferred recognition and searchable electronic ink. [GoslingJ92a] Gosling, James; Forrest, Craig; Frazier, Al; Frank, Ed; Haughton, Patrick; Palrang, Joe; Payne, Jon; Sheridan, Mike; and Warth, Chris "The Star7 PDA Prototype", James Gosling / Green Project demonstration video: available on YouTube 1992 video demonstration of "Java" PDA with touchscreen and inertial scrolling, draggable windows for showing TV schedules. Animated user interface reminiscent of Microsoft "Bob". This was the hardware device for which Java/Oak language was devised. Active elements in user interface are highlighted in color to make them discoverable. Scrollbars with inertia. TV Guide schedule provides more detail information when zoomed in -- compare with "data lens". Gesture language: downstroke or spiral/circular gesture to start drag, etc. Gesture combined with following drag motion. Objects can be dragged onto other objects, which then performs appropriate type-dependent operation. Distributed objects transmitted over network link: e.g. drag object to whiteboard, object is transmitted and shows on other device, both users can manipulate on their whiteboard. [GoughML93a] Gough, M.L. et al "Method and Apparatus for Interfacing With a Computer System", United States Patent Application Ser No 08/060,458, filed May 10, 1993 translucent/transparent windows [Greanias91] Greanias, Evon C. and Donaldson, Robert L. "Remotely Sensed Personal Stylus", United States Patent 5,007,085, April 9, 1991 Transparent digitizer overlaid on a display (CRT example, not flat LCD: see other Greanias). Appears to be a Patent on using the same stylus on any of several workstations, and logging on with the stylus by identifying the particular stylus. Compare with SecureID token from RSA. [Greanias92a] Greanias, Evon C.; Stein, Frank L; Donaldson, Robert L.; and Gray, Michael "Remotely Sensed Personal Stylus", United States Patent 5,117,071, May 226 1992 Capacitive grid Finger touch touchpad, and electromagnetic digitizer in one device. Describes touch mechanism (tip switch?) as solving a problem of not knowing when stylus is actually in contact with surface [Greanias92b] Greanias, Evon C.; Stein, Frank L; Donaldson, Robert L.; and Gray, Michael "Stylus Sensing System", United States Patent 5,149,919, September 22, 1992 Electrostatic (capacitive?) digitizer using a single globe pickup receiver in the stylus, to avoid tilt errors. Special contact sensing mode: appears not to use a physical switch in the stylus, uses height / proximity distance instead. Alternating grid of grounded and signal wires. [Greanias92c] Greanias, Evon C.; Verrier, Guy F.; Arbeitman, Gordon W.; Tannenbaum, Alan R; and An, Yu L "Advanced User Interface", United States Patent 5,157,384, October 20, 1992



Gesture/voice/speech input with multiple user profiles for different commands to different programs. Circle gesture mentioned. Translation of gestures to simulated input for other, pre-exiting devices, such as keyboard and/or mouse, to maintain compatibility with pre-existing applications. Compare to Pencept CAD and other interfaces from 1985, which translated "recognition macro" gestures to defined sequences of keyboard/mouse input. If no meaning yet associated with gesture, pop up a menu for user to define the meaning on the fly. [GreaniasEC92d] Greanias, Evon C.; Stein, Frank L.; Donaldson, Robert; and Gray, Michael "Stylus Sensing System", United States Patent 5,149,919, September 22, 1992 Stylus for transparent grid conductor tablet digitizer. Grid is transmitter, round tip on stylus to have no tilt error (capacitive sensing?). Grid can be either electromagnetic or electrostatic signal source. States disadvantage of 332 Patent was false touch points because "touch" detected indirectly by signal strength, by energizing entire grid with same signal, thus getting less variation in signal strength across surface. [GridPad93a] AST Research, Inc. "AST Computer GridPad 2390 User's Manual", AST Research, Inc. Irvine, CA 1993 User's manual for GridPad pen computer: gestures include tap, double-tap, pig-tail/erase, press-hold ("hold the pen down, pause, drag to select) gesture. Sketch program with electronic ink notetaking and text entry from on-screen/virtual keyboard in separate layers. Multiple keyboard layouts (e.g. alphabetic, symbols, math, plus writing pad for handwriting recognition) [GridSystems92a] GridSystems "Grid System Windows for Pen Computing 1.0", Grid Systems installation kit (hardcopy manual) Windows for Pen Computing installation kit: includes Slate applications software DayTimer Pen Scheduler, WinFax Pen Software, Slate PenBook Reader, Slate LooseLeaf Notetaker, Microsoft documentation for Windows for Pen Computing 1.0 and Windows 3.1 User's Guide. Also PenCell spreadsheet program from Grid (compare with Slate spreadsheet program). Windows for Pen Computing brochure appears to have Pen SDK information? Approach 2.0 for Windows (Dbase with Forms) with pen input, Borland C++ 3.1 Application Frameworks, Microsoft C/C++ and Windows Software Development Kit, Visual Basic 2.0 showing pen input, Pen InputMaster control from StylusTech for Visual Basic, Oracle Card (like Hypercard?) for pen-input [GridSystems92b] YouTube.com "Grid Pen Computer running Windows 3.1", YouTube.com posted approximately year 2000 Video Windows for Pen Computing running on Grid Pen Convertible computer, circa 1992. [GridSystems92c] GRiD Systems, 1992 "Microsoft Windows for Pen Computing 1.0 User's Guide / Guide to Pen Computing", GRiD Systems, 1992 (hardcopy book) Microsoft Guide for Pen Windows: writing with/without letter guides (boxed/boxless handwriting recognition). PenPalette is application-independent general handwriting guide with tick marks for stroke segmentation, button for on-screen virtual keyboard. Gestures include checkmark, backspace (diagonal with retrace), insert space/tab, curlicue to cut, undo, insert new line, copy (alpha) gesture, paste (caret). delete word, insert tab, extend selection. Circled-letter gestures user definable. Circled-C for Edit Copy keystroke, etc. User training of recognizer. Brief description of rotating display/windows from control panel. Notetaking / electronic ink editing, sketch/drawing recognition in Notebook application with pages. hardcopy file also includes Microsoft Windows 3.1 User's Guide, GridPen Essentials from Slate (Day-Timer Pen Scheduler, PenBook, LoosLeaf notetaker). PenCell pen computing spreadsheet from PenWare. Pen Comptuting Quick Reference (PenWindows) Microsoft Windows for Pen Computing third-party products for mobile professionals: Audapoint estimator for auto insurance adjusters, ReadRight handwriting/handprint recognition from Adobe, Concept Technologies Stroke of the Hand cursive word processing/forms processing (compare van Raamsdonk), Ascend PIM Personal Information Manager for Pens, Lexicus Longhand cursive recognition. Pen InputMaster OCX control for Visual Basic. PenMetrics FieldNotes Geographical mapping/annotation system, SalesForce. includes installation diskettes for Windows for Pen Computing [Griepentrog93a] Griepentrog, Scott "David vs. Goliath", Wiki posting at: twiki.stg.net Description of technical support interacting on Microsoft Notebook Application for Pen Windows version 1.00a: described Notebook electronic ink (?) as most promising application for Pen Computing -- compare with Slate Pen Scheduler note-taking application? [GriswoldGN93a] Griswold, Gary N. "A Method for Protecting Copyright on Networks", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 DRM management / access control by software wrappers (secure applications) which communicate with central servier for document access. Only intellectual property rights owners have (legal) right to place documents on networks.



[Gurewich93] Gurewich, Ori and Gurewich, Nathan "Teach Yourself Visual Basic 3.0 in 21 Days", Sams Publishing, 1993 (hardcopy book) Tutorial and workshop on Visual Basic language and development environment [Hagenuk92a] Hagenuk GmbH "Hagenuk: Bedienungsanleitung MT 900 Das Mobiltelefon", Hagenuk GmbH, Kiel Germany 1992 Early smartphone/cellphone with SIM card (64-bit), slide-to-unlock [HajimeA93a] Hajime, Asano "Computer System Provided with Function for Displaying Plural Windows (machine translation)", Patent Abstracts of Japan 05-173743 July 13, 1993, Appl. No. 03-337482 Managing/hit-detection in overlapping windows, upper Z-order windows are transparent where overlap a lower window. [HakenL92a] Haken, L; Abdullah, R.; and Smart, M. "The Continuum: A continuous music keyboard", Proceedings of Int'l Computer Music Conference, San Francisco, International Computer Music Association, 1992 See also technical report of same title, CERL Sound Group, EE&CS, Univ. of Illinois Multi-touch musical keyboard, tracks up to 10 fingers in X, Y, and Z (pressure/force). Describes several multi-touch (X/Y/Z) technologies. Hall-Effect sensor and magnets suspended on compressible medium for Z, conductive/resistive rubber strips for Y, separate strips (keys) for X. Musical multi-touch playing surface: Photoelastic playing surface light by polarized light from underside, greater pressure causes change in polarization of photoelastic material, creating visible rings around touch points. CDD camera(s) under surface detect location and number of rings for X, Y, and Z. Musical multi-touch playing surface: Conductive rubber sheet with high horizontal (X/Y) impedance/resistance, lower vertical (Z) resistance). High horizontal impedance allows touch points to be electrically isolated, array/matrix of detecting pads underneath rubber detect/measure voltage and current from top sheet over rubber. [HardockG93a] Hardock, Gary; Kurtenbach, Gordon; and Buxton, William "A Marking Based Interface for Collaborative Writing", Proceedings of the UIST'93, pp 259-266. See also http://www.dgp.utoronto.ca/OTP/papers/bill.buxton/mate.html MATE Markup Annotator Text / Editor: Collaborative system for marking up text documents: Says modeled on Wang Freestyle: references from 1989. press-and hold "holding down the button on the pen" (although that is the barrel button?), slow motion to drag, quick to annotate. flick gesture, lasso for select/move. Refers to gestures as marks. Annotation (electronic ink) and command (gesture) modes. [HarnL92a] Harn, Lein; Lin, Hung-Yu; and Yang, Shoubao "A software authentication system for the prevention of computer viruses", Proc. CSC '92 1992 ACM annual conference on Computer Science, pp. 447..450 Review of problems of different forms of virus detection: pattern-matching, software integrity checking (checksum), software authentication via PKI. Proposed soluting splits private key of certifying authority (compare with CertCo?) [HartrickTV93a] Hartrick, Thomas Vincent; Stevens, Jeffrey Noel; and Sabia, Nicholas James "Data processing system and method to enforce payment of royalties when copying softcopy books", European Patent Application 93105502.4, Publication number 0 567 800 A1, April 2, 1993 DRM digital rights management for copy protection: must use secure application to access, which uses encoded license information in document file to require royalty payment before copying/printing. [HashimotoO92a] Hashimoto, Osamu and Myers, Brad A. "Graphical Styles for Building User Interfaces by Demonstration", Proc. UIST '92, Nov. 15-18, 1992, p. 117-124 in Garnet system papers: User can define multiple graphical styles for UI. When a style is edited, all widgets change. Shows Set Style GUI for setting which style. dynamic themes: graphical styles can be applied to widgets at run/design time, see "Set Style" dialog window in Fig. 5 [HawkinsJC91a] Hawkins, Jeffrey C. and Wiegman, Timothy L. "Display Output Rotation", International Patent Application Publication WO 91/00586, January 10, 1991 Grid Touchscreen computer: rotate orientation of display between landscape and portrait, and re-layout the windows and user interface [HawkinsJC92a] Hawkins, Jeffrey C. "Object Recognition System", United States Patent 5,125,039, June 23, 1992 Compare with Unistroke patent. Recognition of handwriting by dictionary look-up independent of any character set. Uses recognition dictionary: one entry per object, entries involve Boolean operations only to find match: uses inflection points / chain-codes [HawkinsJC92b] Hawkins, Jeffrey C., Daly, John J., McNamara, James H. and Cole, James F. "Hand Held Computer", United States Patent 5,133,076, June 23, 1992 Hand-held slate computer with handwriting, convertible to a desk-top computer: shows rotation of display: screen orientations called East, West, etc. for various landscape and portrait modes [HayesJP93a] Hayes, John P. "Introduction to Digital Logic Design", Addison-Wesley, 1993 (hardcopy book) Textbook on digitial circuit/logic design. [HellerD93a] Heller, Dan "XView Programming Manual / XWindows: Volume Seve, Updated for XView Version 3.2 by Thomas Van Raalte", O'Reilly and Associates, August 1993



X-Windows / XView programming and specification [HellerD93b] Heller, Dan "XView Reference Manual / XWindows: Volume Seve, Updated for XView Version 3.2 by Thomas Van Raalte", O'Reilly and Associates, August 1993 X-Windows / XView programming and specification [HenryTR91a] Henry, Tyson R, Hudson, Scott E., Yeatts, Andrey K., Myers, Brad A. and Feiner, Steven "A Nose Gesture Interface Device: Extending Virtual Realities", Proc. UIST '91, Nov 11-13, 1991, pp 65-69 Tongue-in-check ("DATANOSE"). Used Polhemus 3-D electromagnetic digitizer mounted in a helmet. Cites to nose-twitching as prior reference for nose gestures. [HideakiK91a] Hideaki, Kawamura "Image processing system for forming a slantwise-mapped or rotated modified image of an original image", United States Patent 4,985,849, January 15, 1991 Simplified shift-register algorithm for rotating or slanting a window or image [HilsDD91a] Hils, Daniel D. "DataVis: a visual programming language for scientific visualization", Proc. 1991 ACM Comp. Sci. Conf., San Antonio TX, March 1991, pp. 439-448 Graphical/visual programming language, based on data flow modules. Compare with LabVIEW? [HilsDD92a] Hils, Daniel D. "Visual Languages and Computing Survey: Data Flow Visual Programming Languages", Jnl. Visual Language and Computing, Vol 3, 1992, pp. 69-101 Review of data-flow visual programming languages. Most visual languages based on data flow, perhaps with addition of control constructs (While, Case statements, etc.). Includes LabVIEW. Hookup (music synthesizers), Fabrik, InterCONS, HIVISUAL, VIVA, Cantata, VIPEX, LabVIEW (laboratory instrumentation), ConMan, viz, Visual ToolSet, PROGRAPH, Show and Tell, ESTL, DataVis. [HopkinsD91a] Hopkins, Don "The Design and Implementation of Pie Menus: There're Fast, Easy, and SelfRevealing (sic)", Dr. Dobb's Journal, Dec. 1991 Description of pop-up circular pie menus, developed in 1986 at U. Maryland in uwm window manager for X-Windows, OpenWindows. [HullJJ92a] Hull, Jonathan J. and Chin, An-Tzu "Semantic Information Extraction with a Thesaurus for Visual Word Recognition", Advances in Structural and Syntactic Pattern Recognition, World Scientific, Singapore, 1992, pp. 342-351 Given a (typed) character recognition result for a word, look up related/similar words in a thesaurus, test the neighbors for better score on recognition. Noted as a method to reduce the computational load of syntactic/semantic context. [Hullender92] Hullender, Gregory N. "Method for Recognizing Handwritten Characters Using Shape and Context Analysis", United States Patent 5,151,950, September 29, 1992 Deterministic Finite Automata (state machine) for using a dictionary of words in handwriting character recognition on the fly, using probabilities of characters: compare with Viterbi algorithm, which is not cited. [IBM92a] IBM "Double-Click Generation Method for Pen Operations", IBM Technical Disclosure Bulletin, Vol 35 No 6, November 1992, p. 3 (partial copy) Double-tap gesture on a pen computer/touchscreen: combination of time, distance, and number of mouse movement events. [IBM93a] IBM "BellSouth/IBM Simon smart phone", (various) Collection of published articles concerning 1993 IBM Simon, a touchscreen-only smart phone with calendar, address book, and other PDA functions built in. Recieved Best Of Show award at 1994 comdex. Design by IBM, manufacturer: Mitsubishi, marketed by Bell South. Touch-screen phone vs. QWERTY cellphone. Preinternet, send/receive Email by dialing into analog modem on Lotus Notes, send/receive analog Fax. DispatchIt back-end software application for field service. Predictive keying on virtual keyboard. File mentions BellSouth working with Apple to integrate cellular phone service into Newton device. Annotation function (electronic ink) for Fax messages, without handwriting recognition. [IBM93b] IBM "PenPoint Operating System Overview and Application Development IBM Version", IBM International Technical Support Center, Boca Raton, Feb 1993, GG24-3978-0 PenPoint documentation from IBM: mentions rotation for portrait and landscape display orientations. Technical description of MIL machine interface layer: comparison to (lesser) BIOS support. Shows multiple live application (multi-tasking/multi-processing). Technical overview of application/document embedding (compare with Windows OLE) with screenshots. Color graphics support. Transparent windows: partial transparency? Deferred data transfer (inbox/outbox). Printer/remote-file-server configuration dialog. [IBM93c] IBM "Image Orientation Sensing and Correction for Notepads", IBM Technical Disclosure Bulletin n347 03-93; Balasubramanian, PS; PDAs or tablet computers usually used in landscape mode: include a sensor to determine orientation, and rotate correct for angle of display. [IBM93d] IBM "IBM Simon", IBM Simon, ca. 1993 (physical device) Functional (?) IBM Simon smartphone/PDA with touchscreen, ca. 1993 [IEEEComputer92a] Christine Miller, editor "New Products: Developments in Multimedia -- Emerging Technology:



Pen in Hand", IEEE Computer, April 1992, pp 89-90 IEEE product listings for pen-computing and touch-computing: Microtouch TruePoint monitor with touchscreen; Pen/GEOS pen operating system with Palm handwriting recognition from GEOWORKS; Coupde-Plume pen handwriting extensions to DOS from Shannon Associates; Slate At-Hand spreadsheet; Slate DayTimer Pen Scheduler; Travelling Software Pen LapLink and PenConnect; CalComp DisplayPad integrated LCD and electromagnetic tablet (tilt, pressure, proximity); Stylos electronic ink document mark-up and annotation [InfoWorld91a] InfoWorld "Windows for Pens beta adds macros, training", InfoWorld, July 22, 1991, page 7 Windows for Pens beta / Pen Windows publicly announced gesture macros definable by user. Refers to new recognizer in December 1991. Additional controls for Visual Basic, compared to PenPoint: delayed recognition a.k.a. "raw ink", ink as a data type, context-sensitive editing (gestures depend on context?) Gestures include tap and double-tap gestures on buttons? [InfoWorld91b] InfoWorld "Microsoft Passion for Go May Have Been Just a Ploy to Steal Its Heart", InfoWorld, Feb. 18, 1991 Windows for Pens / Pen Windows: Microsoft technical staff met regularly with GO/PenPoint, while also working on PenWindows. Compare with Barbarians book by Eller? [InfoWorld91c] InfoWorld "Momenta is a study in compromise", InfoWorld, October 21, 1991 Momenta pen-based computer compared with Grid, NCR, PenPoint, and PenWindows. Pop-up context menus / command compass, , trainable handwriting recognition. Operating System is DOS. [InfoWorld91d] InfoWorld "The Power of the Pen", InfoWorld, March 4, 1991 Article on Tug of War between GO/Penpoint and Microsoft/Pen Windows. Describes PenPoint technology as superior, Microsoft's advantage is business standardization. Review of hardware vendors, architectural comparison of PenPoint and Pen Windows, whether IBM supports PenPoint or PenWindows. Quotes "Information at your Fingertips" from Microsoft. [InfoWorld91e] InfoWorld "Is pen computing for real?", InfoWorld, November 11, 1991 PenPoint, Windows for Pen Computing (Pen Windows), and PenTop: questions whether users are waiting for technology to deliver as promised. Possible killer apps: Ink Development Inkware (later aha! journal notetaker?), PenBook from Slate, Pensee notepad and outlining, SuperScript improved recognition (?). Companion article "Momenta takes different approach to pen systems",. Quotes predictions of 1 million units to be sold by 1995, mostly felett users, 4 million by 1995. [InfoWorld92a] InfoWorld "TouchPen turns PenPoint into touch-based system: MicroTouch System's Inc's digitizer compatible with Go Corp.'s PenPoint", InfoWorld, June 8, 1992 MicroTouch working with PenPoint from GO -- OS required proximity sense. Mentions cursor control with finger [InfoWorld93a] InfoWorld "Visual C++ is a strong development tool: Microsoft gets the C environment right with this robust Windows application builder", InfoWorld, June 21, 1993, page 94 ff. Visual C++ 1.0 replaces Microsoft C/C++ 7.0: compared with Visual Basic: custom controls of Visual Basic may be used in Visual C++. Development environment: Visual C++ 1.0, available with PenWindows: Shows a tool palette, resource editor, tool palette can add and drag-and-drop VBX components, custom controls [InfoWorld93b] InfoWorld "Visual C++ 1.5 integrates OLE, ODBC", InfoWorld, November 8, 1993, page 5 Visual C++ 1.5: SDK with AppWizards, ClassWizards, visual editing, and drag-and-drop. Also mentions Borland C++4.0 as released earlier, with templates. Development environment: Visual C++ 1.5, available with PenWindows (other references). [InfoWorld93c] InfoWorld "CA challenges Visual Basic with app builder for Windows, OS/2", InfoWorld, November 8, 1993, page 19 CA Realizer 2.0 for OS/2, and Windows 3.1. Development environment: "let's builders visually build applications and then generates maintainable Basic-like source code". FormDev tool for designing visual element using standard graphical objects. [InfoWorld93e] InfoWorld "Genetically engineered mice run amok at Windows World", InfoWorld, May 24, 1993, page 34 Report on Windows World conference: Whiskers utility use use middle button (button 3) of mouse, Gliffic Pluse mouse pad that also works with pen/stylus digitizer, used with Pen Windows / Windows for Pen Computing. [IsaacsonP93a] Isaacson, Portia "Electronic ink emerges as a launchpad of the future -- While few products exist, Jot 1.0 standard is a step", Computer Resellers News, June 28, 1993 Review of JOT 1.0 electronic-ink / E-ink format standard for handwriting data. Mentions particularly the tutorial nature of the specification, and open-ended format. Credits Slate as author of the standard: original draft was by Jean Renard Ward, later revised, credit to Dan Bricklin of Slate. [IshiiH91a] Ishii, Hiroshi; and Artia, Kazuho "ClearBoard: Translucent Multiuser Interface for TeamWorkStation",



Proc. 2nd European Conf. on Computer-Supported Cooperative Work, September 25-27, 1991, Amsterdam pp. 163-174 Desktop whiteboard system with video of other user's face/image in translucent/semi-transparent window, so that video does not completely obscure collaborative workspace. However, users are reluctant to write/draw over the video image, alternative is movable video window. Translucent fused overlay and selective overlay. [IshiiH91b] Ishii, Hiroshi; and Miyake, Naomi "Toward an Open Share Workspace: Computer and Video Fusion Approach of TeamWorkStation", CACM, December 1991 Vol 34 No 12, pp. 37-50 Desktop whiteboard system with video of other user's face/image in translucent/semi-transparent window, so that video does not completely obscure collaborative workspace. However, users are reluctant to write/draw over the video image, alternative is movable video window. Translucent fused overlay and selective overlay [IshiiH92a] Ishii, Hiroshi; and Kobayashi, Minoru "ClearBoard: A Seamless Medium for Shared Drawing and Conversation with Eye Contact", Proc. CHI '92, May 3-7, 1992, pp. 525-ff. Transparent whiteboard combined with video conferencing, other side of conference is shown (mirror image) on whiteboard so that the writing appears right-way-around, participants can have eye contact (Drafter-Mirror architecture). Reference to hand-gestures (non-verbal communication) in video conferencing. TeamBoard hardware. [IshiiH93a] Ishii, Hiroshi; Kobayashi, Minoru; and Grudin, Jonathan "Integration of Interpersonal Space and Shared workspace: ClearBoard Design and Experiments", ACM Trans. on Information Systems, Vol 11, No 4, October 1993, pp. 349-375 Transparent whiteboard combined with video conferencing, other side of conference is shown (mirror image) on whiteboard so that the writing appears right-way-around. Reference to hand-gestures in video conferencing. TeamBoard hardware. [ItohM91a] Itoh, Masaharu "Image processing apparatus and method", United States Patent 5,050,225, September 17, 1991 Algorithm for rotating or slanting an image/window via shift-register method [IversonEAR93a] Iverson, E. A. R. "Method to Allow Users to Select Transparent Color for Windows", Research Disclosure n347 03-93, IBM, March 1, 1993 Graphical User Interface GUI user can select transparent color (translucent?) for background of a window, to see what is underneath. [IwaiI93a] Iwai, Isamu and Miura, Yoshiyuki "User Interface Development tools for Pen Computer Applications", Proc. HCI '93, Fith Intl. Conf. on Human-Computer Interaction, 1993, pp 1016-1021 (Abstract only) Widget toolkit/IDE/UIMS for pen computing applications. [JacksonKM92a] Jackson, K.M. and Hruska, J (eds.) "Computer Security Reference Book", CRC Press, 1992 (hardcopy book) Reference work on computer security ca. 1992, collection of chapters. Biometric hardware, PKI, copy protection security tokens, communications security. Several chapters on legal aspects. [JacobRJK93a] Jacob, Robert J.K.; Leggett, John J.; Myers, Brad A; and Pausch, Randy "Interaction styles and input/output devices", Behaviour and Information technology, Vol 12 No 2, pp. 69-79, 1993 gesture and handwriting as alternative to keyboard input. Gesture and other types of input have a probability vector of input, rather than absolute commands. Refers also to two-hand input (Buxton multitouch) and UI having more than one stream of input events, unlike WIMP architecture, force feedback for haptic display of 3D surfaces. [JaffeDA93a] Schloss, W. Andrew and Jaffe, David A. "Intelligent Musical Instruments: future of musical performance or the demise of the performer?", INTERFACE Journal for New Music Research, The Netherlands, December 1993 Essay / personal report on experiences with musical composition "Wild Life" using program electronic instruments. Ensemble playing with human violinist. Cites to Radio Drum, and earlier 1989 ppaer on "Gesture Sensing via Capactivie Moments". Notes that Radio Drum (programmable instrument) sensed position in 3D digitizer, real time. [JanisFL93a] Janis, Frederick L. "Method and System for Variable Authority Level User Access Control in a Distributed Data Processing System Having Multiple Resource Manager", United States Patetn 5,263,158, November 16, 1993 Access control / D.R.M. system using centralized management control service, checks requested intent (type) of access [JohnsonW91a] Johnson, Walter A.L.; Faieta, Baldo A.; and Smith, Z. Erol III "Form Utilizing Encoded Indications for Form Field Processing", United States Patent 5,060,980, October 29, 1991 Printing machine-readable descriptions, data, and instructions on a paper form, to be used in conjunction with OCR to read the pattern on the special section of the form. Numerous citations to other inventions



involving printing data on paper: compare to DataGlyphs, which encoded the whole surface, and to Anoto. [JohnsonW93a] Johnson, Walter; Jellinek, Herbert; Klotz, Leigh Jr.; Rao, Ramana; and Card, Stuart "Bridging the Paper and Electronic Worlds: The Paper User Interface", Proc. InterCHI '93, April 1993, pp. 507-512 Paper-like user interfaces should be called stylus-based, Paper often easier to use: physical user memory for where information is located, visual context with multiple pages, ubiquitousness, etc. Credits Bloomberg and Hecht for encoding data in documents "glyphs" (no citation). Compare with Anoto. [JonesLR92a] Jones, Lawrence R "The current state of human-computer interface technologies for use in dairy herd management", Jnl of Dairy Science, 1991, Number 75: pp. 3246-3256 Use of touch screens (invented in 1960's) in dairy application: includes example of pre-HTML hypertext/hypermedia with touchscreen [Jourjine91] Jourjine, Alexander "Neural network using random binary code", United States Patent 4,996,648 Neural net for handwriting recognition developed during Freestyle project at Wang Laboratories. [Jourjine93] Jourjine, Alexander "Method and apparatus for parallel implementation of neural networks", United States Patent 5,224,066 Alex Jourjine had a very interesting user-independent cursive handwriting recognizer at Wang Labs, without use of a vocabulary dictionary. [KabbashP93a] Kabbash, Paul, MacKenzie, I.Scott, and Buxton, William "Human Performance Using Computer Input Devices in the Preferred and Non-Preferred Hands", Proceedings of the ACM INTERCHI 1993, pp 474-481 multi-hand user interface: prototype used stylus (Wacom pressure) tablet and mouse: multi-touch Cited in Wanderley00 [KableRG92a] Kable, Robert G. "Electrostatic Digitizers Performance with Color LCD Displays", Scriptel Corporation, 1992 Scriptel electrostatic digitizer, resistive film. Mentions high immunity to backlight and display noise. Frontmounted touchscreen. Describes ratiometric sensing of voltage field [KahinB93a] Kahin, Brian "Introduction: The Strategic Environment For Protecting Multimedia", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Overview of business problems of DRM / intellectual property protection. Software copy protection abandoned because of legal exemption for "back-up copies". [KahnRE93a] Kahn, Robert E. "Deposit, Registration And Recordation in an Electronic Copyright Management System", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Test platform (test bed) for DRM management for copyright material at Library of Congress: Rights Management System / rights granted on-line, Registration and Recording System, Digital Library System DLS. Used PKI public key cryptography, digital signatures, notarization, encrypted / private-enhanced email to create certificates on-the-fly. [Kay93] Kay, Alen "The Early History of Smalltalk", ACM SIGPLAN, Vol 28 No 3, march 1993. Appearing in "History of Programming Languages", ACM Press 1996 History of object-based programming, back to Burroughs B200 Mentions GRAIL and Sketchpad systems, 1960's, early pen computing environment Describes early thought to include handwriting recognition input on Dynabook, not thought would work well [KelleyAJ93a] Kelley, A.J. "On the Development of a Force-Feedback Mouse and its Integration into a Graphical User Interface", Dynamic System sand Control, Vol 55 No 1, 1994, pp 287-294 Force-feedback "mouse" using position over electromagnetic plate. Haptic features for user interface include tactile feedback for: rectangular constraints for windows, resistance to motion when moving upward into menu bars, constraining input in scroll bars, icon gravitation, viscous feeling when dragging (window, object, scrollbar), etc. Interfaced to X-Windows. [KempfJ93a] Kempf, James "Integrating Handwriting Recognition into Unix", 1993 USENIX Summer Technical Conference, Cincinnati Ohio, June 21-25 1993 Block recognizer versus cursive recognizer: HRE API for Linux, for incorporating handwriting character recognition into X-Windows [Kent93] Kent, Joel C. "Contact touchscreen with an improved insulated spacer arrangement", United States Patent 5,220,136, June 15, 1993 Resistive sheet digitizer with spacing dots, electrographic tablet [KhanideF92a] Khanide, Fereydun "kurz vorgestellt - Pen-Computer: Pen-Prototyping / PenEntwicklungswerkzeuge: PenRight!Pro und PenPal", c't Magazine, September 1992, page 48 (Abstract only) (in German) Visual Software development packages for pen computers using GRiD API. Pen Computing development environments: Graphical environments ("maskengestützten Applikationen") u: GRiD Pen-Pal,



PenRight! (both developed by GRiD?) [KimuraTD92a] Kimura, T.D. "Hyperflow: a visual programming language for pen computers", Proc. 1992 IEEE Workshop on Visual Languages, 15-18 September 1992, pp. 125-132 visual programming language with pen input: pen-based multimedia computer system. Lasso/line-through gesture for multiple copying, box and arrow gesture recognition. [KimuraTD93a] Kimura, Takayuki Dan "Hyperflow: A Uniform Visual Language for Different Levels of Programming", preprint - Proc. 1993 ACM Computer Science Conference visual programming language with pen input: pen-based multimedia computer system. Refers to gestic command / gesture command on a window. Examples appear to be modeled on flow of circuit/signal processing. [KishimotoK92a] Kishimoto, Kazutomo; Okumura, Masahiro; masimo, Tamon; Sakamoto, Tatsuya; and Kouno, Youji "Method and apparatus for rotatable display", United States Patent 5,134,390, July 28, 1992 Rotatable display: text annotations and windows for labelling function keys (near to screen) are automatically rotated and repositioned to be at appropriate position and orientation landscape/portrait (e.g. at bottom of display screen) [KobayashiM92a] Kobayashi, Minoru and Ishii, Hiroshi "DispLayers: Multi-Layer Display Technique to Enhance Selective Looking of Overlaid Images", Proc. CHI 92, SIGCHI Conf. on Human Factors in Computing Systems p. 66 Extension to ClearFace/TeamWorkStation remote whiteboard collaboration system: show other user's face image semi-transparently over other whiteboard image, at different focal distance, using half-silvered mirrors and two display. [KohlJT91a] Kohl, John T.; Neuman, B. Clifford; and Ts'o, Theore Y. "The Evolution of the Kerberos Authentication Service", (revised) Proc. Spring 1991 EurOpen Conference, Tromso, Norway History and technical description of Kerberos authentication system through version 5, at M.I.T. Project Athena. Single sign-on: user login gives ticket used by all application. Uses secret-key (single key) cryptography, version 5 addressing interoperability with public-key cryptography: cites to RSA digital signatures. See also later 1994 publication. [KollinJS93a] Kollin, Joel S. "A Retinal Display for Virtual-Environment Applications", Proc. SID 93 Intl. Symposium Digest, Vol 24, p. 827 ff. Retinal display (projects image directly onto human retina) mounted on glasses. Compare with Google Glasses. [KruegerLA93a] Krueger, Leroy A.; Rozmiarek, Edward S.; and Tench, James G. Jr. "Method and system for accessing visually obscured data in a data processing system", United States Patent 5,265,202, November 23, 1993 A window showing video made (semi) transparent by making every other scan line transparent, so that user can see application data in another window underneath [KruegerMW91a] Krueger, Myron W. "Artificial Reality II", Addison-Wesley Publishing, Reading MA, 1991 (hardcopy book) Limited discussion of "The Glove as a Gesture Technology": simple hand gestures: because its shape is in three dimensions, it has a larger vocabulary of gestures. Suggests that natural hand gestures (e.g. open/close) better than symbolic gestures. Cites earlier VIDEOPLACE ca. 1972 with simulated "touching" of two users to each other via video projection. [KruegerMW93a] Krueger, Myron W. "Environmental Technology: Making the Real World virtual", CACM, Vol 36 No 7, July 1993, pp. 36-37 Multi-hand / multi-user (multi-touch) videodesk teleconferencing, both a whiteboard and direct manipulation of graphics. Multipoint control (four finger touch points) for manipulating curves, 3-D wind tunnels, videotouch. [KurlanderDJ93a] Kurlander, David Joshua "Graphical Editing by Example", Ph.D. Disseration, Columbia University, July 1993 Constraint-based graphical editing: graphical search-and-replace (with and /wo constraints), inferring constraints from multiple view, graphical command history editiable, graphical macros by example. Cites to Ribine gestures as form of macros. Macros are scriptable e.g. by combining with graphical search for iteration. Graphical searching works by degree of matching by shapes (compare with trainable recognizer), macros can change color attributes. Searching by dynamic information as well as by static image. [Kurtenbach91] Kurtenbach, G. and Buxton, W. "GEdit: A test bed for editing by contiguous gestures", ACM SIGCHI Bulletin, Vol 23, 1991, pp 23-26 Cited in Lipscomb91, Rubin91b: sub-gestures within a stroke Simple gestures to create a small object (square, circle, etc.) using a single-direction flick stroke in one direction: gestures for delete (slash) and select (loop around) with drag mark on the rest of the selection to move [Kurtenbach91a] Kurtenbach, G. and Buxton, W. "Issues in Combining Marking and Direct Manipulation



Techniques", Proc. 4th ACM SIGGRAPH Symposium on User Interface Technology (UIST), pp 137-144 Simple gestures to create a small object (square, circle, etc.) using a single-direction flick stroke in one direction: gestures for delete (slash) and select (loop around) with drag mark on the rest of the selection to move. Describes GEdit, a graphics editor that allows user to create objects and to manipulate objects using multiple gestures (click and drag, strikethrough); Describes click and hold gesture that generates a pie menu of options; mentions PenPoint system as supporting both marking interactions and direct manipulation. press-and-hold: pressure maintained on stylus and pointer it kept still for approximately 1 second, followed by drag. [Kurtenbach91b] Kurtenbach, G. "Making Marks Self-revealing", SIGCHI Bulletin, October 1991, pp 60-ff. Flick gestures in different directions, pop-up menu for pie-menu if you press-and-hold. [Kurtenbach91c] Kurtenbach, Gordon "Editing Time-line Data Using Marking Menus", Video demonstration, Dynamics Graphics Lab, University of Toronto, 1991 Also known on YouTube.com as Marking Menus in ConEd Example. Marking menus /gestures in UI to edit video annotations: pop-up menu when mouse/stylus is held stationary (hold/press gesture). Dwell/pressand-hold: press at 0:13, press-and-drag at 0:25, mark(stroke) t 1:30 and 2:28, at 3:25 shows Apple Mac menu, which had click. [Kurtenbach92] Kurtenbach, Gordon and Baudel, Thomas "HyperMark: issuing commands by drawing marks in HyperCard", SIGCHI Conf. on Human Factors in Computing Systems, Monterey, CA, 1992. pp. 64-64 (poster session) Gesture Markings on buttons in HyperCard. Feedback? Describes a system called HyperMark for Apple's HyperCard system wherein buttons react not only to a mouse press but also to "marks" that trigger other actions; recognizes that this results in fewer buttons (saves screen real estate). Multiple gestures on a button (gesture buttons). [Kurtenbach93] Kurtenbach, Gordon and Buxton, William "The Limits Of Expert Performance Using Hierarchic Marking Menus", Proc. INTERCHI '93, 24-29 april 1993, pp. 482-487 Hierarchic marking menus: heirarchical levels of pie/radial menus, user can combine "zig-zag" motions through multiple levels of menus. Includes an undo/no-selection gesture by moving pen back to center before lifting stylus, or moving pen to new location before lifting. Undo gesture is pointing to center. Notes that mouse is worse than stylus with digitizer/touchscreen. Test system employed Momenta pen-based computer (which already had marking/pie menus), but Momenta not listed in references. Dwell/press and hold the pen still, or drawing a mark (a stroke) [KurtenbachGP93b] Kurtenbach, Gordon P., Sellen, Abigail J; and Buxton, William S.a. "An empirical evaluation of some articulatory and cognitive aspects of "marking menus"", Human Computer Interaction, Vol 8 No 1, pp. 1-23 User usability study of heirarchic marking menus: response time not solely a function of number of menu items: press-and-hold for approximately 1 second pops up pie menu. [KurtenbachGP93c] Kurtenbach, Gordon Paul "The Design and Evaluation of Marking Menus", Ph.D. thesis, Graduate Dept. of Comp. Sci., University of Toronto, 1993 Kurtenbach thesis on marking menus: self-revelation (due to pop-up menu on dwelling), guidance (from pie menu), and rehearsal (from stylus motion in pie menu like the mark). Pen item is selected when stylus crosses outer boundary of pie menu radial item. Previewing of menu selections. Animation of demonstrated marks as on-line help. UI works by pen-motion only, barrel buttons are "events distinct from men movement". Dwell/press-and-hold: dwelling is mid-stroke, press-and-wait is at start, but one example says "press and hold" [KwokJC92a] Kwok, Jac Chak-Lam "An innovative integrated development environment for the Pocket PC PDA", Master's Thesis, Massachusetts Institute of Technology, EE/CS, 1992 Mentions digitizer problems with handwriting recognition, based on Ward and Kluklinski articles [KyoceraRefalo91] Kyocera "Kyocera Refalo KX-1601 Product information, 1991", Collection from Mobile Computers Museum, http://www.kako.com/backup/museum/kyocera/kyocera.html, 2002 and PocketComputing / Caz Pocket Computers Collection 2006 Kycocera Refalo: fully electronic day-timer, DOS-based Pen palmtop, Handwriting recognition, ATOK, JUSTWindow. Unit was shaped like a three-ring binder, with the display and writing surface on the left inside: included separate pointing device (cursor-key pad) in addition to stylus. MS-DOS, additional leaves/pages could be inserted with additional hardware, worked by electromagnetic coupling. Web page Copyright 1995-1999 by Eiji Kako, additional Refalo materials in file. [LaLonde91] LaLonde, Wilf R. and Pugh, John R. "Inside Smalltalk, Volume II", Prentice-Hall International, 1991 Introduction to Smalltalk language and environment. (Electronic file.) [LahtinenPI91a] Lahtinen, Pekka Ilmari "Multi-function telephone apparatus", European Patent Application E 0458563 A2, November 27, 1991 Portable telephone user interface that recognizes telephone numbers in messages on the display and lets the user dial them. Uses patterns for finding telephone numbers in a display



[LandayJA93a] Landay, Jams A. and Byers, Brad A. "Extending an Existing User Interface Toolkit to Support Gesture Recognition", Proc. CHI '93, Interact '93, pp. 91-92 in Garnet system papers: cites to PenPoint for gestures. Adding gesture support to widget input: proposes using Rubine adaptive/trainable recognition. Recognizer passes some graphical features to interactor: e.g. bounding rectangle (but not hotpoint?) No support for context of gesture (object it is written on). Garnet user interface system extended to support gesture input (Rectangle, Circle, Copy, Delete gestures: delete is one-stroke "X"). Interactor class passes name/ID of gesture, and features (such as bounding box). Compare with PenCept recognition-macro gestures for extents? [LawlessMT93a] Lawless, Michael T. and LaVine, Nils D. "Reconfigurable Simulator Specifications for Future Main Battle Tanks Within the Close Combat Test Bed", U.S Army Research Institute for the Behavioral and Social Sciences, Fort Knox Field Unit, Training Systems Research division, Report AD-A268 134, March 1993 User interface for a battle tank simulator: mentions need for reconfigurable glass cockpit, touch screen technology on display has advantages from user interface controls placed on the screen. Touchscreen is a modified Microtouch unMouse using capacitive touch-sensing with a microswitch below the screen. Dragging by clicking and sliding. [LevineS91a] Levine, Stephen R.; Harui, Alex J.; Schirpke, Michael W.; Donoghue, Karen; Ajgaonkar, Donna "Apparatus for Manipulating Documents in a Data Processing System utilizing Reduced Images of Sheets of Information which are Movable", United States Patent 5,060,135, October 22, 1991 Freestyle from Wang: Pen/stylus direct manipulation for annotating document images. Only made use of electronic ink, no handwriting recognition or gesture shape recognition. Touch-and-lift (similar to tap) and touch-and-move (drag) operations. Allowed for both ink annotations and voice recording annotations, which could be played back as a "recorded" whiteboard session. No actual remote whiteboard communication. Gave the appearance of being an operating environment, but was only for annotating captured images, taken from DOS application screens or Fax input. Separate Patent on means for suspending and restoring a running DOS application program. Describes stylus-based document management system wherein thumbnails (called "stamps") of documents are responsive to "touch and lift" action (tap) for selecting the item and "touch and move" action to move the item to a different position on the display. [LevineS91b] Levine, Stephen R. and Ehrlich, S. "The Freestyle System: A design perspective", in Human-Machine Interactive Systems, A. Klinger, Ed. Plenum Press, New York, 1991, pp. 3..21; also Baecker95a Towards the Year 2000 Freestyle from Wang: in particular describes voice/electronic-ink annotations with recorded real-time pointing with proximity cursor of Hitachi digitizer tablet [LevineSR93a] Levine, Stephen R; Abrams, Kenneth H; Burns, Raymond S; Harui, Alex J; Lakness, David R.; and Rudis, Ronald F. "Apparatus for Document Annotation and Manipulation Using image from a Window Source", United States Patent 5,231,579, July 27, 1993 Freestyle patent: annotation (electronic ink and voice/audio) of documents in Freestyle. States that documents are produced as snapshots of windows from an application, and the documents are annotated separately from the application. Typo in first inventor name: Levin, LevinSR Stephen R. [LewisJR92a] Lewis, James R. "An Alternative Digraph Base Typing Key Layout For Single-Finger Or Stylus Input", IBM Corporation, Technical Report 54.710, October 27, 1992 Alternative keyboard layout, keys positioned so that most likely 2nd character (character pairs) are adjacent or near [LewisJR92b] Lewis, James R. "Literature Review of Touch-Screen Research from 1980 to 1992", IBM Corporation, Technical Report 54.694, August 20, 1992 Review of touchscreen literature at about the time of PenWindows, PenPoint, PenDOS, and Pen/OS/2. Mentions last contact / lift-off as less error-prone than first contact or touch area. Touch-feedback strategies include highlighting the object the user has selected. Has specific description of hold (of pressand-hold) to select then object changes sate (highlight). [LewisJR92c] Lewis, James R.; Kennedy, Peter J.; LLomia, Mary J. "Improved Typing Key Layouts For Single Finger Or Stylus Input", IBM Corporation, Technical Report 54.692, September 18, 1992 Analysis of QWERTY, Dvorak, alphabetic virtual keyboard layouts versus digraph arrangement of standard keyboard, and digraph on 5x5 (plus extras) arrangement. Targets stylus/finger input on touchscreen. [LewisJR92d] Lewis, James R. "Typing Key Layouts For Single-Finger Or Stylus Input: Initial User Preference And Performance", IBM Corporation, Technical Report 54.729, December 17, 1992 Users prefer and are faster with standard QWERTY virtual keyboard layout at first, so should be default instead of higher-speed keyboard layouts [Lexicus92] Lexicus "iSKETCH Digital Ink for Wireless Communications", http://www.motorola.com/lexicus/html/isketch.html Compression software for digital ink images on hand-held platforms



[LinnRJ93a] Linn, R.J. (Jerry) "Copyright And Information Services In The Context Of The National Research And Education Network", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Notes on DRM management / copyright protection mandated by 1991 High Performance Computing Act, unenforceable requirements. Technical requirements for Authentication (data integrity, source of material), fees collection, limited re-distribution. Protected rendering. Proposes legislative changes. [LipscombJS91a] Lipscomb, James S. "A Trainable Gesture Recognizer", Pattern Recognition, Vol 24 No 9, September 1991, pp 895 to 907 Angle filtering and multi-scale/step gesture recognition. Noise and wild tablet/digitizer points filtered out at end of processing, not pre-processing. Angle filtering very non-invariant recognition, so funny scaling was required. Copy/move using gesture recognition without separate vector part of move: just two circles and arc between as one stroke. Gestures have semantic effect (they do something), are single, so a recognition speed delay less critical than with handwriting character recognition. [Loh91] Loh, Skiu-Chang "On-line Handwritten Character Recognition Apparatus with Non-Ambiguity Algorithm", United States Patent 5,034,989, July 23, 1991 Handwritten characters are recognized as a set of primitives / primitive strokes, consisting of short lines, segments, curves. Contains table of single-stroke primitives (unistrokes?) which are labeled uniquely with letters of the alphabetic: recognition of letters/words consists of a dictionary look-up of the resulting strings of characters [LoomisJM93a] Loomis, Jack M.; Golledge, Rignald G.; and Klatzky, Roberta L. "Personal Guidance System for the Visually Impaired using GPS, GIS, and VR Technologies", Proc of 1993 Virtual Reality VR Conference, csun.edu GPS aid for visual impaired/blind, giving information about surrounding environment as well as location. Output to user by means of artificial auditory beacons located at correct/appropriate binaural location relative to user. [LoprestiDP93a] Lopresti, Daniel P. and Tomkins, Andrew "Pictographic Naming", Proc. CHI '93 / INTERACT '93 Conf. Companion on Human Factors in Computing Systems, 1993, pp. 77-78 Approximate matching of scribbles/marks, to use handwritten electronic ink as names of files. Searching by approximate matching of scribbles. Handwriting recognition HWX delayed. Compare with Delayed Recognition in Slate DayTimer Pen Scheduler? [LoprestiDP93b] Lopresti, Daniel P. and Tomkins, Andrew "Approximate Matching of Hand-Drawn Pictograms", 3rd International Workshop on Frontiers in Handwriting Recognition, Buffalo NY May 1993 Avoid need for handwriting recognition when indexing documents by approximate matching of hand-drawn pictograms. Mentions difficulty of handwriting recognition of (Asian) languages. compare with ink indexing in Slate DayTimer/PenScheduler? [LoricchioDF91a] Loricchio, David F. and Lewis, James R. "User Assessment Of Standard And Reduced Size Numeric Keypads", Proc. Human Factors Society 35th Annual Meeting, 1991, pp. 251-252 Small physical keyboards: users always prefer full-size keys, or at least wider keys, more than narrower key spacing bothers. [MCmicrocomputer93a] MC microcomputer "Accorclo Olivetti/EO: Radicale rivolulione nella comuaicazione personale", MC microcomputer, April 1993, Number 128 (in Italian) EO Personal computer (PenPoint) to be marketed by Olivetti [MacKenzieIS92a] MacKenzie, I. Scott "A Comparison of Input Devices in Elemental Pointing and Dragging Tasks", Proc. GI Graphics Interface '92 Performance/usability study: tapping (tap gesture) time vs. Fitt's Law, user timing performance [MacNeillD93a] MacNeill, David "Newton Notes: Messaging Card and NewtonMail: We pick up and deliver", On The Go Magazine, October 13, 1993 NewtonMail E-Mail application using electronic ink instead of text [MacTech93a] Bresee, Don and Ticktin, Neil "Newton Toolkit: You've seen the Newt and what it can do", MacTech, the Journal of Apple Technology: Vol 9 No 11, November 1993 Description of Newton Toolkit, NewtonScript. Compares NewtonScript to TCL (THINK Class Library), with inheritance in heirarchies, prototype inheritance, routing; packages compared to Think C. [MacTech93b] Ticktin, Neil "It's a Newt!", MacTech, the Journal of Apple Technology: Vol 9 No 11, November 1993 Review of Newton handwriting PDA. Not much SDK information, but mentions HyperCard in same article as Newton [Machart92] Machart, Beverly Hellen, and Wang, John Shihyuan "Graphic data display system", European Patent EP 0 226 019 B1, February 5, 1992, assigned to International Business Machines Corporation. Filing date in 1986 A signature which has disconnected stroke can be manipulated an transmitted as a single object. To indicate end of signature/character, uses second button on pointing device: appears



similar to Pencept PenPad. [MackR91a] Mark, Robert and Montaniz, Frank "Comparison of Touch and Mouse Interaction Techniques for a Graphical Windowing Software Environment", Proc. of Human Factors Society 35th Annual Meeting, 1991, pp. 290-294 Selection gesture (tap/lift-off) on touchscreen: Application provides visual indication of selection by highlighting before lift off: also mentions auditory feedback (click?). Mentions difficulty of double-tapping (with stylus) for double-clicking (with mouse), but does not mention cause (see PenWindows and Tablet PC notes). Very short "taps" are rejected as unintentional input. Highlighting feedback on touchscreen. [MackinlayJD91a] Mackinlay, Jock D.; Robertson, George G.; and Card, Stuart K. "The Perspective Wall: Detail and Context Smoothly Integrated", Proc. CHI '91, pp. 173-179, 1991 -- Also Xerox PARC Technical Report UIR-R1991-03 Fisheye/3-D three dimensional view of information showing a main window of information, context/related information on "walls" of main window shown in perspective. Cites to Rooms (sets of windows), Information Visualizer. Compare with General Magic? [MacqB93a] Macq, Benoit and Quisquater, Jean-Jacques "Digital Images Multiresolution Encryption", Proc. CNI Conference, 1993 Encryption of video streams: encryption messes up compression in multiple ways. Proprosed encryption leaves low-resolution data intact, higher-resolution (customer pays for) encrypted. [MakiK93] Maki, Ken "The AT&T EO Travel Guide", John Wiley and Sons, 1993 (electronic book) User guide for EO Personal Communicator. Describes Gestures of GO/PenPoint operating system, including flick gestures, brackets, circle, caret, down-right, up-right, etc. Includes hardware specifications for EO 440. [MakkuniR91a] Makkuni, Ranjit; Zdybel, Frank Jr.; and Stults, Robert A. "Gesture-modified Diagram for Retrieval of Image Resempling diagram, with Parts Selectable for Further Interactive Retrieval", United States Patetn 5,010,500, April 23, 1991 User sketches roughly what they are looking for (with mouse), system finds similar line drawing (in video?) with similarity in features. Describes sketching input as gestures. [ManderR92a] Mander, Richard; Salomon, Gitta; and Wong, Yin Yin "A 'Pile' Metaphor for Supporting Casual Organization of Information", Proc. CHI '92, May 3-7, 1992, pp. 627-634 Desktop/tabletop user interface: user can group/organize documents into a 2.5D pile or stack. Postagestamp representation in pile. Browsing documents without un-stacking: currently browsed document indicated by a tab or being pulled out of pile slightly. Compare with Wang Freestyle? [MartinP92] Martin, Partricia A; Huntington, John T II; McNally, J. Michael; Barrett, David M.; Ward, Jean R. "Computer with Tablet Input to Standard Programs", United States Patent 5,148,155, September 15, 1992 Simulated devices for input to a computer with integrated digitizer/display: devices included writing pad, simulated mouse, simulated virtual tablet, simulated virtual soft keyboard. Devices implemented in a separate control processor, no special device drivers. Long button press on simulated mouse button locks mouse button into "down" state for press-and-drag. press-and-hold as counterexample? See also 5,491,495 Ward et al. Three patents on "The Guide" portable slate computer from Wang Freestyle group. [MasimoT93a] Masimo, Tamon; Hanazawa, Hiroshi; Masuzaki, Hidefumi; and Ito, Satoshi "Display Apparatus with Rotatable Display Screen", United States Patent 5,189,404, February 23, 1993 Automatically rotate orientation of user interface/messages when display is rotated to vertical elongation (portrait) orientation or horizontal elongation (landscape). Compare with Radius Technology, Sakamoto 94? [MaulsbyD92a] Maulsby, David L. "Prototyping an Instructable Interface: MOCTEC", Proc CHI '92, May 3-7 1992, pp. 153-154 System learns to recognize patterns in text from examples by user, such as recognizing who won a game in a series from a list of teams and scores. Highlighting of focus items. See also MaulsbyD89. [MaulsbyD93a] Maulsby, David L. and Witten, Ian H. "Metamouse: An Instructable Agent for Programming by Demonstration", Chapter 7 in "Watch What I Do: Programming by Demonstration", Halbert, Kurlander, Turransky eds., 1993, acypher.com/wwid System learns to recognize graphical operations of user by pushing a graphical turtle on a display touchscreen, such as sorting graphical objects by size. [MayorT92a] Mayor, Tracy; Ray, Garry; and Paul, Fredric "Microsoft's numbers game. (Microsoft Corp.'s Microsoft Windows 3.1 graphical user interface and Microsoft Windows for Pen Computing operating system)", Lotus Magazine, June 01, 1992 Announcement of Pen Windows / Microsoft Windows for Pen Computing, extensions to Windows 3.1 for pen-based or stylus-based computers [McCallaTR92a] McCalla, Thomas Richard "Digital Logica and Computer Design", McMillan, 1992 (hardcopy book) Upper-level textbook on digital logic, with focus on construction of an ALU/CPU. [McLeanJG92a] McLean, James G.; Pickover, Clifford A.; and Reed, Alvin R. "Three Dimensional Mouse via Finger



Ring or Cavity", United States Patent 5,095,302, March 10, 1992 3D three-dimensional mouse, Z axis input by a finger ring user can pull up or down, depth mounting post measured by string rotor, piezoelectric pressure transducer (PPT) (actually force), optical pick-up, or pressure (force) transducer under thumb. [McNultyT93a] McNulty, Tom and Suvino, Dawn M. "Access to Information: Materials, Technologies, and Services for Print-Impaired Readers", American Library Association, 1993 (hardcopy book) Industry Overview of print-related accessibility technologies (talking books, screen readers, large print, etc.) for visual impaired, 1993. Mentions IBM ScreenReader with version for OS2/Presentation Manager, others prior to 1986. 1986 given as date for fast and furious development: Business Vision by Artic Technologies; Flipper by Omnichron, JAWS by Henter-Joyce (later Freedom Scientific); Screen-Reader by IBM, Softvert by Telesensory systems, Vocal-eyes by GW Micro: separate notes on outSpoken. JAWS and IBM Screen-Reader use dual-cursor instead of screen review: Screen-Reader uses separate additional keypad. [MichelS93a] Michel, Steve and Coleman, Dale "MacWrite Pro Inside & Out: A comprehensive Guide to MacWrite Pro", Osborne / McGraw-Hill, 1993 (hardcopy book) MacWrite word processing / electronic publishing. Find/Search uses wildcards, displays actions to change or not change text. Can also find patterns by font attributes (bold, etc.) [MicroTouchUnMouse92] MicroTouch Systems Inc. "UnMouse Product information", MicroTouch Systems Inc., Methuen Massachusetts Touchpad (relative motion) digitizer to replace a mouse: includes an absolute-position mode, and the use of pre-defined function templates to define additional programmable keypad. Compare programmable keyboard with Kaplow Pressure/force-sensitive touchpad uses light touch to simulate proximity sensing / mouse tracking on tablet, press harder to simulate touch contact UnMouse marketed as accessibility aid device for the handicapped File contains additional materials also [Microsoft91a] Microsoft "Microsoft Mouse Programmer's Reference", Microsoft Press, 1991, ISBN 1-55615-336-8 (hardcopy book) Description of MENU.COM, *.MNU files, MENUMAKE.DOC, MENUREAD.ME. Mouse menus allow for support of mouse in DOS applications that are not "mouse-aware". MENU.COM uses static menu definitions, expert menus are *.COM files with specific knowledge of application. Compare with keyboard macros, and with recognition macros by PenCept and CIC for handwritten gestures. [Microsoft91b] Microsoft "Microsoft Announces Visual Basic at Windows World '91", Microsoft Knowledge Base on Compuserve, "Visual Basic" through June 25, 1991: fetched 2012 from yfondeur.tripod.com/vbkbase1.htm Announcement of Visual Basic by Microsoft with support for Pen Windows / Windows for Pen Computing. [Microsoft91c] Microsoft "The Windows 32-bit API: An Overview", Microsoft Corp., 1991 Technical roadmap from Microsoft for transition from 16-bit Windows APIs and DOS base to 32-bit API for Windows 3.1, Windows NT, and Windows for Pen Computing. [Microsoft91d] Microsoft "MS-DOS 5.0 User's Guide and Reference -- Gateway Edition", Microsoft Corp., 1991 (hardcopy book) MS-DOS 5.0 reference manual, command line reference [Microsoft92a] Microsoft "Differences Between VB 3.0 Standard and Professional Editions", support.microsoft.com, article Q113334, 1992, republished October 29, 2003 Revision 2.0 as ID 113334 Visual Basic 3.0 Pro edition: Pen controls: Bedit, HEdit, PenInk on BitMap for electronik ink, Pen On-screen Keyboard synthetic keyboard for input to the active control. Mentions pop-up context menus (or just popup menus) and pen BEdit and HEdit control, on-screen "synthetic keyboard" (like SIP soft input panel?). Equates pop-up/popup menu and context menu as the same thing. Pen Computing development environments: Visual Basic 3.0/Pro with pen controls included [Microsoft92b] Microsoft "The Windows Interface: An Application Design Guide", Microsoft Press, Microsoft Programming Series, 1992 (hardcopy book) Section on The Pen Interface (Pen Windows): pen techniques include tap, press-hold-drag, write-draw. List of gestures include pig-tail, up-right (Tab), down-left (newline), undo, check-mark, with list of keyboard equivalents. Mentions circle gestures, both positional and non-positional for pen-aware and penunaware applications. Use of pointers (cursors) to provide feedback for fine positioning, drag areas, what pen actions are currently enabled. Pop-up menus with either mouse button. Two editions: one contains floppies with software, different price, same content except for advertising in the back. [Microsoft92c] Microsoft "Microsoft Knowledge Base Articles: Pen Windows", Support.microsoft.com, fetched 2011/2012/2013 File contains information on original (or early) publication date. Pen Windows: "How to Rotate a Pen Windows Application" Rotate the window of a single application Pen Windows: "Creating Pen Windows Dictionaries" for shapes and glyphs for handwriting recognition Pen Windows: "SYSTEM.INI Settings Required for Windows Pen Extensions" Pen Windows: "How to Set Up Desktop Computer for Developing



Pen" Pen Windows: "Applications that Run with or Without Pen Extensions" [Microsoft92d] Microsoft "Microsoft Chicago Feature Specification / Windows for Pen Computing", Microsoft, 1992. Revision 1.00 9/30/92. Fetched 2011 iowa.gotthefacts.org/011607/5000/PX05516.pdf Microsoft Feature specification for Windows for Pen Computing / PenWindows. Mouse gestures, Pen compatibility in display device drivers (inking data). [Microsoft92e] Microsoft "Microsoft Excel for the Macintosh", Microsoft, 1992 (hardcopybook)(Data CD) Microsoft Excel spreadsheet program, 1992. Includes Function Reference, two volume User Guide. [Microsoft92f] Microsoft "Microsoft Windows for Pen Computing 1.0.7", Microsoft, 1992: From abandonware web site 2010 Installation files (?) for Pen Windows. [Microsoft92g] Microsoft "Microsoft Word User's Guide, Apple Macintosh Series, Version 5.0", Microsoft, 1992 (harcopy book) Microsoft WORD for Macintosh. Pop-up help menus/balloons (on dwell?) a.k.a. tooltips. Help/context menus by help key, then clicking on element. Compare with right mouse button in Windows. [Microsoft93a] Microsoft "Understanding Pen Drive functionality under Windows", support.microsoft.com, article Q94701, republished January 8 2003 as ID 94701 Mouse/Pen driver for Microsoft Windows for Pen Computing 1.0 (Pen Windows 3.1). Sample driver was for a Wacom tablet: Scriptel also supported (other reference). Screen rotation. States that pen behaves similar to a mouse when Windows for Pen Computing is not present. Includes support for non-integrated separate tablet. Tablet data rate and resolution can be set dynamically. Additional materials in paper file. [Microsoft93b] Microsoft "Microsoft Visual C++ User's Guides: Visual Workbench User's Guide; App Studio User's Guide", Microsoft Corporation, 1993 Tutorial documentation on Microsoft C++ Development System for Windows Version 1.0. Supplants Programmer's Workbench (PWB). Not same as more expensive professional edition. For Visual C++ 2.0? Describes browser windows, GUI editor with component palette, similar to SIP Soft Input Panel in Tablet PC. [Microsoft93c] Microsoft "Microsoft Visual C++ Programmer's Guides: C++ Tutorial; Class Library User's Guide; Programming Techniques", Microsoft Corporation, 1993 (hardcopy book) Microsoft Visual C++ Development System for Windows. Shows Toolbar editor for editing GUIs on page 79 of Scribble example. For early Windows 3.1. [Microsoft93d] Microsoft "Q94701: Understanding Pen Driver Functionality Under Windows", Microsoft Corporation, 1993 (republished 2003 as Article 97401 Pen Driver (mentions WACOM tablet) for Windows for Pen Computing / PenWindows 1.0, Windows 3.1. Refers to "Architecture of Windows Pen Drivers". Note that PenWindows has "pen palette" pop-up handprint recognizer, compare with SIP soft input panel pop-up keyboard [MillenDR93a] Millen, David R. "Pen-Based User Interfaces", AT&T Technical Journal, May/June 1993, pp. 21-27 Overview of PenPoint OS with core gestures, Windows for Pen Computing core and six additional gestures (does not list tap). GO MiniNote ink-editing/sketch ink note application. Brief description of prototype AT&T PDA communications device with ink communications, handwriting recognition, diamond-shaped icons on display -- few details. [MinnemanSL91a] Minneman, Scott L. and Bly, Sara A. "Managing a trois: a study of a multi-user drawing tool in distributed design work", Proc. CHI '91, pp. 217-224 Study of shared drawing surface / multi-user whiteboards. Audio communication only better than Audio/video communication with whiteboard, as users were less distracted and drew better. No details on hardware, but looks a bit like Scriptel transparent digitizer over plasma display, horizontal for drawing-desk mode. [MolvaR92a] Molva, Refik; Tsudik, Gene; Van Herreweghen, Els; and Zatti, Stefano "KryptoKnight Authentication and Key Distribution System", Proc. ESORICS '92 Second European Symposium on Research in Computer Security, pp. 155-174 Layered network protocol for PKI private key distribution, identity delegation, authentication of users and of document origin and contents (digital signature). Encryption modularized to accomodate US export controls. [Momenta91a] Momenta Corp. "Momenta User's Reference Manual", Momenta Corp. Oct 1991 GUI for pen computer. Cited in PierK92a for pop-up menus, pop-up keyboard SIP panel. Cited in Kurtenbach93 for minimum distance on press-and-hold to drag. Page 9.19 changing "viewing orientation" by rotation [Momenta91b] Momenta Corp. "The Momenta Computer: Background Information", Momenta Corp. 7/26/91 Slate tablet computer, mentions Tin Oxide (ITO?) with weak voltage gradient for touchscreen resistive digitizer, voltage conducted back to computer via tether/cable, which darkens LCD pixel to create electronic ink. Command compass marking menu. Sketch recognition. Soft-pedals handwriting recognition



as "not there yet". Applications appear to be based on electronic ink, on MS-DOS. Implementation is SmallTalk 80. [Momenta91c] Momenta Corp. "A Week in the Life: The MOmenta Computer in Action", Momenta Corp. 1991 MS-DOS pentop computer. Marketing material of using Momenta slate-format computer in a working day. [Momenta91d] Shapiro, Eben "Business Technology: Computers Without Keyboards", New York Times, November 14, 1990 Tablet/pen computers from Momenta, Go (PenPoint), IBM NCR, Grid Systems (owned by Tandy). File contains additional references on Momenta Pentop Computer, etc. [MontanizF91a] Montaniz, Frank and Mack, Robert "A Comparison of Touch Interface Techniques for a Graphical Windowing Software Environment", Proc. of Human Factors Society 35th Annual Meeting, 1991, pp. 286-289 Selection gesture (tap/lift-off) on touchscreen: Application provides visual indication of selection by highlighting before lift off: also mentions auditory and visual feedback (click?). Tab/hold/drag (press-hold) like click-hold-drag gesture with mouse. Highlighting feedback on touchscreen. [More93a] More, Edward S., and Aiken, John C. "Electro-optic Slate for Direct Entry and Display and/or Storage of Hand-entered Textual and Graphic Information", United States Patent 5,194,852, March 16, 1993 digitizer tablet combined with LCD display: shown in a PDA sample, hand-held slate computer also shows chain-code recognition of connected and disjoint characters Handwriting recognition is done in separate area from electronic ink: side-menu GUI for text-editing [MoskowitzD93a] Moskowitz, David and Kerr, David, et al "OS/2 2.1 Unleashed", SAMS Publishing, 1993 (hardcopy book) OS/2 Presentation manager, palette manager, scheme palette: dynamic scheme/theme management via GUI as pop-up context menu on mouse button 2. [MyersBA92a] Myers, Brad A. and Zanden, Brad Vander "Environment for rapidly creating interactive design tools", The Visual Computer, Springer Verlag, 1992, No 8: pp. 94-116 In Garnet system papers: describes Garnet UIMS toolkit. Look and feel independent, examples are lists, buttons, sliders. Humanoid toolkit built on top of Garnet. Prototype-based objects, objects saved to file. Dynamic editing of graphical objects, even while they are being viewed. Automatic graphical update. Has dynamic themes? [MyersBA92b] Myers, Brad A. and Rosson, Mary Beth "Survey on User Interface Programming", Proc. CHI '92, May 3-7 1992, pp. 195-202 In Garnet system papers: one need cited was problem with code being too tightly coupled to a particular toolkit. Most users used Motif, OpenLook, or commercial windowing e.g. DEC, Microsoft Windows [MyersBA92c] Myers, Brad A.; Guise, Dario A.; and Zanden, Brad Vander "Declarative Programming in a Prototype-Instance System: Object-Oriented Programming Without Writing Methods", Proc. OOPSLA 92, pp 184200, May 3-7 1992, pp. 195-202 In Garnet system papers: define (much of) interactive behavior in declarative parameters, fewer call-backs in GUI definition. Compares Garnet to SELF (prototype inheritance?). Edits to prototype object immediate appear in all instances, even in running system: example is buttons. Persistence via retained-object model. Gesture-Interactor class: e.g. free-hand "X" gesture to delete [MyersBA92d] Myers, Brad A. "State of the Art in User Interface Software Tools", Carnegie Mellon University report CMU-CS-92-114, February 1992, also in Advance in Human Computer Interaction, Vol. 4, 1992, Baecker95a (Toward the Year 2000) UIMS for Direct Manipulation. Designing a user interface is an iterative design process. [MyersBA93a] Myers, Brad A. "Garnet: Uses of Demonstrational Techniques", In "Watch What I Do", Allen Cypher, ed., MIT Press, 1993, pp. 219-233 In Garnet system papers: behaviors and styles of UI object can be defined at runtimes. Includes prototype/by-demonstration support for pen gestures. GILT (Garnet Interface Layout Tool) support graphical styles: style is a named set of properties that can be applied to widgets. Has dynamic themes? [MyersBA93b] Myers, Brad A.; Wolf, Richard; Potosnak, Kathy; and Graham, Chris "Heuristics in Real User Interfaces: Panel Discussion", Proc. INTERCHI '93, 24-29 April 1993, pp.l 304-307 In Garnet system papers: mostly about heuristics in interactive programs, examples are renumbering in word-processing program to be like numbering (to the extent detectable) user has provide as example. [MyersBA93c] Myers, Brad A. "The Second Garnet Compendium: Collected Papers 1990-1992", Carnegie Mellon University Report CMU-CS-93-103, February 1993 Single publication of multiple Garnet papers on UIMS design systems [MyersBA93d] Myers, Brad A. "Why are Human-Computer Interfaces Difficult to Design and Implement?", Carnegie Mellon University Report CMU-CS-93-183, July 1993 User interfaces and UIMS difficulties: UI design is iterative, complex, and hard to test due to interactions and usability considerations, also because new UI technologies (UI, pen-based gesture recognition, Undo for multiple users, etc.) become available.



[MyersBA93e] Myers, Brad A. "User Interface Software Tools", Encyclopedia of Computer Science and Technology, 1993, pp. 371-405 See Myers 95 reference, same title. [NCR91a] NCR "NCR 3125 User's Manual", NCR Corporation, October 1991 NCR PenWindows/PenPoint pen computer: also CIC PenDOS with DOS. Power management: System Standby when no stylus or keyboard input : stops CPU, disk heads parked and motor off, sensor tablet (touchscreen) in sleep mode (until woken up), serial ports off. Autosave does not save non system RAM data? Suspend mode? Wake-up by tapping display. [NCR92] NCR "NCR 5991 Signature Capture/Electronic Payment Terminal", NCR Corporation, an AT&T Company Signature capture digitizer terminal with integrated display, shows actual sales receipt on display not just opaque signature capture pad. Uses Scriptel electrostatic front-surface digitizer. [NCR93a] NCR "NCR 3125 con Windows 3.1", YouTube.com, fetched 2011 Video of NCR 3125 pen/touchscreen computer running Windows for Pen Computing / Pen-Windows, circa 1993 [NCR93b] NCR "NCR 3125 con Windows 3.1", YouTube.com, fetched 2011 Additional Video of NCR 3125 pen/touchscreen computer running Windows for Pen Computing / PenWindows, circa 1993 [NIST92a] National Institute of Standards and Technology NITS / National Security Agency NSA "Federal Standards for Information Technology Security (Volume 1 + Volume 2)", NIST, 1992 Draft standard for security procedures in IT for US Federal Government: to replace "Orange Book" "Trusted Computer System Evaluation Criteria": TCB trusted computing bases (e.g trusted system). Vol 1 Profile Protection Development, Vol 2 Registry of Protection Profiles (those considered acceptable). [NairA91a] Nair, A. and Leedham, C.G. "Preprocessing of line codes for online recognition purposes", Electronics Letters, Vol 27 No 1, Jan 3, 1991, pp. 1-2 (Abstract only) non-linear smoothing and filtering of digitizer tablet input to remove noise: compare with Ward 85/87 [NakagawaM93a] Nakagawa, Masaki; Machii, Kimiyoshi; Kato, Naoki; and Souya, Toshio "Lazy recognition as a principle of pen interfaces", Proc. CHI '93 INTERACT '93 and CHI '93, Conf. Companion on Human Factors in Computing Systems, pp. 89-90, ACM New York, NY, USA 1993 Boxed input: defer/delay recognition until user finishes writing, so that user is not distracted by recognition results or making corrections while thinking. Compare with deferred recognition in Slate applications for PenPoint/Pen Windows, Wang Freestyle device [NeXT92a] NeXT Computer "NeXTSTEP Release 3: A Demonstration with Steve Jobs", NeXT Computer: Video fetched from YouTube.com and methodshop.com, 2010 Date approximately 1991 or 1992. Spelling pop-up menus in UI, embedded menus in context? [NeXTSTEP92a] NeXT Computer, Inc. "NeXTSTEP General Reference, Volume 1", NeXTSTEP Developer's Library, Release 3, Addison-Wesley Publishing, 1992 (hardcopy book) NeXTSTEP Operating System and application user interface design. Build tools. [NeXTSTEP93a] NeXT Computer, Inc. "NeXTSTEP Object-Oriented Software, Development Tools and Techniques", NeXTSTEP Developer's Library, Release 3, Addison-Wesley Publishing, 1992 NeXTSTEP Operating System and application user interface design, IDE. Build tools. [NewmanW92a] Newman, William and Wellner, Pierre "A desk supporting computer-based interaction with paper documents", Proc. CHI '92, pp 587-592 Projected desktop on physical desktop, with machine vision recognition objects: describes projecting a virtual calculator for use on a physical desktop. [NorwoodDD91a] Norwood, Donald D. "Hybrid Information Management System for Handwriting and Text", United States Patent 5,063,600, November 5, 199 Electronic-ink notetaker application and mark-up. Screenshots who personal information manager / PIM with calendaring, notetaking, ink (addressing) of e-mail and fax. Suggests use of structured electronic ink and GUI instead of handwriting recognition. Cites to Wang Freestyle, Grid, IBM Paper-like interface, Toshiba Pen PC (1991). Compare to Slate DayTimer / PenScheduler, van Raamsdonck? [Novobilski92] Andy Novobilski "PenPoint Programming", Addison-Wesley, August 1992 (electronic book) General application programming tutorial for PenPoint. Chapter 7 "Using the Pen" describes acetate layer for temporary display of electronic ink prior to recognition: thus showing actual shape, compare with early "write anywhere" features of Tablet PC. Field types Integer, Date, Fixed, Text for character recognition by context. Compare with "Factoids" for Tablet PC from Microsoft. Example applications include twodimensional mathematics/arithmetic calculator, crossword puzzle, direct manipulation of character tile objects. orientation/rotation for landscape mode p. 198, prOrientation [OConnerRJ92a] O'Conner, Rory J. "Apple banking on Newton's brain", San Jose Mercury News, April 22, 1992 Upcoming release of Apple Newton, describes recognizing "fax Scully" and invoking fax application [OSF93a] Open Software Foundation "OSF/Motif Style Guide: Release 1.2", Prentice Hall, 1993 (hardcopy book)



Mouse with up to three buttons: BSelect for leftmost button (except for left-handed), BTransfer middle button for moving and copying, and BMenu rightmost button for popping up menus. Also mentions mousebutton gestures such as MultiClick (double/triple/etc. mouse clicks), and MultiPress: clicks of mouse button then a normal mouse hold, and BSelect Motion 2 (notation of click). Pointer shapes (cursors) to show state of interaction: pointing arrow, I-beam for Text, X for outside of application input: fine-point and move cursors, has both hour-glass and watch "busy" pointers. Separate upper-right arrow for menu selection state. Defines pop-up menus as (context menus), in that they relate to the area pointed to. [OSF93b] Open Software Foundation "OSF/Motif User's Guide: Release 1.2", Prentice Hall, 1993 (hardcopy book) Page 3-9, definition for pop-up menus in BMenu for three-button mouse. Drag gesture/operation in process canceled with "Cancel" button. [OSF93c] Open Software Foundation "OSF/Motif Programmer's Guide: Release 1.2", Prentice Hall, 1993 (hardcopy book) Popup menus. Various visual effects for dragging over/under. [OlsenDR92a] Olsen, Dan R. jr. "Bookmarks: An Enhanced Scroll Bar", ACM Trans. on Graphics, Vol 11 No 3, July 1992, pp. 281-295 Bookmarks for electronic document: marks, tabs, or labels, next to document scroll bar. Part of Garnet project. Compare with Russian Patent on scrollbar with marks? [OlsenDR93a] Olsen, Dan R.; Foley, James D.; Hudson, Scott E.; Miller, James; Myers, Brad "Research directions for user interface software tools", Behavior and Inf. Tech., 1993, Vol 12, No 2, pp. 80-97 Listed in Garnet project papers: Overview of UIMS tools, with emphasis on malleable/changeable programming: programming by demonstration, constraint systems, rule-based systems, re-use. Implementations refer to pipes, shell-scripts, and other features more common on Linux than on Windows etc. [Oram91] Oram, Andrew and Talbott, Steve "Managing Projects with make, second edition", O'Reilly and Associates, 1993 (reprinted with corrections) (hardcopy book) Unix "make" project builder. Refers to recursive invocation of make, GNU make. First Edition 1986, Second Edition 1991. [Organek93a] Organek Technology "PenPut - Beyond character recognition!", Product announcement: PenPut, and SmartCase. Organek Technology, 695 Mariposa Avenue, Suite 304, Oakland CA 94610, (510) 438-9461 Zone-based gestures instead of characters: pen moves in one of eight directions and three different distances to make a character, next character is made without lifting the pen: Unistroke character recognition for whole words, using special directional abbreviations Unistroke short-hand input of whole words for English: claims input speeds to 35 WPM [PCMag91a] PC Magazine "The 8th Annual PC Magazine Technical Excellence Awards 1991: Standards and Operating Systems Winner: PenPoint / Robert Carr and the PenPoint Development Team", PC Magazine, 1991 Award to PenPoint as operating system, mentions connectivity and gestures in pen/tablet computer. [PCMag93b] PC Magazine "Pen Pals - Personal Digital Assistants - You Can Take Them Anywhere -- But are they ready?", PC Magazine, Vol 12 No 17, October 12, 1993, Cover story Review of PDAs/pen computers: Tandy/GridPad, Amstrad Pen Pad PDA600, Apple Newton MessagePad, AT&T EO 440 Personal Communicator (PenPoint), and Tandy Z-PDA/Casio Z-7000 Personal Digital Assistant, NEC Versapad, Dauphin DTR-1 Desk Top Replacement, Fujitsu 325Point, Grid PalmPAD. Pen operating systems PenRight, GO PenPoint, CIC PenDOS (no Pen Windows?), Toshiba DynaPad, etc. Mentions landscape/portrait orientation rotation [PCWeek91a] PC Week "On Go's Heels, Microsoft Plans Pen-Based OS", PC Week, Jan. 14, 1991 Press clippings on GO/PenPoint and Microsoft/PenWindows early announcements, PenWindows following PenPoint and Grid, etc. Different approaches. Multiple vender support for PenPoint. Borland: compatibility with Windows less important. Go's PenPoint Redefines Handwriting Recognition, August 20, 1990 Go Corp. Gains Ground in Pen-Software Race, July 23, 1990 Pen-Based PCs Herald Era of Handwritten Input: Applications Evolving Slowly / NCR Readies Stylus System; October 8, 1990; Mentions CIC Handwriter Spencer F. Katt July 23, 1990 GO Corp. Is Mum on New Operating System; August 13, 1990 [PCWeek93a] PC Week "Apple adding tablet-sized PDA", PC Week, December 6, 1993 Tablet-size Newton Pen Computer, codename BIC, data/fax modem (landline) [PCWeek93b] PC Week "Eo to marry pens, cell phones", PC Week, p. 25, vol 10 no 49, December 13, 1993 Press notice about Loki (device) Amstel (version of PenPoint) smartphone. Under one pound weight. Eo 440 and 880 add-on cellular modem. [PRNewswire93a] PRNewswire "First Newton - The MessagePad - Hits the Market", PRNewswire, July 30, 1993 Newton MessagePad introduced, with wireless communications, OS/Newton to be licensed. Can send/beam documents between Newton devices. [ParadisoJ93a] Paraidso, J. "Synchronous Proximity Detection for Stretched Wire Alignment Systems", C.S. Draper Laboratory, Cambridge Mass, Technical report GEM-TN-93-447, August 1993



Capacitive proximity/position sensor for muon barrel gun project: includes X/Y grids of wires for X/Y position. Checking alignment of a stretched wire parallel to another conductor line. Compare with X/Y electrostatic grid tablets? [ParizeauM93a] Parizeau, Marc and Plamondon, Rejean "Allograph Adjacency Constraints for Cursive Script Recognition", Proc of Third IWFHR International Workshop for Handwriting Recognition, Buffalo NY, 1993, pp 252-261 Probabalistic segmentation of cursive handwriting into separate characters: operates without word dictionary / linguistic knowledge [ParkerTA91a] Parker, Thomas Anthony "Access control in a distributed computer system", European Patent Application 0 456 386 A2, November 13, 1991 Token-based login/authentication system: user logs in, gets priviledge attribute certificate for accessing all resources. Access and use rights to specific resources checked by central use manager server. [PavlidisT92a] Pavlidis, Theo; Swartz, Jerome; and Wang, Ynjiun P. "Information Encoding with Two-Dimensional Bar Codes", IEEE Computer, June 1992, pp. 18-28 Overview of two-dimensional bar codes circa 1992: Code 16K, Code 49, PDF417, Identcode MLC-2D, Datacode Philips dot code, Softstrip, Manchester code. [PedersenER93a] Pedersen, Elin Ronby; McCall, Kim; Moran, Thomas P.; and Halasz, Frank G. "Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings", Proc. INTERCHI '93, April 1993; pp. 391-398 Remote whiteboard application on Xerox Liveboard with gesture user interface: line (horizontal) gesture to edit electronic ink to create space, lasso/loop gesture, compound gestures. Double-tap causes previous stroke to be interpreted as a gesture, holding pen barrel button (Pencept?) to indicate gesture. Up to three pens/user simultaneously (multi-touch): users could thus share selections and graphical tools. [PenMagic93a] PenMagic Software Inc. "PenMagic Numero! Reference Manual: Your guide to using Numero and designing paper: Release 2.0", PenMagic Software Inc., 1993 (hardcopy book) Handwriting spreadsheet application for PenPoint tablet-computer platform. Integrated with PenPoint document-centric user interface. Application specific gestures e.g. circle-tap and tap-tap-press. Some PenPoint gestures redefined: e.g. caret. [PenMagic93b] PenMagic Software Inc. "PenMagic Numero! Getting Started: Your Guide to learning Numero Release 2.0", PenMagic Software Inc., 1993 (hardcopy book) Handwriting spreadsheet application for PenPoint tablet-computer platform. Application specific gestures include circle-line. Standard PenPoint pigtail gesture for delete. [PenMagic93c] PenMagic Software Inc. "PenMagic MagicScript Visual Scripting System Reference Manual: Everyone's guide to scripting integrated applications for PenPoint", PenMagic Software Inc., 1993 (hardcopy book) (hardcopy book) Graphical scripting language: drawing UI for establishing elements and data connectors. Compare with PenApps from Slate, Visual Basic for Windows for Pen Computing. [PenPoint91a] PenPoint "PenPoint UI Design Guidelines", GO Corporation, Feb 15, 1991 User interfaces / gesture /stylus design guidelines for PenPoint OS operating system. Mentions proportional flicking gestures for scrolling: longer flick gesture, scroll farther. PDF page 65 Choosing a screen layout: rotation to change orientation [PenPoint91b] PenPoint "Video Demonstration, PenPoint Corporation 1991", GO Developers Summit, 1991, Jerry Kaplan Video presentation about PenPoint/GO operating system. Mentions Slate Corporation PenApps, electronic ink mark-up/annotation, ink as a datatype. Mnemonic gestures: e.g. "u" gesture to underline. Square bracket gestures to extend text selection. Approx 32:00 moving marquee animation after press and hold. Also shows portrait mode orientation (rotation) [PenPoint91c] PenPoint "Using Penpoint: Developer Release - Beta Edition", GO Corporation, 1991 User interfaces / gesture /stylus design guidelines for PenPoint OS operating system.PDF page 208 Choosing a Screen Layout (rotation of screen orientation) [PenPoint91d] PenPoint "PenPoint Pocket Guide: Using the Notebook / Eleven Basic Pentpoint Gestures", GO Corporation, 1991 Double-sided 8 1/2 x 11 laminated sheet. Internal hyperlinks within/between documents, notebook editing, installer. Basic PenPoint/Go gestures: Bracket left, Bracket right, Caret (insert via pad), Check (option/properties sheet), Circle (edit pad), Cross-out (X/Delete), Pigtail (delete), flick, insert space, press, tap, tap-press (copy). [PenPoint92a] PenPoint "PenPoint Architectural Reference Volume 1; PenPoint Architectural Reference Volume 2, 1992", GO Technical Library, Addison-Wesley Publishing, Includes long list of gestures, shown in a Font table. Orientation/rotation page 262, Vol2 363 [PenPoint92b] PenPoint "PenPoint User Interface Design Reference", GO Technical Library, Addison-Wesley Publishing, 1992



Page 133 list of gestures, some of which are upper-case character e.g. letter B for Bold: Compare with PenCept recognition macros circa 1985. Flick gestures, compound gestures of multiple uses of the same sub-gesture strokes, with semantically similar meaning. Handles: resize handles at the corners and at the midpoint of the edges of sides of object's bounding rectangle. Selection can be extended by dragging (a handle on a bounding box?). Describes user interface of the PenPoint operating system; Describes use of various gestures (page 15-16, 23-27), creation of reference buttons wherein tapping the button once turns to a page in the document and double-tapping floats the document in front of the notebook (page 20) PenPoint gestures for moving (tap and hold) and copying (double-tap and hold) objects wherein a dashed outline or double outline is provided to the user as feedback (page 68, 286), icons that respond to several gestures (page 74), reference buttons that respond to several gestures (page 79), designing icons wherein bitmap (active region) is larger than icon (page 218), designing icons that give the user useful feedback by changing the icon's picture to reflect the state of the application (page 222), determining the center of a gesture's "bounding box" (page 231), and incorporating selection feedback when designing objects (page 278-79). PDF page 277 PenPoint app window orientation/rotation, also check boxes for preferences [PenPoint92c] PenPoint "PenPoint Application Writing Guide: 1st ed. 1992; 2nd ed. 1993", GO Technical Library, Addison-Wesley Publishing, March 1992 Operating system based on Notebook metaphor rather than Desktop metaphor: Running documents are instances of application classes, visual components of a notebook, Section name and page number location combine to form a location in the OS file system. Refers to browser component (for files and other objects). File system is not separate memory, describes installation process as different from MS/DOS. User Interface UI Toolkit handles switch between landscape and portrait display orientation/rotation by adjusting horizontal and vertical coordinate layout. PDF page 18 automatic layout when orientation changes [PenPoint92d] PenPoint "PenPoint API Reference Volume 1; PenPoint API Reference Volume 2", GO Technical Library, Addison-Wesley Publishing, March 1992 (electronic book) Operating system using gestures based on Notebook metaphor rather than Desktop metaphor -- Class Manager, PenPoint Application Framework, ImagePoint; PenPoint UI Toolkit; File System; Text Component for text editing with gestures, System Service; PenPoint Resource Files. prOrientation: resource id for Screen orientation (display rotation) for landscape or portrait modes, also portrait and landscape reversed. [PenPoint92e] PenPoint "PenPoint Development Tools", GO Technical Library, Addison-Wesley Publishing, March 1992 Operating system based on Notebook metaphor and using gestures rather than Desktop metaphor -PenPoint DB debugger, profiling, Mini-Debugger, DOS file system utilities, S-Shot Screen Capture Utility; Running PenPoint on a PC [PenPoint92f] PenPoint "The PenPoint Technical Papers", GO Technical Papers, GO Corporation, 1992 Pen-based Operating system using gestures based on Notebook metaphor rather than Desktop metaphor. Technical papers describing main features and market opportunities of mobile pen-computing in general and of PenPoint in particular. "Position Paper: GO Corp.'s PenPoint (TM) Handwriting Recognition Technology": Context engine, handwriting recognition system replaceability. "Position Paper: GO Corp.'s PenPoint (TM) System-Wide Object-Oriented Programming": OO classes, sub-classes. "Background Information: GO Corp.'s PenPoint (TM) Operating System Communications Capabilities": deferred/intermittent connectivity, services can be installed/updated dynamically without reboot, delayed binding (compare: Multics) so that order of installation not important. Deferred printing. "Background Information: GO Corp.'s PenPoint (TM) Development Environment": Source-level symbolic debugger, supports Class Manager objects, scriptable debugger, breakpoints at source and message (method?) level. "Background Information: GO Corp.'s PenPoint (TM) Operating System For Mobile, Pen Computers": Quick stop (hibernate) and start-up because only data memory need be saved: execute-in-place no code relocation on program load. Shared reentrant copies of libraries/classes between applications. Shows autolayout. [PenPoint92g] PenPoint "Getting Started with PenPoint version 1.0", GO Corporation, 1992 Pen-based Operating system using gestures based on Notebook metaphor rather than Desktop metaphor. Section on Gesture Profiles, operations of same semantic gesture (e.g. select) in different contexts. [PenPoint92h] PenPoint "PenPoint for the Samsung PenMaster", Samsung Part Number SPM-PON, 1992 (hardcopy) Unopened software package: PenPoint operating system with user manuals and media for Samsung Penmaster [PenPoint92i] PenPoint "PenPoint: Using GOWrite Version 1.0", GO Corporation, 1992 Manual for handwriting recognition / gesture text editor for PenPoint operating system. Unboxed and Boxed/guideline input, scratchout/rubout gesture can encompass multiple boxes, caret-tap gesture for



opening a writing pad. Also cites PenTOPS by Sitka Corporation. [PenPoint92j] PenPoint "PenPoint Quick Reference", GO Corporation, 1992 8 page, laminated reference for PenPoint OS. Floating/embedded documents, single/double flick gestures, circle-tap gesture to translate electronic/digital ink, left/right gesture to change between gesture/writing modes in MiniNote note-taker with electronic ink editing: "U" combine gesture to combine selections, +/plus gesture to add to selection, clean-up gesture to re-space handwritten input. [PenPointPress91] PenPointPress "Press announcements on GO/PenPoint", (Various) Press announcement related to GO/PenPoint: business with IBM, etc. [PenRight93a] AST Research "PenRight! Application User's Guide", PenRight! Pro Software Development Kit for C programmers version 3.5, July 1993, AST Research, Fremont CA (electronic book) Contains chart of handwritten character shapes for discrete characters, strongly reminiscent of PenCept PenPad and CIC Handwriter. Accented characters, dieresis (umlaut), tilde. Blank is struck-through lowercase B, or underbar, not inverted "U". Boxed fields: Overstrike fields, or insert (compare with PenCept patent?): text-editing UI includes an insertion bar gesture for handwriting, line break (compare with PenWindows and PenPoint) pop-up keyboard. All for MS-DOS. Pop-up virtual keyboard and SIP input panel? [PenRight93b] AST Research "PenRight! Pro Software Development Kit for C programmers, Version 3.5", AST Research, Fremont CA July 1993 (hardcopy book) Pen development SDK: includes floppies, documentation (6 manuals). [PenRight93c] AST Research "PenRight! Pro Tools User's Guide", PenRight! Pro Software Development Kit for C programmers version 3.5, July 1993, AST Research, Fremont CA (electric book) Pen development SDK: Forms designer showing properties of fields. Resource editor: Bitmap Canvas, Code attributes, Generic data. CodeStart editor shows connections between forms, application consists of linked forms with code event handlers for particular events in objects on the form: CodeStart editor associates code scripts (which may contain embedded "C" code fragments) to actions. Separate handler actions for each graphical element (e.g. button). Commands to change document from landscape to portrait orientation on the display. [PenRight93d] AST Research "PenRight! Pro Learning Guide", PenRight! Pro Software Development Kit for C programmers version 3.5, July 1993, AST Research, Fremont CA (hardcopy book) Pen development SDK: Forms designer showing properties of fields. Resource editor: Bitmap Canvas, Code attributes, Generic data. CodeStart editor shows connections between forms, application consists of linked forms with code event handlers for particular events in objects on the form: CodeStart editor associates code scripts (which may contain embedded "C" code fragments) to actions. Separate handler actions for each graphical element (e.g. button). [PenWindows92a] PenWindows: Microsoft Corporation "Microsoft Windows for Pen Computing: Programmer's Reference, Version 1 Designed to work with Windows 3.1", Microsoft Press, 1992 (electronic book) Pen Message Interpreter, Pen Macro Layer for gestures: does not contain list of gesture shapes. Pages 16..20, description of differences between mouse and electronic ink drivers, coalescing of mouse messages to report final position (lift) only, etc. Refers specifically to shape recognition for gestures. Uses term "enhanced pen aware" or "pen-enhanced" instead of "pen-centric" for applications designed specifically for pen/gesture/handwriting input. Page 8, difficult to point at single pixels (small targets) with stylus touchscreen accuracy. Defines strokes to down strokes (stylus in contact) and up strokes (stylus not in contact), page 44. press-and-hold tap-and-hold TapNHold gesture does selection p. 184, 257. Page 7, Display Considerations portrait to landscape mode orientation (rotation) [PenWindows92b] PenWindows: Microsoft Corporation "PenWindows: Overview of the Pen API - PENAPI.HLP", Microsoft Corporation - PENAPI.HLP for PenWindows 1.0 HLP file with documentation for Pen Windows 1.0 -- 1992 [PenWindows92c] Microsoft Corporation "Microsoft Windows for Pen Computing: Programmer's Reference", Microsoft Corporation, 1992 (hardcopy book) Windows for Pen Computing / Pen Windows: also known as Pen Extensions. Programming documentation for Pen Windows 1.0, for Windows 3.1. Note: Text is NOT identical to PenWindows92a. [PerlinK93a] Perlin, Ken and Fox, David "Pad: An Alternative Approach to the Computer Interface", Proceedings of SIGGRAPH 93 Virtual workspace, zooming into different sections to simulate the user-interface metaphor of a large physical workspace. Refers to this as a "Pad" interface. Shows zooming into sections of a calendar: compare with Slate "DayTimer" Pen-Scheduler product from 1991 (not cited, perhaps he was not familiar with it: Bricklin patent?). [PerrittHH93a] Perritt, Henry H. Jr. "Knowbots Permission Headers and Contract Law", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia



Association Intellectual Property Project, January 1994 DRM management / access control on NREN digital library for EDI electronic document interchange. Every work/document has an attached "permissions header", defining terms of use: NREN verifies request messages with permissions headers. Models include LEXIS/NEXIS, WestLaw, etc. Suggest encryption (other than PKI signatures) inappropriate to DRM access control, just adds overhead. Reviews copyright law and conflicts between legal contracts on access. [PetzoldC92a] Petzold, Charles "Programming Windows 3.1, Third Edition", Microsoft Press, 1992 (hardcopy book) Windows 3.1 UI and programming guide, 1992. Checker example shows using SendMessage to emulate mouse events from keyboard input. Describes buttons: touchdown on touch/click inside button, but a bit vague on moving outside of button before releasing. [PhilippsKL93a] Philipps, Kenneth L. "MetaInformation, The Network Of The Future And Intellectual Property Protection", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 DRM management conference. Essay on the role of meta-information in (in)security, on the example of network packet (ATM) data fiels. Historical note on "packetized" communication by couriers back to Roman times: did not know the content of the message, but who sent/received and when. [PierK92a] Pier, Ken and Landay, James A. "Issues for Location-Independent Interfaces", Xerox PARC Technical Report ISTL92-4, December, 1992 Discussion of pop-up menus and gestures for UIs that are physically scalable to work on very large surface, where location of a fixed menu on the surface may be inconvenient: gestures/input should be anywhere on the surface (like Freestyle?). Notes that mouse menus with stylus not usable. Gestures as accelerators without lifting the stylus: make an E to bring up a pie/marking menu and continue with the pie-selection stroke without the menu actually popping up. Does not mention menus built into multi-button mouse or tablet puck Mentions pie menus in X10 window system 1986 (also press-and-hold-still). X, C, circle, box, press-hold gestures. Refers to ambiguity of input as gesture/application input as "interference problem". Describes stylus button easy to use as a solution for this modality -- compare with Pencept in 1985? Windows for Pen Computing an add-on to Windows 3.1. Shows pop-up keyboard? pop-up handwriting bar : SIP Soft input panel? Has press-hold, and potentially press-move by moving after gesture without lifting stylus. Mentions press-and-hold "command compass" of Momenta [PinkasD93a] Pinkas, Denis and Caille, Phillippe "Method for obtaining a securitized cleartext attestation in a distributed data processing system environment", United States Patent 5,214,700, May 25, 1993 Public-key attribute certificates used as authorization token (attestation) in heirarchical authorization system. Advantage is token data does not need to be decrypted, don't have to contact central authority for permissions or checking. [PlaisantC91a] Plaisant, Catherine "HCIL touchscreen toggle switches (University of Maryland) Video presentation", University of Maryland, 1991. Fetched from YouTube 2014. Touchscreen virtual toggle and slider switches: May be the CHI '92 video. [PlaisantC92a] Plaisant, Catherine and Wallace, Daniel "Touchscreen Toggle Design", Proc. CHI '92, May 3-7, 1992, pp. 667-668 Touchscreen virtual toggle and slider switches: See also CHI '92 video (reference in file) Compare with Apple slide switch, Fig 2 bottom left? Also teaches equivalents of slider toggle and lever toggle (traditional toggle switch). [PollackA91a] Pollack, Andrew "Technology: Positioning the Electronic Stylus", The New York Times, March 17, 1991 Report on tablet digitizer for the new products known as pen computers: Wacom, Scriptel, Kurta, Calcomp, Summagraphics, Numonics, Seiko, Hitachi adapting opaque digitizers, Resistive coating from GridPad, Microtouch. Electrostatic from Scriptel. Cites Jean Renard Ward on digitizer impact on handwriting recognition accuracy. [PollackA91b] Pollack, Andrew "A Battle in Pen-Based Computers", The New York Times, January 22, 1991 Report on PenPoint from Go Corporation, plus Microsoft hot on the heels of PenPoint. Big Market expected. Mentions also Grid Systems (owned by Tandy/Radio Shack). Investors in Go include Mitch Kapor, Apple. Bill Gates following Mitch Kapor to Japan. Quotes Phillipe Kahn saying Microsoft was copying Go. Quotes Vern Rayburn of Slate, storying electronic ink image itself. [PressJ91a] Press, Jim "Secure Transfer of Identity and Privilege Attributes in an Open Systems Environment", Jnl. Computers and Security, Vol 10, No 2, April 1991, pp. 117..127 Review of various architectures for privilege attribute certificates PAC for granting access rights to files/object. ECMA Security Architecture. Third-party trusted arbitrators, single-sign-on SSO, trusted computing base (TCB). Cites to ECMA-138. [QuerciaV93a] Quercia, Valerie and O'Reilly, Tim "X Window System User's Guide: OSF/Motif 1.2 Edition, Volume



3 of The Definitive Guides to the X Window System", O'Reilly and Associates, Inc., 1993 (hardcopy) Pop-up and Tear-off menus. Command-line options to X Windows clients. [RamanaR92a] Ramana, Rao; Card, Stuart K.; Jellinek, Herbert D; Mackinlay, Jock D., and Robertson, George G. "The Information Grid: A Framework for Information Retrieval and Retrieval-Centered Applications", Proc. UIST 92, Nov 1992, pp. 23-32 X-buttons. Describes InfoGrid document management system that uses XButtons as described in Robertson article; describes feedback where thumbnail of document (button) expands into full size page when button is moved to document area. [RashidR93a] Rashid, Richard "Speech Transcript - Rick Rashid, Microsoft Professional Developers Conference 2003, October 29, 2003", www.microsoft.com - Microsoft News Center Asserts that Tablet PC came out of work at Microsoft Cambridge research lab, Charles (Chuck) Thacker. Demo applications (from Pen Computing group at Brown University? see Andries van Dam) are Math Pad, Magic Paper (sketch recognition) by Randall Davis at MIT AI lab. (No mention of Pen Windows) [RegisterDS93a] Register, David S. and Groover, Robert III "Portable Computer with Physical Reconfiguration of Display Connection for Stylus and Keyboard Entry", United States Patent 5,241,303, August 31, 1993 Convertible pen computer: touchscreen/digitizer+display swivels and folds to cover keyboard [RehgJM93a] Rehg, James M. and Kanade, Takeo "DigitEyes: Vision-based Human Hand Tracking", Carnegie Mellon University, report CMU-CS-93-220, 31 December 1993. Also in Proc. of European Conf. on Computer Vision, May 1994, Stockholm 27-DOF (!) sensing of hand position, 10 frames/second for hand position, pose, and joint angles. 3D mouse pointer interface, tracking multiple fingers (thumb and fourth/pinkie) as the left and right mouse buttons. Tracking with single camera, or multiple cameras. Compare with Wellner, other hand-tracking, hand gesture interfaces. [ReiherP93a] Reiher, Peter; Page, Thomas Jr.; Popek, Gerald; Cook, Jeff; and Crocker, Stephen "Truffles - a secure service for widespread file sharing", Proc. PSRG Privacy and Security Research Group Workshop on Network and Distributed System Security, February 1993 Trusted Ficus File System: Controlled (secure) file sharing across networks in different administrative domains, extending Unix security to replicated (high-latency-replication) files. File update inconsistency handled by periodic re-sync. [RheingoldH93a] Rheingold, Howard "The Virtual Community: Homesteading on the Electronic Frontier", AddisonWesley Publishing Company, Reading, MA 1993 General Magic, former architects of Apple Macintosh created personal intelligent communicator, stylus based PDA similar to Apple Newton. Thrust of book is electronic democracy: world network communication (later: blogs and email and internet) will make democratic revolutions hard to prevent. [Robertson91a] Robertson, George G., Henderson, D. Austin Jr. and Card, Stuark K. "Buttons as First Class Objects on an X Desktop", Proc. UIST 91, Nov 11-13 1991, pp. 35-44 Gesture-based buttons, actions tailorable/definable by user/designer, as for other first-class objects e.g. icons, menus, windows. Gestures mentioned include flick left/right/up/down, click, rubout, check, insert (inverted "V"?) Electronic ink ("mouse track") shows actual marking, removed after recognition. Gesture must start inside button due to X-Windows, therefore gestures such as circling/lasso not possible. Caret/insert/edit gesture brings up gesture (actually macro) editor to define which of standard gestures should be associated with an object, and what they should do. Relates to user interface toolkit called "XButtons" (for the X window system) (see http://en.wikipedia.org/wiki/X_Window_System); buttons support multiple gestures (click, flick, rubout, check) using mouse or pen input; compares XButtons to Apple's HyperCard system buttons and Xerox EmbeddedButtons. Has "holding down the left mouse button" does same as clicking right button. John Kelly on Lucent/Microsoft mentions X-windows and tablet/stylus. press-and-hold? [RobertsonGG93a] Robertson, George G. and Mackinlay, Jock D. "The Document Lens", Proc. UIST '83, Nov. 3-5 1993, Atlanta, pp. 101-108 Show multiple pages of document as thumbnails or tablet layout, magnify/zoom area of interest in rectangular section with text blended ("greeked"; frustrum/pyrimidal projection) on sides so not obscured. Compare to App Lens. [RosemanM92a] Roseman, Mark and Greenberg, Saul "GroupKit: A Groupware Toolkit for Building Real-Time Conferencing Applications", Proc. CSCW 92, november 1992, pp. 43-50; also in Baecker95a Toward the Year 2000 Toolkit for groupware application: concept of transparent overlays (with multiple cursors for multiple users) over the windows of an application. Toolkit used for experimenting with strategies for workspace control/sharing. [RosenbergC93a] Rosenberg, Craig and Moses, Bob "Future Human Interfaces to Computer Controlled Sound Systems", AES Audio Engineering Society 95th Convention, October 7..10, 1993, New York Overview/Introduction to user interface technologies and their advantages for computer-based audio



equipment. Includes touch keypads, gesture recognition (hand gestures using data-glove-type devices, not touchscreen gestures), XYZ 3D spatial tracking (electromagnetic, optical camera), haptic feedback (tactile, force-feedback, proprioception), dwell for visual tracking . [Rubine91] Rubine, D. "Specifying gestures by example", ACM SIGGRAPH '91 Proceedings, Computer Graphics Vol 25 No 4, 1991, pp 328-337 Trainable, unistroke/single-stroke gestures, single-stroke X and delete symbols, using hotspots, etc. Recognition of gesture on every input point "eager recognition", allows transition to direct manipulation without lifting stylus. Multi-touch gestures (LeeSK85) Cited in Lipscomb91, Arvo00. Paper file also contains numerous reference *to* Rubine for gesture UIs, sketch understanding and recognition, etc. [Rubine91b] Rubine, D. "Integrating gesture recognition and direct manipulation", Proc. of the summer '91 USENIX Technical Conference, 1991, available at http://reports-archive.adm.cs.cmu.edu/anon/itc/CMU-ITC100.pdf Describes mouse gestures in a trainable system; refers to "eager recognition" where recognition starts before stylus lift / end of stroke. "GRANDMA": Gesture recognition and direct manipulation in the same interface, by using sub-gestures in a direct-manipulation stroke, recognition of gestures within a stroke. Shows similar (single-stroke) gestures for moving text and moving drawing objects (taken from Buxton). Describes GRANDMA, a tool for building gesture-based applications using single stroke gestures (continuous lines); includes definition of gesture. [Rubine91c] Rubine, Dean Harris "The Automatic Recognition of Gestures", Ph.D. Thesis, Carnegie Mellon University, December 1991, CMU-CS-91-202 Describes mouse gestures in a trainable system; refers to "eager recognition" where recognition starts before stylus lift / end of stroke. "GRANDMA": Gesture recognition and direct manipulation in the same interface, by using sub-gestures in a direct-manipulation stroke, recognition of gestures within a stroke. Shows similar (single-stroke) gestures for moving text and moving drawing objects (taken from Buxton). Mentions 95% recognition rate means 1/20 gestures misclassified, therefore need easy access to abort and undo operations in user interface. Mentions Nakatani multi-touch device observed at Bell Labs to track up to 10 fingers simultaneously, and radio drum to sense 3-D positions of multiple styli/fingers. Brief review of PenPoint with press-and-hold, and Button Box press-and-hold. Mentions multiple-finger gesture recognition via MDP software, Sensor Frame hardware. Refers to multiple-finger gestures as multi-path (not just multi-touch), since more than one path/position may be traced using multiple fingers. Describes GRANDMA, a tool for building gesture-based applications using single stroke gestures (continuous lines); includes definition of gesture. Gesture recognition and definition system: trainable recognizer, simple API programming language for defining semantics/operations using GRANDMA Smalltalk MVC Model/View/Controller or Model/View paradigm. Refers to single-path gestures (mouse/status) and multipath gestures (multi-touch finger input with several fingers). Cited in Goodisman as comprehensive reference. Multi-touch input using DataGlove, Sensor Frame, or multiple-finger touch pad. Trainable classifier/recognizer for multi-touch gestures. G-clef music score editor patterned after Buxton (but used recognition of shapes, not of directional changes?) [Rubine92] Rubine, Dean Harris "Combining Gestures and Direct Manipulation", Proc. ACM Conf. on Human Factors in Computing Systems (CHI'92) pp 659-660, May 1992 Defines gesture as handmade mark to give a command to a computer, using placement/size/orientation of mark also. Gestures terminated and recognized on dwell (gesture), user may continue input (e.g. line gesture, dwell, drag line). GDP: Single stroke (Unistroke) gestures only, for mouse-based drawing program. GSCORE: Musical score editing. Eager recognition when user pauses, rather than at end of stroke. See also CHI Video, 1992. [RussellGF92a] Russell, Gregory F. "Pulsed Pen for use with a digitizer tablet", United States Patent 5,138,118, August 11, 1992 Electromagnetic digitizer tablet, stylus transmitter. Stylus signal is pulsed, may be modulated by FSK, PSK, AM, PWM pulse-width modulation to encode stylus buttons. Tablet electronics power down between pulses to save power. [SachsE91a] Sachs, E; Roberts, A; Stoops, D. "3-Draw: a tool for designing 3D shapes", IEEE CG&A, Nov 1991, Vol 11 No 6, pp. 18-26 6 DOF 3-D digitizer, using two hand-held sensors to sketch surfaces in 3-space. Compare with Polhemus digitizer. 3D gestures? (See also earlier 1989 IEEE SMC article, same title) [Sato91] Sato, Kimikatsu Sagamihara; Toyoda, Ryuichi; and Masaki, Takeshi "Pen-type Computer Input Device", United States Patent 5,027,115, June 25, 1991 Similar to PenMouse: mouse-like input device shaped like a pen, but not a tablet digitizer because it only picks up relative motion. Sensing of position is an electromagnetic (or optical pattern) pickup of a nibbed writing ball in the pen. [SchilitBN93a] Schilit, Bill N.; Adams, Norman; Gold, Rich; Tso, Michael M.; Want, Roy "The ParcTab Mobile



Computing System", Proc. Fourth Workshop on Workstation Operating Systems, 1993, pp 34-39 Description of ParcTab: small networked PDA with touchscreen interface: location-aware computing (office location), ubiquitous computing (worn like an ID badge). Windows migration: X Windows applications can follow user from workstation to workstation based on position information from ParcTab. (No GPS) [SchlosserPA92a] Schlosser, Philip A. and Kable, Robert G. "Digitizer Technologies for Pen-Based Computers", Pen Computing: the 1992/92 Industry Outlook, 1992 Business whitepaper on Scriptel transparent touchscreen/digitizer technology with proximity sensing. [SchmandtC93a] Schmandt, Chris "Phoneshell: the Telephone as Computer Terminal", Proc. of ACM Multimedia Conf, 1993 Telephone access to voice mail, Email, and rolodex databases using synthesized voice output [Scriptel91a] Ball, Brian R. "Scriptel inks pact with Apple: Pen-based computer only a year away?", The Daily Reporter, Tuesday, November 26 1991 Scriptel electrostatic digitizer with Apple pen computer: pre-Newton, pre-iPhone tablet computer prototype from Apple MacIntosh? [Scriptel92a] Scriptel Corporation "Scriptel Announces Manufacturing License Agreement with SamSung", Press Release, Scriptel Corporation, November 16, 1992 Samsung licenses Scriptel electrostatic digitizer for pen-computing market: cordless stylus. File contains additional material Scriptel inks pact with apple: Pen-based computer only a year away; NCR5991 Signature Capture / Electronic Payment terminal; Dauphin 5000 Pen-base Notebook computer (DOS/OS2). [Scriptel92b] Scriptel Corporation "Scriptel: Digitizers for Pen-Based Computers", Whitepaper, Scriptel Corporation, 1992 Cordless stylus, Sensor panel, ratiometric measurement isolates from electronic interference [Scriptel92c] Scriptel Corporation "Rediscover the Power of the Pen: Scriptel PenWriter", Product brochure: Scriptel Corporation, 1992 separate touchscreen unit, includes display [Scriptel92d] Buckler, Grant "Scriptel to develop cordless stylus", Scriptel press release: Jan 8, 1992 cordless stylus for scriptel digitizer, with single-chip controller. Technology agreement with Apple Computer, Wang Freestyle. Battery-powered stylus with 12 months typical service life [Scriptel92e] Scriptel Corporation "Scriptel Corporation: Key Facts", Scriptel Corporation, 1992 (date approximate) General business information for Scriptel Corporation: "Single layer digitizer technology for pen-based computer products, making it possible to write directly on a computer screen or lap-size display tablet without using a standard keyboard" [SearsA91a] Sears, Andrew and Shneiderman, Ben "High Precision Touchscreens: Design Strategies and Comparisons with a Mouse", Int'l. Journal of Man-Machine Studies, Vol 34 No 4, April 1991 Describes advantages of touchscreens over physical buttons, switches, and other controls. Mentions takeoff (lift up) selection with touchscreens has fewer accuracy errors, compared to click-down as with mouse. [SearsA91b] Sears, Andrew "Improving touchscreen keyboards: Design issues and a comparison with other devices", Interacting with Computers, Vol 3 No 3, 1991, ppl 253-269 Physical layout of touchscreen virtual keyboard to account for targetting error, slow response of touchscreen sensor. [SearsA92a] Sears, Andrew; Revis, Doreen; Swatski, Janet; Crittenden, Rob; and Shneiderman, Ben "Investigating Touchscreen Typing: The effect of keyboard size on typing speed", Univ. of Maryland Technical Reports CS-TR-2662 / CAR-TR-553, October 1992 Touchscreen keyboards using "lift off strategy" -- drag cursor to position, activate on lift not on touch: Larger keyboard layouts easier to use. [SearsFW91a] Sears, Francis W.; Zemansky, Mark W.; and Young, Hugh D. "College Physics, Seventh Edition", Addison-Wesley, 1991 (hardcopy book) Standard physics reference and textbook [SensorFrameCorp92a] Sensor Frame Corporation "The Sensor Frame Graphic Manipulator Final Report", NASA Technical Report NASA-CR-194243 / N94-70016 (unclassified), release date 5/8/92 Optical detection of two- and three-finger gestures in both 2-Space and 3-Space, using a single CCD detector in front of a display: described a touch screen. Vendor of optical sensors cancelled them: developed VideoHarp [ShaferD91] Shafer, Dan and Ritz, Dean A. "Practical Smalltalk: Using Smalltalk/V", Springer-Verlag, New York, 1991 Introduction to Smalltalk language and environment. (Electronic file). [Sharp93a] Sharp Wizard Hotline "Sharp OZ-9600 Electronic Organizer", Sharp Wizard Hotline PDA-type product, somewhat similar to Apple Newton, showing electronic ink, stylus input. Electronic ink notes referred to as Scrapbook Writing. Also refers to OZ-8000/7000, YO-600/ZQ-5000 models, wireless infrared data transfer



[Sharp93b] sharp.co.jp "Sharp Expert Pad Model 7000 product information", Sharp.co.jp Sharp Expert, clone/licensee of Apple Newton. File contains additional information. [SharpZaurusPDA93a] Sharp Zaurus "Wikipedia: Sharp Zaurus PDA", Wikipedia, fetched 2013 Sharp Zaurus PDA, predecessor to Apple Newton [SheinE91a] Shein, E.; Hamann, G.; and Brownlow, N. "WiViK: a visual keyboard for Windows 3.0", Proceedings of the 14th Annual Conference of the Rehabilitation Engineering and Assistive Technology Society of North America (RESNA (91), RESNA Press, Washington, DC, pp. 160-162 Adaptive technology virtual keyboard, for use with any pointing device: tablet, mouse, headmouse, etc. [Shenton91] Shenton, Martha E.; Kikinis, Ron; McCarley, Robert W.; Metcalf, David; Tieman, James; and Jolesz, Ference A. "Application of automated MRI volumetric measurement techniques to the ventricular system in schizophrenics and normal controls", Schizophrenia Research, Vol 5, 1991, pp 103..113, Elsevier Science Publishers Cites 1985 Scriptel electrostatic digitizer used with DEC display for reading MRI / X-Rays [ShneidermanB91a] Shneiderman, Ben "Touchscreens now offer compelling uses", IEEE Software, Vol 8 No 2, March 1991, pp-93-94, 107 Shows ring-binder metaphor on touchscreen user interface: compare with Refalo. Mentions use of velocity and motion sensing with touchscreen to control volume, pitch, brightness, speed and other attributes/characteristics in music/graphical UI. [ShoensK93a] Shoens, K.; Luniewski, A; Schwarz, P.; Stamos, J.; and Thomas, J. "The Rufus System: Information Organization for Semi-Structured Data", Proc. 19th VLDB Conferenc,e Dublin Ireland 1993, pp 1..12 System recognizes data of interest in unformatted text, and indexes items such as proper names, words, telephone numbers and postal/ZIP codes. Methods associated with classes encapsulate actions user can take on data -- automatic generation of links/actions in text data. Highlighting of phone numbers, active documents? [SigelC92a] Sigel, Claude "Real-time recognition of pointing information from video", United States Patent 5,168,531, December 1, 1992 Optical tracking of pointing finger instead of mouse or tablet [SignOnSystems92] Sign-On Systems, Inc. "Sign-On pen computing", SignOnSystemsinc.com/pen.htm Software development tools for Windows for Pen Computing, signature verification. Pen Computing development environments: Sign-On for Fox, a FoxPro Pen Interface library for Microsoft's Widows for Pen Computing. Includes CIC signature verification library: digitizer, light-pen, or pen-based computer [SimmelSS93a] Simmel, Sergiu S. and Godard, Ivan "Metering and Licensing of Resources: Kalas General Purpose Approach", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Kala persistent data server / D.R.M. system, licensing per user and per use. Heirarchical/recursive (sublicensed) distribution of payments/royalties. [SinclairIR91a] Sinclair, Ian R. "The HarperCollins Dictionary of Computer Terms", Harper Collins, 1991 (hardcopy book) Technical dictionary of computer terms 1991 [SinghG91a] Singh, Gurminder and Green, Mark "Automating the Lexical and Syntactic Design of Graphical User Interfaces: The UofA* UIMS", ACM Trans of Graphics, Vol 10 No 3, July 1991, pp. 213-254. Also at CHI '89 UofA / University of Alberta UIMS. User can define "style sheets" or defaults for UI parameters, and adjust them? [SinghG93a] Singh, Gurminder, moderator; Linton, Mark; Myers, Brad A.; Szczur, Marti "From Research Prototypes to Usable, Useful Systems: Lessons Learned in the Trenches", Panel: UIST '93, November 3-5, 1993, pp. 139-143 Customer is not always right (specification/user requirements): if you build something useful, people will want it. Building prototypes (a useful one) can be counterproductive, often just as hard as building something real (and also useful). Usefulness of incremental development, frequent reviews (SCRUM/Agile development?) [SinghP92a] Singh, Pradeep and Weil, Lisa "Microsoft Windows for Pen Computing", Proc. IFIP 12th World Computer Congress, Madrid, September 1992, pp. 23-31 Basically an extended product announcement of Windows for Pen Computing / / Windows for Pens / PenWindows at IFIPS. USer-assignable gestures of circle "b", circle "a", etc. [Sklarew92] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 5,157,737, assigned to Grid Systems Corporation Fremont, California, October 20, 1992 Abstract refers to transparent sensing surface (tablet digitizer: touchscreen) over display, electronic ink, character recognition, and learning mode. Abstract reads almost identically to Sklarew Patent 4,972,496: this one refers to "terminal disclaimer". Claims very specific to text editing with gestures: adjustment of margins, deleting font symbols, etc. Editing symbols include brackets around text, caret "gesture", etc.



Refers to two editing symbols, where the second must be different from the first. [Slate91a] Slate Corporation "Slate Corp. has Announced PenApps", Computer Business Review, Feb 5, 1991 Slate announces PenApps IDE for pen computing [Slate91b] Slate Corporation "Grid Pen Essentials from Slate", Slate Corporation, 1991 Marketing description of Grid Convertible / Grid SL touchscreen pen computer bundle: physical case, extra stylus/pen, Slate application Day-Timer Pen Scheduler, WinFax Pen Software, PenBook e-book reader, LooseLeaf notetaker appliction (sketch input), [Slate92a] Slate Corporation "Slate PenApps Application Builder Reference Manual", Copyright 1992 by Slate Corporation (hardcopy book)(Data CD) PenBasic, ink data type, deferred translation, sketch fields; input targetting with "gravity" (non-sic) to allow writing outside of field boundaries, sketch fields: For both PenWindows and PenPoint. -- Compare "gravity" / input targetting with other systems, including graphical editors and CAD systems from 1980's. History slot and history gesture for recalling previously-entered items into a pop-up list: "Selecting is better than Writing". Software application and documentation, release 0.9 [Slate92b] Slate Corporation "Slate PenBook Publishing Pack", Copyright 1992 by Slate Corporation (hardcopy book)(Data CD) Software application and documentation: DOS Authoring utility (Creating Bookfiles) to convert PS Postscript documents to electronics books bookfiles, Windows for Pen Computing Reader. Reader gestures use flicks for page forward/back, double flick for chapter, triple flick first/last page, letters (B for Bookmark), pressand-hold for text selection. Penpoint-style tabs on right. [Slate92c] Slate Corporation "Slate At-Hand Spreadsheet: The powerful, portable, pencentric spreadsheet for PenPoint", Copyright 1992 by Slate Corporation (hardcopy book)(Data CD) Also written as @-Hand. The original PenCentric spreadsheet application announced by Dan Bricklin / Slate: makes extensive use of PenPoint and other gestures, including multiple flick gestures in the corners of cells (Dan Bricklin: Co-inventor of VisiCalc spreadsheet application). Zoom up/down gesture is up/down arrow gesture. [Slate92d] Slate Corporation "Day-Timer Pen Scheduler", Copyright 1992 by Slate Corporation Extensive use of ink editing, free-form organizer program, based on ink data type rather than text data. Included task scheduler / planner without handwriting recognition. Software application (installation media) and documentation, for PenPoint and for Windows for Pen Computing (a.k.a. PenWindows) [Slate92e] Slate Corporation "JOT - Interchange format", Slate Corporation, Scottsdale Arizona Joint Microsoft/Slate/Lotus,GO,Apple/General Magic interchange format for electronic ink data: original work initiated at Slate Corporation (Ward, Bricklin, et al). Includes encoding of pressure, angle, other data, with both lossless and lossy data reductions, delta-encoded format. [Slate92f] Slate Corporation "Slate PenApps Application Builder Developer's Guide", Copyright 1992 by Slate Corporation (hardcopy book)(Data CD) PenBasic, ink data type, deferred translation, sketch fields; input targetting with "gravity" (non-sic) to allow writing outside of field boundaries, sketch fields: For both PenWindows and PenPoint. -- Compare "gravity" / input targetting with other systems, including graphical editors and CAD systems from 1980's. History slot and history gesture for recalling previously-entered items into a pop-up list: "Selecting is better than Writing". Software application and documentation, release 0.9. File also contains installation floppies for PenApps 0.9 for Windows 3.1. [Slate92g] Slate Corporation "Slate PenApps Application Builder Quick Reference Version 1.0 for Windows for Pen Computing / Pen Windows 1.0(Data CD)", Copyright 1992 by Slate Corporation (hardcopy book) PenBasic, ink data type, deferred translation, sketch fields; input targetting with "gravity" (non-sic) to allow writing outside of field boundaries, sketch fields: For both PenWindows and PenPoint. -- Compare "gravity" / input targetting with other systems, including graphical editors and CAD systems from 1980's. History slot and history gesture for recalling previously-entered items into a pop-up list: "Selecting is better than Writing". Software application and documentation, release 1.0 [Slate92h] Slate Corporation "Slate Day-Timer Pen Scheduler Quick Reference", Copyright 1992 by Slate Corporation electronic-ink note taker and information organizer, page layouts modeled after DayTimer. Lasso gesture to select or edit electronic ink. Pigtail and X gesture for delete, etc. Zoom on scheduler calendar entry enlarges, ink can be written at smaller scale. PenWindows and PenPoint. [Slate92i] Slate Corporation "Delrina WinFax Pen Software User's Guide: Version 1.0 for Windows for Pen Computing", Copyright 1992 by DelRina Corporation (hardcopy book) Fax application with address book, extended for Pen-Windows, references to Slate Pen Scheduler [Slate92j] Slate Corporation "Slate At-Hand Spreadsheet User's Guide, Version 1.0 for PenPoint", Copyright 1992 by Slate Corporation Penpoint pen-computing spreadsheet program, associated with Dan Bricklin. Spreadsheets can be shown



in portrait and landscape views, penpoint gestures for editing data. Pop-up cell editor or keyboard for entering handwritten data into small cells. [Slate92k] Slate Corporation "Slate At-Hand Spreadsheet Quick Reference, Version 1.0 for PenPoint", Copyright 1992 by Slate Corporation Penpoint pen-computing spreadsheet program, associated with Dan Bricklin. Spreadsheets can be shown in portrait and landscape views, penpoint gestures for editing data. Pop-up cell editor or keyboard for entering handwritten data into small cells. [Slate92l] Slate Corporation "Slate At-Hand spreadsheet program designed specifically for pen-based computers", Slate Corporation, January 27, 1992 Press announcement: Slate Announces Full-featured Pencentric Spreadsheet for GO's Penpoint Operating System. Announcement of At-Hand @hand spreadsheet application for touchscreen computer running Penpoint. Included Pencentricity Backgrounder / The Pen-Based Market Arrives, gesture and input targeting (compare with Wang Freestyle), cursorless movement (direct writing into field, Ink as a Data Type, Intelligent drawing (compare Sketch recognition / sketch editing) [Slate92m] Slate Corporation "Slate Day-Timer Pen Scheduler User's Guide: Version 1.0 for PenPoint", Slate Corporation, 1992 User's guide/manual for Slate Day-Timer Pen Scheduler calendaring / note-taking / PIM application for pen/tablet computers (PenPoint: see also PenWindows version). Structured simulated paper modeled on Day-Timer pages, primarily using electronic ink (sketches, handwriting, etc.) User can zoom and write notes, zoom back and ink is shown small scale "Tiny Text". Ink notes on day pages show up in small scale on weekly, monthly pages. Deferred/delayed recognition of ink to characters. Ink in fields/forms need not be completly in displayed box. Zoom gestures are flick-up/down, scolling by scrollbars. Graphical index shows pictures/images of portions of pages. Unlimited depth undo/redo, with checkpoint/revert (checkpoint commits). Lasso gesture to select. [Slate92n] Slate Corporation "Slate Day-Timer Pen Scheduler Quick References: Version 1.0 for PenPoint", Slate Corporation, 1992 Quick Reference guid for Slate Day-Timer Pen Scheduler calendaring / note-taking / PIM application for pen/tablet computers (PenPoint: see also PenWindows version). [SmalltalkReport91a] Smalltalk Report "Momenta Demonstration", Smalltalk Report, November/December 1991 Report on demonstration of Momenta pen computer with command compass / pie-menus / marking menus: development language is object-based Smalltalk with objects shared between applications, running on DOS operating system [SmalltalkReport91b] Smalltalk Report "WindowBuilder: An interface builder for Smalltalk/V Windows", Smalltalk Report, September 1991 Platform-independent GUI builder in SmallTalk [SmithR91a] Smith, Randall "The Alternate Reality Kit: User Interface Overview", CHI 91 Video Presentation. Copyright at start of video is 1987. Shows Alternate Reality Kit with simulation of physical objects, 2-D gravity. Also shows virtual switches, slide switches, slider switches for changing simulation characteristics. See also Sutherland "The Ultimate Display" simulation of physics. [SmootL92] Smoot, Lanny S. "Light-pen System for Projected Images", United States Patent 5,115,230, May 19, 1992 Projection whiteboard with light-pen: calibration method? Quadrilateral correction? Parallax correction? Compare with optical digitizers. NOTE: No known relation to Daniel R. Smoot of National Bureau of Standards [SoftwareIndustryReport91] SoftwareIndustryReport "Microsoft turns screws on tiny Go Corp. in fight for pen software market", Software Industry Report, Millin Publishing, March 4, 1991 Refers to Microsoft lining up many manufactures to lure them away from GO/PenPoint [SoftwareIndustryReport92] SoftwareIndustryReport "Japanese PenPoint gets broad initial support", Software Industry Report, Millin Publishing, September 7, 1992. Available at: www.amazon.com, archived digital document (partical copy) Multiple hardware vendors in Japan for GO/PenPoint systems, multiple application vendors doing Japanese versions of PenPoint tablet applications. [Sony92a] Sony Corporation "Sony PalmTop product information", Sony Corporation References, Product information on Sony PalmTop handwriting-recognition / touchscreen (stylus) portable computer. [SovieroMM92a] Soviero, Marcell M. "Your world according to Newton", Popular Science, September 1992, pp 4549 Newton PDA: public article about prototype. Calendar: touch a date, get exploded/zoomed information view about that date. Describes Newton automatically learning text phrases that may be embedded/trail an input, such as "fax this" or "format this". Scribble/scratch-out gesture erasing part of a drawing. Palm-top



computers include HP 95LX. Sony OZ-9600 and PV-F1 Action Manager, Sony PTC-300 PalmTop pen-input hand-held machines. Editorial introduction credits John Sculley at Apple for Knowledge Navigator (Newton? ), Ted Nelson for Project Xanadu global database (Google?), and Alan Kay for Dynabook (Newton? -- no handwriting recognition actually) [SpeeterTH93a] Speeter, Thomas H. "Apparatus for dynamically varying the resolution of a tactile sensor array", United States Patent 5,237,879, August 24, 1993 Touchpad/touch sensor array matrix: speed up sampling by coupling adjacent lines is pairs, so 1/2 the resolution, 1/4 the scanning time. Compare with course/fine measurements in HP digitizers. [SpraguePJ93a] Sprague, PeterJ. and Lipscomb, Thomas H. "Information distribution system", United States Patent 5,247,575, September 21, 1993 Digital rights management DRM system using encryption keys to access content: keys changed periodically. General information about package/document in header data. Usage charges stored in client device between times. Information packages can be broadcast (digital broadcast distribution: see TVBS), user/client can store packages of interest and get licenses. Compare with Glen Benson e-license. [StaplesL93a] Staples, Lorreta "Representation in Virtual Space: Visual Convention in the Graphical User Interface", Proc. ACM InterCHI '93, April 1993, pp. 348-354, 542 Semi-transparent / translucent windows, can be placed over other windows, user can see what is below. Mostly about designing GUIs user interfaces to look like analog / real-world mode interfaces (e.g., CD player controls), so that user knows how to use the interface. Degree of transparency under user control, can be done in any window. Shows 3D perspective of displayed document objects, some in background (compare with General Magic?). [StearnsRG93a] Stearns, Richard G. "Capacitive Tactile Sensor Array and Method for Sensing Pressure with the Array", United States Patent 5,225,959, July 6, 1993 Capacitive grid multi-touch touchscreen/tablet using small capacitive sensors and elastomeric dialectric: large area possible, integration with flat displays. Capacitors are between horizontal and vertical conductor crossing point, deformable elastomeric dielectric between. Pressing on top moves capacitor plates closer together, increasing capacitance. [StifelmanLJ93a] Stifelman, Lisa J.; Arons, Barry; Schmandt, Chris; and Hulteen, Eric A. "VoiceNotes: A Speech Interface for a Hand-Held Voice Notetaker", Proc. InterCHI '93, April 1993, pp. 179-186 Recorder for short voice notes to self, prototype tethered to Mac PC (not to a Newton). Prototype for computer with neither keyboard nor display (nor touchscreen input?). Notes organized into categories: Todo list, calls, appointments, etc. [StraussPS92a] Strauss, Paul S. and Carey, Rikk "An Object-Oriented 3D graphics Toolkit", Computer Graphics, Vol 26, No 2, July 1992, pp 341-349 Direct manipulation toolkit for 3D scenes: picking (mouse click) along a ray to determine object pointed to. Trackball operation (rotate) mode for mouse input. [SturmanDJ92a] Sturman, David J. "Whole hand input", Ph.D. Thesis, MIT, 1992 (Copyright 1991) Whole-hand UI user interface input using VPL data-glove: direct manipulation experiment using multiple fingers and hand position to control six-legged walking device, whole-hand and hand-motion gestures. Thumb-based gestures for throttle. Distinguishes between gesture (with motion shape) and posture (static gesture). VPL DataGlove includes a gesture editor for defining gestures. [Swaine91] Swaine, Michael "A Conversation with Robert Carr", Dr. Dobb's Journal, 1991 Robert Carr, head of development at GO/Penpoint: mostly a review of the OS design, file system, objectlike message passing on PenPoint gesture GUI [SzekelyP92a] Szekely, Pedro; Luo, Ping; and Neches, Robert "Facilitating the Exploration of Interface Design Alternatives: The HUMANOID model of interface design", Proc. of SIGCHI '92, May 1992, pp. 507-515 Graphical User-interface design system, references to slots, objects, etc. Compare with Visual Basic. Humanoid92 shows development system with slots and widgets for objects. Refers to input gestures, mentioning direct-manipulation, mouse-based gestures: e.g. dragging gesture. [SzekelyP93a] Szekely, Pedro "User Interface Prototyping: Tools and Techniques", Proc. of INTERCHI'93 UI design tools: Paper and Pencil, Facade (UI with no application behind it), mentions NeXT Step, Prototype for the Macintosh, and WindowsMAKER by BlueSky. Humanoid92 shows development system with slots and widgets for objects. [TakafumiI92a] Takafumi, Ito; Hiroyuki, Sakamoto; and Yutaka, Kamiyama "Display/input control system for software keyboard in information processing apparatus having integral display/input device", European Patent Application EP-464712A2, 1992 On-screen movable virtual keyboard. Press-and-hold time on touchscreen key can auto-repeat key (or shift case?). Virtual keyboard automatically positioned so as not to cover up current input field in a form -- user can then move to another location. [Takahashi91] Takahashi, Toimoichi, and Kishino, Fumio "Hand Gesture Coding Based on Experiments Using a



Hand Gesture Interface Device", SIGCHI Bulletin, Vol 23 No 2, April 1991, pp 67-74 DataGlove product used to read hand alphabet gestures for Japanese Kana, similar to American manual alphabets for the deaf (finger spelling?) [TanakaD92] Tanaka, David "Pen-based systems - computer innovations", Computer Dealer News, November 30, 1992 Review of pen computing 1992, distinguishes between pencentric and pen-aware approaches. Cites market studies of expected large sales. DayTimer PenScheduler from Slate, LetterExpress from PenMagic. Quotes Portia Isaacson on problems with digitizer technologies. [TangJC91a] Tang, John C. and Minneman, Scott "VideoWhiteboard: video shadows to support remote collaboration", Proc. CHI '91 SIGCHI Conference on Human Factors in Computing Systems, pp. 315-322 Whiteboard system using TV/video cameras and semi-transparent whiteboard (no digitizer): users can see enough of shadow of counterpart user to recognize hand and body gestures effectively. Cites to 1979 paper. [TangJC93a] Tang, John C.; Minneman, Scott L.; Bly, Sara A.; and Harrison, Steve R. "Video Computational Shared Drawing Space", United States Patent 5,239,373, August 24, 1993 Shared drawing space/whiteboard system, including sound and audible communications. Compare to realtime Freestyle? Mentions in passing use of transparent electromagnetic digitizer/tablet; other citations to video tracking of stylus, resistive tablet. [Tannenbaum93] Tannbenbaum, Alan R.; Zetts, John M.; An, Yu L.; Arbeitman, Gordon W.; Greanias, Evon C; and Verrier, Guy F. "Graphical User Interface with Gesture Recognition in a Multiapplication Environment", United States Patent 5,252,951, October 12, 1993 Gesture/voice/speech input with multiple user profiles for different commands to different programs Circle gesture mentioned Translation of gestures to simulated input for other, pre-exiting devices, such as keyboard and/or mouse, to maintain compatibility with pre-existing applications. Compare to Pencept CAD and other interfaces from 1985, which translated "recognition macro" gestures to defined sequences of keyboard/mouse input Recognition of gestures/voice/speech with hot spot coordinates and hot spot window, mapping to macros. If gesture not assigned meaning, user may define on the fly [Tappert91] Tappert, C.C. "Speed, Accuracy, Flexibility Trade-Offs in On-Line Character Recognition", Int. J. Pattern Recognition and Artificial Intelligence, Vol. 5, pp. 79-95, June 1991, also in Character and Handwriting Recognition, P.S.P. Wang (Ed.), World Scientific, 1991, also see IBM Research Report RC13228, October 1987 Cited by Marlin Eller, Microsoft Pen Computing group [TappertCC93a] Tappert, C.C. and Ward, Jean Renard "Pen Computing: Fad or Revolution?", Information Display, Vol 3 no 92, pp. 14-19 Description of Pen Computing circa 1993 for SID Society for Information Display. Description of electronic ink. [TaysiBM92a] Taysi, Burak M. "Gesture System for a Graph Editor", Master of Science thesis, Washington University 1992, St. Louis Missouri. www.cs.wustl.edu/cs/techreports/1992/wucs-92-35.part2.ps.Z Design of aparticular (trainable) gesture recognition user interface UI. General review of gesture commands and state of the art circa 1992, numerous references. Cites examples of proofreader marks, paper-like interface, silicon paper . Mentions Kumon pen computer for children (1992), NeXT tablet interface platform. Comments on weak gestures of Pen Windows, contrast with PenPoint. Examples of modified "arrow" gestures, alignment (sideways "H"), others. Table of gestures for Windows for Pen Computing and for PenPoint (Lombard): graphics missing in printed copy [TelepresenceNews92a] Telepresence News "Telepresence News", Telepresense News newsletter, fetched 2011 from www.dgp.toronto.edu/tp/news/news04.html Cited by Buxton92a on Active Desk project, remote desktop/whiteboard [Tognazzini92a] Tognazzini, Bruce "Tog on Interface", Addison-Wesley, 1992 (hardcopy book) User-interface essays by Bruce Tognazzini. Some mention of digitizers: Data Glove, Polhemus 3-D digitizer (actually it was 6-D, it includes the angles of orientation of the stylus). No mention of handwriting recognition or gesture, or of the Newton, but does mention direct manipulation. User-selectable scrolling speed parameters. [Tognazzini92b] Tognazzini, Bruce "StarFire: The Director's Cut", www.asktog.com, video fetched 2010 StarFire video showing anticipated developments in user interfaces and ubiquitous computing, circa 1992. Shows two-finger (multi touch) stretch/zoom gesture, tablet computer with position and geographic sensing (GPS). Also shows two-finger hold-and-move gesture to make a copy: running time 2:07, twofinger select/copy/paste gesture (off-screen) at 5:09. Flick gesture to delete at 7:45 [Tognazzini92c] Tognazzini, Bruce "StarFire: The Making Of", www.asktog.com, video fetched 2012 Bruce Tognazzini explaining background of StarFire video, two-year human factors and computer science effort. [Tognazzini92d] Tognazzini, Bruce "StarFire: The Annotated Script", www.asktog.com, fetched 2012



Annotated script for StarFire, including "pinch" gesture. Excerpted from Tognazzini's book. Also shows multi-touch. [Toshiba93a] Toshiba Corp. "Dynapad T100X Pen Computer", Dynapad PenWindows, Toshiba Corp., 1993 (physical device) Physical device (functional). Electronic file also contains sales brochure and additional information (press releases, hardware specifications, video links, etc.) for Toshiba Dynapad T100-X Dynapad pen computer, running either PenPoint OS or PenWindows / Windows 3.1 with pen extensions. [ToshimitsuS92a] Toshimitsu, Salto; Mayumi, Oka; Atsuhiro, Ootake; and Mason, James "Personal computer capable of altering display luminance through key operation", European Patent Application 0 454 552 A2, January 8, 1992 Portable computer, adjust luminance/brightness/contrast from key on keyboard [TriTech92] TriTech Microelectronics International, Itd. "PICK: Pen Interface Controller Kit", TriTech Microelectronics International PTE Ltd, Singapore Scriptel electronics chip set for electrostatic/capacitive digitizer for pen computing [Tsuyama91] Tsuyama, Chizuka and Asagaya-minami, Signami-ku "System for Use in Processing a Speech by the Use of Stenographs", United States Patent 4,985,929, January 15, 1991 Speech is converted to a set of stenographs: appears to be a claim of an automatic speech-recognition voice typewriter Mentions use in automatic transcriptions and records of proceedings, international conferences (automatic translation?) Conversion of speech to phonetic letters or symbols, such as Pitman shorthand, based on statistical methods: up, down, and curved strokes [TygarJD93a] Tygar, J.D. and Yee, Bennet "Dyad: A System for Using Physically Secure Coprocessors", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Description of Dyad system at CMD Carnegie-Mellon University. DRM, physically-secured processors not possible with personal computers and workstations, defines "secure processor", deficiencies of trusted file servers, audit trailes, electronic currency. User authentication via smart cards. Mentions "electronic currency" without central servers: compare with BitCoin? [UboisJ92a] Ubois, Jeff "Pen-based PIMs coming with Pensoft Perspective (personal information manager)", MacWEEK, February 17, 1992 Pensoft Perspective and Slate PIM: mentions Rosetta pen-computing/handwriting extensions to Apple Mac, to follow release of Newton. Phone number and contact management. [UngarD91] Ungar, David and Smith, Randall B. "Self: The Power of Simplicity", Lisp and Symbolic Computation: An International Journal, 4, 3, 1991 Substantial revision of 1987 reference, same title and authors. [UpthegroveL93a] Upthegrove, Luella and Roberts, Tom "Intellectual Property Header Descriptors: A Dynamic Approach", Two day workshop April 2..3 1993, Coalition for Networked Information, et al M.I.T. Published 1994 in "Technological Strategies for Protecting Intellectual Property in the Networked Multimedia Environment, Jnl. of Interactive Multimedia Association Intellectual Property Project, January 1994 Description of DRM system LCS Library Collections Services Project at CWRU Case Western Reserve System library for electronic distribution of intellecutal properties. Relies on "compliant applications" that enforce the DRM architecture: ownership, permitted uses, royalty compensation, authentication of users. [UttS93a] Utt, Steven "Re-thinking the Pen-on_Display User Interface", Stanford University Program in HumanComputer Interaction, Seminar on People, Computers, and Design, April 14, 1993 (Abstract only) Presentation by UI designer from Momenta Corp. which used marking menus / pie menus for pencomputing UI. States that NewCAD partners build UI and handwriting recognition software sold to Momenta Corporation in 1989. [Vetusware92a] Vetusware.com "Windows for Pen computing 1.0", Vetusware.com, "the biggest free abandonware downloads collection in the universe" Installation files (date 1992) for PenWindows 1.0. RAR format. [VogtFH92a] Vogt, Friedrich H. ed. "IFIP Transactiona: Personal Computers and Intelligent Systems", IFIP Transacation A-14, Vol 3, Proc. IFIP 12th World Computer Congress, Madrid Spain, September 7-11, 1992 Contains Singh article on Windows for Pen Computing. [WantR92a] Want, Roy and Hopper, Andy "Active Badges and Personal Interactive Computing Objects", IEEE Trans. on Cons. Electronics, Vol 38 No 1, February 1992, pp. 10-20 Investigation of portable (palm-sized) computing devices. Later work on Xerox Parc Active badge touchscreen hand-held [WardJR91a] Ward, Jean Renard "Pen-Based Products", correspondence in "ACM Forum", Communications of the Association for Computing Machinery, Vol 34 No 12, December 1991, pp 18-19 Reply to Larry Press at ComDex trade show: that Linus was not the first pen computing company or



portable tablet product. [WardJR92a] Ward, Jean Renard and Bricklin, Dan "The History of Pen-Based Computing - March 1992", Video of presentation at Boston Computer Society / Slate Corporation, March 1992. Available at http://www.youtube.com/watch?v=4xnqKdWMa_8 Slate Corporation video of repeat of presentation to Boston Computer Society on history of pen and touch computing, going back to 1914. Rand Tablet, Pencept, GRAIL, etc. Does not mention Elisha Grey Telautograph. File also has copy of published outline of talk from 1996/1999 "Notes on the (relatively unknown) History of pen-based computer". [WardJR93a] Ward, Jean Renard and Schultz, Debra "Under the Hood: Digitizer Renaissance", BYTE Magazine, January 1993 Article on pen computing (PenPoint, Pen Windows, etc.) making digitizers and pen input (electronic ink, gesture commands, annotations, written/recognized characters) setting relevance of digitizer technology. distinguishes (at that time) digitizers from touchscreens, as digitizers have the performance to capture handwriting. Hand/palm rejection a problem of touchscreens, not digitizers. Description of digitizer/touchscreen technologies: electromagnetic, capacitive, resistive, light pen. [WattersC92a] Watters, Carolyn "Dictionary of Information Science and Technology", Academic Press, 1992 (hardcopy book) Technical dictionary [WeberG93a] Weber, Gerhard "Adapting direct manipulation for blind users", Proc. CHI/INTERCHI '93, Conf. companion proceedings pp. 21-22 Mouse routing / cursor routing pointing technique for blind computer users on tablet or touchscreen. User can explor display with light touch (with screen reader), press touch to start mouse routing: move mouse to current location. [WeimanL92a] Weiman, Liza and Moran, Tom "Newton: A Step toward the Future", MacWorld, August 1992, pp. 129-131 Announcement of Newton: mentions phone dialer, user highlights phone number in text in display. Mentions handwriting recognition / pen-based X-Windows systems work by Xerox Imaging Systems. Also Tusk Super All-Terrain (ruggedized), TriCom Pen3865x, Poqet PoqetPad, PI systems Infolio, NEC UltraLite, NCR 3125, Momenta Pentop, Microslate Datalite, IBM ThinkPad (PenPoint), Grid PalmPad and PenRight, Linus WriteTop Grid GRIDPAD, DFM TraveLite, Data Entry Systems Scriptwriter. Cites to CIC but not to PenCept. [WeimerC93a] Weimer, Clay "I am a Walking Office", USENET posting September 1, 1993, available 2014 via Google Groups, Wanderbook.com Blog posting on use of EO/GO PenPoint pen computer: automatic dialing via "D" gesture on name in a document, usable while standing up (versus laptop) or in a lounge chair. [WeiserM91a] Weiser, Mark "The Computer for the 21st Century", Scientific American, Vol. 265, No. 3 (September 199l), pp. 94-104. Republished in Mobile Computing and Communications Review, Vol. 3, No.3, pp. 3-11, 1999; also Baecker95a Towards the Year 2000 Review of tablet, whiteboard, and other prototypes for Ubiquitous Computing at Xerox PARC up to 1991. Live boards, wireless page-size Pad computer with multibutton stylus, handheld/PDA tab computer, pen user interface. Mentions video in portable device [Wellner91a] Wellner, Pierre "The DigitalDesk Calculator: Tangible Manipulation on a Desk Top Display", Proceedings of UIST '91, November 11-13, 1991, pp. 27-33 Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Refers to tangible manipulation because user is moving both electronic desktop and physical paper documents, rather than direct manipulation with a mouse. Compare with later Microsoft Surface. Cites to Krueger83 for Videodesk, and Mandala. Image differencing and object location at different scales to track finger pointing: optimized algorithm for subtracting images in real-time (approx 7 frame/second). Tap gesture feedback of actual input by displaying indicated number in projected calculator: refers to "tangible feedback", so second kind of feedback? [Wellner91b] Wellner, Pierre "The Digital Desk", Video demonstration, 1991 Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Refers to tangible manipulation because user is moving both electronic desktop and physical paper documents, rather than direct manipulation with a mouse. Shows two-finger zoom/select and pan gestures etc. at running time 03:27. Compare with later Microsoft Surface. Shows multi-touch gestures. [Wellner93a] Wellner, Pierre "Interacting with Paper on the DigitalDesk", Communications of the ACM, July 1993. Also EuroPARC Tech Report EPC-93-195 Mentions Freestyle, and asserts that people "like" paper compared to electronic documents. Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Mostly a userinterface experiment in combining electronic simulation with real paper documents. Four-point calibration



to correct for trapezoidal/projection distortion. Shows multi-touch gestures, e.g. two index finger selection to draw two corners of a selection rectangle. [Wellner93b] Wellner, Pierre D. "Self Calibration for the DigitalDesk", Rank Xerox Research Center, Technical Report EPC-1993-109 Digital camera over desk, functionally similar to digitizing tablet, describes calibration: quadrilateral, fourpoint warping, keystoning, translation, scaling, shear, non-uniform scaling, bending distortions and calibration. [Wellner93c] Wellner, Pierre and Freeman, Steve "The Double Digital Desk: Shared Editing of Paper Documents", Rank Xerox Research Center, Technical Report EPC-1993-108 Two user groupware, each at station with Digital camera over desk, functionally similar to digitizing tablet. Shared remote editing worked better for document layout than document text. Compare with later whiteboard systems. [Wellner93d] Wellner, Pierre D. "Adaptive Thresholding for the DigitalDesk", Rank Xerox Research Center, Technical Report EPC-1993-110 Scanning of static images with uneven lighting in optical tabletop system (Digital Desk): single quantization threhold does not work well on printed documents/surface with uneven lighting (mostly on one side), suggests scanning line line at a time and determing local area threholds. [Wikipedia93a] en.Wikipedia.org "NewtonScript", en.wikipedia.org/wiki/Dylan_programming_language: reference fetched in 2009 Brief history of NewtonScript, prototype-based inheritance language with persistent objects. Mentions that packages in Newton correspond to Snapshots (memory dumps?) of SELF: other languages include Lua, Smalltalk, and Lisp [Wikipedia93b] en.Wikipedia.org "Newton (platform)", en.wikipedia.org: reference fetched in 2009 Brief history of Newton handwriting PDA: Newton Toolkit, handwriting applications. [WobberE93a] Wobber, Edward; Abadi; Martin; Birrell, Andrew; and Lampson, Butler "Access Control Subsystem and Method for Distributed Computer System using Locally Cached Authentication Credentials", United States Patent 5,235,642, August 10, 1993 in DRM system, identifying requesting systems by cached table of unique encryption keys: concerns that PKI slower than shared-key authentication [WolfCG92a] Wolf, Cathering G.; Rhyne, James R.; and Brigge, Laura K. "Communication and Information Retrieval with a Pen-based Meeting Support Tool", Proc. CSCW '92, pp. 322-329 We-Met, pen-based whiteboard tool for meetings, mentions in passing indexing of electronic ink notes using ink, compare with Slate Day-Timer electronic ink note-taking tool? [WolfeA91] Wolfe, Andrew L. and Barrett, Gary L. "Edge Linearization Device for a Contact Input System", United States Patent 5,041,701, August 20, 1991 Correction of edge distortion in resistive-film digitizer using adjustment resistances / longer leads in each contact at the edges [WooTYC93a] Woo, Thomas Y.C. and Lam, Simon S. "A Framework for Distributed Authorization", 1st Conf. on Computer and Communications Security '93, November 1993, pp. 112..119 Language GACL for distributed authorization system for access control lists ACL. Expressive syntax for access authorizations: negative access authorization as well as positive, inheritance, closure incomplete authorizations (access cannot be determined) return failure. [WymanRM93a] Wyman, Robert M. "Filters in License Management System", United States Patent 5,260,999, November 9, 1993 Access rights controlled through licenses: application (wrapper) presents license and desired access to license server, which checks rights. Includes permission request to call function on another server/processor. Capability systems. [WymanRM93b] Wyman, Robert M. "Management interface for license management system", United States Patent 5,204,897, April 20, 1993 Access rights controlled through licenses: application (wrapper) presents license and desired access to license server, which checks rights. Includes permission request to call function on another server/processor. Capability systems. [Xerox92a] Xerox Corporation "The Xerox PARCTAB", Available from BillBuxton.com "The Buxton Collection", fetched 2013 Xerox PARCTAB palm-sized mobile computer / PDA, worn like a pager. Three function buttons and stylus user interface on display touchscreen. [YamanamiT93] Yamanami, Tsuguya; Senda, Toshiaki; and Murakami, Azuma "Coordinates Input System", United States Patent Re. 37,187, March 2, 1993 (Reissued 4,786,765, November 22, 1988) Electromagnetic Wacom tablet digitizer with cordless stylus [YoshimuraI91a] Yoshimura, I. and Yoshimura, M. "Off-Line Writer Verification Using Ordinary Characters as the



Object", Pattern Recognition, Vol 24 No 9, September 1991, pp 909-915 Writer (not just signature) recognition in Japanese Kanji writing: OCR skeletonization based: claims 99.5% accuracy: uses arcs and angles/vectors [ZandenBV91a] Zanden, Brad Vander and Myers, Brad A. "The Lapidary Graphical Interface Design Tool", Proc. SIGCHI 91, pp. 465-466 in Garnet system papers: GUI design tool that allows for designer to make a change to a prototype object and apply it as a style change to other widgets. [ZhaiS93a] Zhai, Shumin; Milgram, Paul; Drascic, David "An evaluation of four 6 degree-of-freedom input techniques", Proc. CHI '93 / INTERACT '93, pp. 123-125 Comparison of 6-DOF 3D Spaceball trackball, 6-DOF Ascension Bird (3D sensors similar to Polhemus) on glove, using trackball button as a clutch to disengage. Test bed was MITS (Manipulation in Three Space) virtual environment [ZhaoR93] Zhao, R. "Incremental Recognition in Gesture-Based and Syntax-Directed Diagram Editors", Proceedings of INTERCHI, April 1993, Amsterdam, pp 95-11 Diagram editing / sketch recognition with gesture/gestural user-interface for direct-manipulation, modeless interaction. Cooperative, multi-level model for diagram parsing (context information). States number of strokes varies in sketching. Notes that low-level recognizer (no context) appears to be immediate to user, but high-level recognizer (with context) has some noticable interaction delays. biblio90 (Prev page)

Main Page







Main Page


[ATT95a] A.T. & T. "ATT Video of ATT Personal Communicator Videon", AT&T 1995, fetched from YouTube (URL on file) Vision of future technology. Shows personal hand-held video communication (similar to Skype) with pen touchscreen interface. [AbrahamRI95] Abraham, Robert I; Moore, Richard E.; Rich, William L.; Shackelford, Floyd W.; Tiller, John R. Jr.; and Briggs, Richard S. Jr. "Slot Objects for an Object Oriented Programming System", United States Patent 5,410,702, April 25, 1995 Implementation of a slot object: citations back to 1982 [ActiveDesk95a] Buxton, Bill "The Active Desk and Prototyping the Future", Alias/Wavefront, 1995, Toronto Canada. Link to video on www.buxton.com (2011) Active Desk whiteboard / collaborative drawing space prototype using large Scriptel digitizer tablet and rear-projection display. Compare with Microsoft Surface, Digital Desk [AgarwalLK94a] Agarwal, Lalit K. "A System for Supporting Mark-based Interaction in Motif", Brown University, Dept. of Comp. Sci. report CS-94-M15, Master's thesis, May 1994 Add gestures (marks) to point-and-click direct manipulation Motif widgets without modifying Motif implementation: filter recognizeds gestures, callback hooks into mouse events, calls existing API function in widget. [Agulnick94] Agulnick, Todd; Carr, Robert; Hoeber, Tony; Kaplan, S. Jerrold; Low, David R. and Ouye, Michael "Control of a computer through a position-sensed stylus", United States Patent 5,347,295 assigned to GO Corp, Foster city, California, September 13, 1994 Describes using a pen stylus for entering commands via character recognition / gestures into a computer, recognizing and executing command after out-of-proximity/end-of-stroke/time-out. Compare with Ledeen, which did not have gestures made on the display. Specific reference to first, second and third gestures in a sequence for a compound gesture (multiple flicks, multiple taps, etc.). Specific reference to direction of motion of the gesture. Animated marquee on selection after press-and-hold. Describes feedback to user (idealized form of gesture displayed, or question mark in a circle) on what gesture recognized. If gesture



not appropriate in context, error returned (displayed). Proximity sensing for termination of gesture, highlighting of active objects (compare with tooltips) [AhlbergC94a] Ahlberg, Christopher and Shneiderman, Ben "The Alphaslider: A Compact and Rapid Selector", Proc. CHI 94, April, 1994, Boston, pp. 365..371 Slider/selector/scrollbar for scrolling through long alphanumeric sorted lists (e.g. rolodex or dictionary): variable-speed scrolling (acceleration) if slider thumb moved more quickly. Acceleration not work well, especially since user could just tap on alphabetical index (linear alphabetically) next to scroll bar. Also micrometer (fine positioning) mode by moving mouse perpendicularly to scroll motion: confusing to users. [AhmadS95a] Ahmad, Subutai "A Usable Real-Time 3D Hand Tracker", Proc. 28th Asilomar Conf. on Signals, Systems, and Computers, 1995 Optical (camera) hand tracking system for smart desk: track hand, multiple fingertips, finger joints. [AllardDJ95a] Allard, David J.; Beatty, Brent A.; Canova, Francis J. Jr.; DeBauche, Bradley J.; Hsieh, Danile M.; Johnson, Debra A.G., Lanier, Charles S.; Lewis, James R.; Tiller, Byron K.; Villafana, Willam; and Yee, Raymond L. "Personal Communicator Having Improved Contrast Control for a Liquid Crystal Touch Sensitive Display", United States Patent 5,422,656, June 6, 1995 IBM Simon: up/down physical buttons for volume on cell phone switched to contrast-adjustment mode either by pressing both buttons together, or tapping on a particular spot on the touchscreen. [AllardDJ95b] Allard, David J.; Beatty, Brent A.; Canova, Francis J. Jr.; DeBauche, Bradley J.; Hsieh, Danile M.; Johnson, Debra A.G., Lanier, Charles S.; Lewis, James R.; Tiller, Byron K.; Villafana, Willam; and Yee, Raymond L. "Personal communicator having a touch sensitive control panel", European Patent Application 94307982.2, EP0651544A2, May 3, 1995 IBM Simon: cellphone/smartphone with touchscreen, can show screens in both landscape and portrait orientation based on user on-screen rotation button. [AndersonPT95a] Anderson, Peter T. "Pulsed-DC position and orientation measurement system", United States Patent 5,453,686, September 26, 1995 Polhemus 3D digitizer/tablet position sensor: uses pulsed DC magnetic fields, so is affected by nearby metallic or ferrous objects. Compare with Ascension Techology? [AndrewC94a] Andrew, Christopher and Bloomfield, Marc. A. "Palette Manager in a Graphical User Interface Computer System", United States Patent 5,371,844, December 6, 1994 Select new color from a palette, drag onto a UI element to change its color in OS/2. Dynamic themes/scheme manager via GUI. File also contains EP published patent application 0 591 571 A1. [Apple94a] Apple Computer, Inc. "Apple MacIntosh Tablet Prototype", Apple Computer, 1994 (date approximate): YouTube posting 2007 Video of Apple TabletBook, prototype Apple MacIntosh slate-format tablet computer, touchscreen on display. [Apple94b] Apple Computer, Inc. "Inside MacIntosh - Imaging with QuickDraw", Apple Computer, 1994 Programming reference for QuickDraw graphic on MacIntosh. Regions or maskBits used for masking -- any number of closed regions defined in drawing coordinates vs. bitmap coordinates. modes (RasterOp) on copyBits (BitBlt) include SrcOr, srcXor, srcCopy. Automatic dithering and highlight (background color replacement) in modes. Modes blend (average), sum (addPin, addOver) [Apple95a] Apple Computer, Inc. "Apple Outlines Newton Wireless Comm Solutions", Apple Computer press release, January 30, 1995 Apple Newton support for wireless communications, not on a single-vendor network [Apple95b] Apple Computer, Inc. "Macintosh Human Interface Guidelines", Apple Computer, 1995 User-interface guidlines for Apple Macintosh: compare with similar publications for Microsoft Windows. Mentions stylus pen (tablet) as equivalent pointing input device. Pointers/cursors for scroll bars and controls (arrow), drawing and re-sizing (crosshairs), selecting and inserting text (I-beam), selecting fields in spreadsheet/array (plus sign), wait (wristwatch clock). Pressing == press and hold? References to academic UI literature. Cites to Buxton, Schneiderman for direct manipulation. Also Cites MinskyMR84 (gestures). [AppleGazette95a] Apple Gazette "Apple Prototypes: 5 Products We NeverSaw -- Apple PDA Videophone", AppleGazette.com, fetched 2014 Apple prototype products, publicly known but not marketed: W.A.L.T. Wizzy Active Lifestyle Telephone with stylus -- compare with IBM Simon (also developed with BellSouth); Apple videophone PDA; Applel PenLite / Macintosh PowerBook Duo Tablet computer (full tablet/touchscreen computer, not PDA) [AppleNewton94a] Apple "Welcome to Newton (promotional video)", Apple computer, Inc. 1994 Video demonstration of Apple Newton user interface, including "heavy mark" gesture. Video in electronic files: transcript fetched from guidebookgallery.org 2010. Text file has complete transcription of text. Zigzag gesture to rubout (gesture). Copy-and-paste by moving text to edge of (small) screen. press-and-hold gesture on Newton "heavy mark", with highlight. Press-and-hold highlight is to move text (12:50),



different drag and tap-drag gestures for move and for copy. At 12:20 shows rub-out gesture recognized in inking/writing mode. 12:40 pop-up virtual keyboard. [AppleNewton94b] Apple "Apple Newton Connection 2.0 Update", Apple computer, Inc. 1994 File transfer / connection utility between Apple Newton and Apple MacIntosh. Also calendar Date Book synchronization. Network connection? Parallel cable? [AppleNewton95a] Apple "Newton: Apple MessagePad Handbook, 1995", Apple computer, Inc. 1995 Apple Newton: describes various gestures and UI features, including "heavy mark" selection gesture: press-and-hold in text? Describes rotate button to change screen orientation [AppleNewton95b] Apple "Newton Technology An overview of a new technology from Apple", Apple computer, Inc. 1995 Apple Newton: whitepaper about product concepts for Apple Newton: Apple Newton fax/phone (desktop) with notetaking, Newton Map, fold-out three panel touchscreen pen computer, Newton touchscreen wristwatch (larger panel -- wrist-mounted computer), Newton classroom whiteboard, Newton family message center (Message pad hanging on refrigerator) [ArefW95a] Aref, Walid; Barbara, Daniel; and Vallabhaneni, Padmavathi "The Handwritten Trie: Indexing Electronic Ink", Proc. ACM SIGMOD '95, San Jose, CA, 1995, pp. 151-162 Indexing (tree based) of electronic-ink names of objects by applying hidden Markov Models HMM to match the handwritten characters one at a time, take best match. Matching handles mix of hand-printing and cursive forms of a name. Compare HMM with actual recognition? [ArefWG95b] Aref, Walid G.; Kamel, Ibrahim; and LoPresti, Daniel P. "On Handling Electronic Ink", ACM Comput]ng Surveys, Vol. 27, No, 4, December 1995, pp. 564-567 Automatic indexing of electronic ink notes doing approximate ink matching (AIM) instead of recognition. One application is indexing of multimedia objects. [ArvoJ95a] Arvo, James ed. "Graphics Gems II", Cornell AP Professional Press, 1995 Second volume in "Graphic Gems", collection of papers: 2D Geometry and Algorithms, Image Processing, Frame Buffer Techniques, 3-D Geometry and Algorithms, Ray Tracing (transparent or translucent objects), Radiosity, Matrix Techniques, Numerical and Programming Techniques, Curves and Surfaces [Baecker95a] Baecker, Ronald M.; Grudin, J.; Buxton, W.A.S. and Greenberg, S. "Readings in Human-computer Interaction: Toward the Year 2000, Second Edition", Morgan Kaufmann Publishers, 1995 (hardcopy book) Cited in MyersB96: ITS from IBM Research as automatic generation/generator tool for user interfaces. Contains Case study C: Wang Freestyle, pen-based voice/electronic-ink annotation system. Describes substantial public interest and acclaim when Freestyle released in 1988 (citations) See individual articles [BarlowJP94a] Barlow, John Perry "The Economy of Ideas: A framework for patents and copyrights in the Digital Age. (Everything you know about intellectual property is wrong.)", Wired Magazine, March 1994 Essay on legal, cultural, and other (non-technical) aspects of copyright protection and intellectual property, in the contest of D.R.M. digital rights management. [BartlettJF94a] Bartlett, Joel F. "Computer System and Method for Displaying Images with Superimposed Partially Transparent Menus", United States Patent 5,283,560, February 1, 1994 Transparent menus/windows in graphical user interface, does not block view of elements on screen underneath [BatesCL94a] Bates, C.L., Ryan, J.M. and Watts, B.T. "Non-Opaque Selection Method for Concealed Window Resources", IBM Technical Disclosure eBulletin, Vol 37 No 06a, June 1994, pp. 251-256 To see windows underneath/obscured by other windows, user selects a mode whereby all windows become transparent: only the title bar and the frames are drawn. User can click on title bar (or frame) to bring that windows to the top, and transparency dropped. [BedersonB94a] Bederson, Benjamin B.; Stead, Larry; and Hollan, James D. "Pad++: Advances in Multiscale Interfaces", Proc. CHI '94, April 1994, Short Papers pp. 315-316 Infinite resolution multimedia sketchpad: infinite resolution display windowed into smaller display area: parallel lazy loading, only load/render data that is current viewable: example is browsing photos. [BedersonB94b] Bederson, Benjamin B. and Hollan, James D. "Pad++: A Zooming Graphical Interface for Exploring Alternate Interface Physics", Proc. UIST '94, Nov. 2-4, 1994, pp. 17-26 Alternative graphical user interface to icons menus and windows, show small snapshots / postage-stamps of simulated desk objects. Directory browser is array of small thumbnail images of pages/documents. [BedersonB94c] Bederson, Benjamin B. and Hollan, James D. "Pad++: A Zoomable Graphical Interface", Proc. CHI 94 Multiple scale / multi-scale data visualization user interface: example is directory browser, with thumbnails inside of thumbnails and multiple levels [BedersonB95a] Bederson, Benjamin B. and Hollan, James D. "Pad++: A Zoomable Graphical Interface System", Proc. CHI '95 Zoomable user interface, example is "world view" of documents, similar to thumbnail. Compare with



Telesis "world view". [BedersonB95b] Bederson, Benjamin B. and Hollan, James D. "Advances in the Pad++ Zoomable Graphics Widget", Third Annual Tcl/Tk Workshop, July 1995, www.usenix.org Multiple graphical representations of data, constraint-based graphical editing [BetaArchive94a] BetaArchive.co.uk "Screenshots Gallery Windows CE 1.0 Alpha", BetaArchive v2.1 Collect, Learn Preserve! BetaArchive.co.uk, fetched 2014 Screenshots of Microsoft touchscreen PDA user interface, mostly from built-in tutorial. On-screen keyboard by touch of stylus to bottom-of-screen menu or an icon. Handwriting in boxed text fields: tap on field to bring up alternate word or character dictionary pick-list. Electronic ink stored as (erasable) bitmap image? Navigation tabs to right of screen -- compare with PenPoint? Screen rotation / orientation portrait/landscape chosen manually by user, similar to PenWindows dialog. [BierEA94a] Bier, Eric A.; Stone, Maureen C; Fishkin, Ken; Buxton, William; and Baudel, Thomas "A Taxonomy of See-Through Tools", Proc. CHI '94, pp. 358-364; also Baecker95a Towards the Year 2000 See-through transparent/translucent tools can be placed over an object without obscuring objects under the tool. One option is a gesture-interpreting tool. (Compare with gesture-buttons.) Two-handed input (trackball and mouse cursor: multitouch?) trackball moves a virtual transparent layer, such as a transparent selection/framing tool and a mouse click to make an action happen. [BierEA94b] Bier, Eric "Toolglass and Magic Lenses: The See-Through Interface", Shown at CHI '94, available on youtube.com Video demonstration to accompany paper of same title. At about 2:00 shows dragging a "Fill Color" menu with transparent parts over an image. Dual-pane clipboard for picking up diverse multiple objects. [BierEA94c] Bier, Eric "A Taxonomy of See-Through Tools (Video Presentation)", Xerox PARC, 1994, available on youtube.com Video demonstration to accompany paper of same title. At about 0:25 shows a tool sheet (menu with buttons) with transparent sections being dragged over an image. Transparent pie menus / marking menu gestures. Unistroke gestures on a tool: compare with gesture buttons. 3:10 gesture interpreter tool / handwriting pad (Goldberg unistroke recognition). Mnemonic gestures: e.g. "C" for copy, "T" for translate/move -[BierEA94d] Bier, Eric "Enhanced Dynamic Queries Using Magic Lenses and Magic Lenses as Art (Video Presentation)", Xerox PARC, 1994, available on youtube.com Video demonstration. Shows tool window with frame and buttons (but most is transparent) being dragged over a map, and certain features of the map are altered (pixels combined?) Also, at about 0:59 shows translucent overlaying of two tool windows. [BierEA94e] Bier, Eric "The Movable Filter as a User Interface Tool (Video Presentation)", Shown at CHI '94, available on youtube.com Video demonstration. At about 0:50 shows window tool with circular "view" area that changes color of pixels underneath (combined pixels), with part of tool transparent, also with buttons and controls in the window. [BlaisG95a] Blais, Gerard and Levine, Martin D. "Registering Multiview Range Data to Create 3D Computer Objects", IEEE PAMI, Vol 17 No 8, August 1995, pp. 820-824 Mathematical process for aligning/matching portions of surface from multiple views from a single camera. Compare with structure-through-motion. [BlandfordRR94a] Blandford, Robert R. "Personal Computer Diary", United States Patent 5,347,579, September 13, 1994 PIM/Diary that keeps change/delta diff record of changed entries so that previous entries can be reconstructed, and automatic monitoring for dates and other phrases to prompt user to add to diary. Compare to marking menus/keyboard? [Blickenstorfer95] Blickenstofer, Conrad H. "Editorial: Vertical market hardware manufacturers growing stronger, but they're now facing a challenge from consumer electronics giants and their marketing savvy", Pen Computing Magazine, Issue No. 6, August/September 1995 Mentions Scriptel WriteTouch front-mount digitizer for Windows95 and Windows for Pen Computing: cites Windows 95 as having better pen-computing support than Windows for Pen Computing 1.0, although tablet drivers are not included. Norand PEN*KEY6100 vertical market pen tablet device. [BlinnJF94a] Blinn, James F. "Jim Blinn's Corner: Composting, Part 1: Theory", IEEE CG&A, September 1994, pp. 83-87 Tutorial on alpha blending / compositing / matting /overlaying (synonyms), mentions anti-aliasing as one use, primarily "over" operator for bitblt/raster-op from Porter and Duff. [BlinnJF94b] Blinn, James F. "Jim Blinn's Corner: Composting, Part 2: Practice", IEEE CG&A, November 1994, pp. 78-82 Tutorial on alpha blending / compositing / matting /overlaying (synonyms). Focuses on optimizations of



integer arithmetic for 8 and 16 bit integer operations on 8-bit (32-bit) RGBA values, rounding and quantization errors. [BoieRA95a] Boie, Robert A.; Ruedisueli, Laurence W.; and Wagner, Eric R. "Computer mouse or keyboard input device utilizing capacitive sensors", United States Patent 5,463,388, October 31, 1995 Capacitive matrix grid digitizer: objects near intersections of wires affect capacitance between wires. Gets area measurement of adjacent intersection points, computes centroid/peak from adjacent (independent) sensors nodes. Used for mouse and for touch keypad in one device. Compressibility of fingertip to infer pressure (force). multi-touch capable? Compare to Buxton 1985 multi-touch capacitive tablet. See also Radio Drum, Boie 1984 paper on capacitive sensors, references on "capacitive moments". [BoruckiB94] Borucki, Gregory "Surface acoustic wave touchscreen sealing system", United States Patent 5,332,238, July 26, 1994 Sealing for SAW touchscreen tablet: see citations for actual surface acoustic wave touchscreen technologies Elographics IntelliTouch Systems Manual, 1988 [BosscherPJ94a] Bosscher, Peter J. and Yeazel, Dean "Phase I Report: Using handheld Data Collectors in Highway construction for Accounting and Materials Management", Report to the Wisconsin Department of Transportation, January 10, 1994 Recommends PenRight! Pro as leading pen-centric development environment, Windows for Pen Computing lacks pen-centric features, non-pen-centric development tools are unfeasible. No mention of Newton or PenPoint. [BraultJJ95a] Brault, Jean-Jules; Plamondon, Rejean; and Laframboise, Andre "A Gesture-based Editor for Short Handwritten Passages", Proc. Vision Interface '95, pp. 162-169 Electronic ink text editor, electronic ink automatically segmented based on spacing without character recognition, gestures include lasso and brackets to select, arrow to move text, double-ended arrow to copy, caret to insert, X to delete. Compare with van Raamsdonck. [BritzDM95a] Britz, David M. "Personal Communicator Having Orientable Video Imaging Element", United States Patent 5,414,444, May 9, 1955 Smartphone / communicator with video camera and display: camera is mechanically steerable [BrooksLE95a] Brooks, Lamar E. "Hand-held Electronic Writing Tool", United States Patent 5,434,371, July 18, 1995 tablet-free stylus/pen with tip-force (pressure) sensor and accellerometers in tip, handwriting recognition (actually character recognition). Compare with Crane/SRI accellerometer pen (not cited). Uses special delimiter characters/gestures (e.g. right/left corners) to mark start/stop of recognition. [BrownMH95a] Brown, Marc H. and Shillner, Robert A. "DeckScape: An Experimental Web Browser", Digital Systems Research Center Research Report 135a, March 1, 1995; Proc 3rd Int'l World-Wide Web Confg., Darmstadt, April 1995 Web browser, pages can be stacked in multiple decks on screen: no 3-D effect on desks -- compare with Freestyle documents. Suggested use is keeping related web pages together in a single deck. [BurnsJ95] Burns, John and Lloyd, Sheelagh A. "Position-sensing Apparatus", United States Patent 5,442,147, August 15, 1995 Optical digitizer tablet: Note similarity to Anoto: printed surface of pattern cells, making use of subpatterns. Refers also to a "single-pixel" sensor stylus. [BusinessWeek95] Business Week "PenRight! launches first cross platform pen-based mobile computing application development tool", Business Week magazine (Business Wire), July 24, 1996 (abstract only) PenRight cross-platform development environment for pen computing: PenRight! for Windows create Windows and DOS applications for PDAs, tablets, handhelds, etc. from single source code [Buxton94x] Buxton, W. "Combined keyboard / touch tablet input device", Xerox Disclosure Journal, 19(2), 109111 See also http://www.billbuxton.com: User flips keyboard top over to expose touchscreen: touch tablet is multi-touch sensitive and pressure (force) sensitive, with virtual sliders and further touch elements. [Buxton94y] Buxton, W. "Two-handed Document Navigation", Xerox Disclosure Journal, 19(2), 103-108 Two overlapping operations by user as same time using pointing input in both hands. Describes scrolling gestures: smooth scrolling as touch-slide-release, page turning with touch-rapid flick-release, and jumping to scrollbar touch location on a touch surface. Also mentions wireless touch tablet PARCtab. See also http://www.billbuxton.com [Buxton95a] Buxton, William A.S. and Kurtenbach, Gordon P. "An improved graphical keyboard", European Patent Application 94120248.3, Publication Number 0 660 218 A1, June 28, 1995 Graphical on-screen keyboard using gestures for modifier keys such as shift and control: e.g. tap "a" key for lower-case a, upward flick for Upper case, right flick gesture for Control-a, right-upward angle gesture for control-shift-A, etc. Mentions interchangibility of other types of input devices: mouse, tablet, trackball, etc. press-and-wait / PenStill / press-and-hold for marking menu, gesture and wait at end of mark also for



multiple heirarchy of menus. Four events: PenDown, PenDrag, PenUp, PenStill (with no movement for a period of time). [Buxton95x] Buxton, Bill "Chapter 7: Touch, Gesture and Marking", http://www.dgp.utoronto.ca/OTP/papers/bill.buxton/haptic.html: also same title in Baecker95a collection Toward the Year 2000 haptic input overview. Date approximate based on dates of most references, this draft obtained 2002. [BuxtonWB94c] Buxton, Bill "Toolglass and Magic Lenses: The See-Through Interface", University of Toronto, 1993. Available at billbuxton.com Video presentation of two-handed user interface with tool in non-dominant hand, tablet stylus in dominant hand. Companion video to "A taxonomy of see-through tools" and "Toolglass and magic lenses". [CabralAW95a] Cabral, Authur W. et alet al "Object-Oriented Graphic System", United States Patent 5,455,599, Oct. 3, 1995 Object-based graphical development system with polymorphic inheritance [CaelliW94a] Caelli, William; Longley, Dennis; and Shain, Michael "Information Security Handbook", Macmillan Press Ltd., 1994 (hardcopy book) Collection of tutorials on information/computer security. Personnel and administrative policies, legal aspects. Long treatment of cryptography/cryptology/PKI, basic techniques of virus/worm including hooking. Access control primarily about physical access / authentication of users. [CappsSP94a] Capps, Stephen et al "Method and apparatus for correcting words", United States Patent 5,367,453, November 22, 1994 Handwriting recognition that let use display pop-up list of alternative recognition results for words (from spelling dictionary) by entering a gesture to bring it up, user can pick appropriate word, or use virtual keyboard to type in correct word. [CappsSP94b] Capps, Stephen et al "Method for manipulating objects on a computer display", United States Patent 5,345,543, September 6, 1994 Rotate an object with a stylus by means of a center axle mark, and a handle used to rotate. [CappsSP94c] Capps, Stephen P.; Beernink, Ernest H.; Meier, John R.; and Temkin, David "Method and apparatus for correcting words", United States Patent 5,367,452, November 22, 1994 On pen computer or tablet, pop-up list of alternative handwriting recognition results, user can pick from list or enter word via virtual keyboard. Figures show pop-up keyboard of Apple Newton. Correction gestures (insert, delete gesture, etc.). Refers to keyboard gesture (tap on keyboard image), correction gesture (tap on menu of alternative words). [CappsSP95a] Tchao, Michael C. and Capps, Stephen P. "Pointing gesture based computer note pad paging and scrolling interface", United States Patent 5,398,310 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Gestures on buttons or command-input areas to edit electronic-ink notes: notes not necessarily ink. Few if any references before. Compare with van Raamsdonk, Pencept, and CIC. [CappsSP95b] Capps, Stephen, and Beernink, Ernest H. "Method and apparatus for formatting paragraphs", United States Patent 5,479,596, December 26, 1995, Assigned to Apple Computer Inc Automatic re-flow / formatting of paragraphs from handwritten (or "pointing" input): compare to van Raamsdonk. Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Few references before 1990 Compare with van Raamsdonk, Pencept, CIC [CappsSP95c] Capps, Stephen et al "Method and apparatus for the manipulation of text on a computer display screen", United States Patent 5,442,742, August 15, 1995 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC [CappsSP95d] Capps, Stephen et al "Visible clipboard for graphical computer environments", United States Patent 5,404,442, April 4, 1995 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC [CarrDA94a] Carr, David A. "Specification of Interface Interaction Objects", Proc. CHI '94, Boston Massachusetts Shows graphical/touchscreen slide switches and toggles, three-state "secure switch" toggle. Compare to Apple slide-to-unlock gesture? [CarrDA95a] Carr, David Andrew "A Compact Graphical Representation of User Interface Interaction Objects", Ph.D. dissertation, Dept. of Computer Science, Univ. of Maryland, 1995 User Action Notation for user interface objects. Example: Secure switch graphical object for preventing accidental manipulation of a device under computer control, such as master power switch for factory machinery. Compare to Apple slide-to-unlock gesture?



[CerdaM95a] Cerda, Michael "EO Interest Document", www.utexas.edu/ftp/microlib/eo/html Personal note on EO with PenPoint: lists of collected EO/PenPoint materials. EO 440 hardware specifications. EO based in UK? [ChangL94a] Chang, Larry and MacKenzie, I. Scott "A Comparison of Two Handwriting Recognizers for Pen-based Computers", Proceedings of CASCON '94, pp. 364-371, Toronto: IBM Canada Compare Pen-Windows and CIC Handwriter recognizers: the CIC handwriting recognizer performed better, both with and without constraints. [ChattyS95a] Chatty, Stephane; Lecoanet, Patrick; and Mertz, Christophe "Toward pen-based interaction with ATC tools", Air Traffic Control Quarterly, Vol 3 No 4, 1995, pp 205-228 GRIGRI (French slang for "scribble") system using pen-computing gesture recognition in user interface for air traffic control ATC. Modeled on flight strips for air traffic control. See also "Digistrips". Numeric entries for flight numbers via menus. Cites PenPoint. Lists fifteen single-stroke gestures, some of which are alphabetic: set warning, lasso (select), C, erase/pigtail gesture, undo alpha gesture, h change heading, etc.. Mentions that pen-based computing is bad for large ATC display screens (arm fatigue). Refers to gesture input as "mark-based input". Uses semi-transparent windows /button so as not to obscure radar image underneath window. [ClarkeTA94a] Clarke, T.A. "An analysis of the properties of targets used in digital close range photogrammetric measurement", Proc. Videometrics III. SPIE Vol. 2350. Boston. pp. 251- 262, 1994 Summary and overview of physical types of targets used in 3D capture: e.g. LED's, patterned crosses, reflective films, etc. No mention of using features of existing objects. [CloughWA95a] Clough, William A.; Ouelette, Daneil; De La Sablonniere, Serge "Portable Computer with Touch Screen and Computer System Employing Same", United States Patent 5,379,057, January 3, 1995 Portable slate pen-computer using touch screen (not proximity digitizer): Microslate, Inc. of Canada. Describes having an automatic application generator. Refers to drop-down pick list as a "library" of answers for touch-screen selection. [ComputerBusinessReview94a] Computer Business Review "Hand-helds on the Battlefield: Grunts are to get the Texas Micro Grunt", Computer Business Review, 10 August 1994 The Grunt: Texas Microsystems 3.5 pound Windows for Pen Computing portable computer using MicroTouch touch-screen: pen or special pen/stylus. [ComputerWoche95a] Computerwoche "Mobile Computing Griffelcomputer haben sich in einigen Projekten bewährt -- CruisePAD, PoqetPad", Computerwoche1995 : available at www.computerwoche.de Pen-based computers / pen computers useful in data collecting with UPS, engineering maintenance on ships, of immediate access to data (stored in device) and faster to input test or delivery data. Mentions that generally no or little use of handwriting recognition, data entry by picking on pre-designed forms, adjusting sliders, sketches. PoquetPad hand-held pen computer, CruisePAD separate tablet/display for wireless access to PC within 100 meters [CoronaJ95a] Corona, James; Henderson, Randal L. and Laib, Gregory D. "Method and System for Independent Control of Multiple Windows in a Graphics Display System", United States Patent 5,475,812, December 12, 1995 Transparent overlay windows in graphical user interface, pixel blending [CottrellSR95a] Cottrell, Stephen R. "Method to Provide Security for a Computer and a Device Therefor", United States Patent 5,465,084, November 7, 1995 Instead of password, user inputs a pattern of touches or events for authentication: slide-to-unlock? [Cripps95] Cripps, Marcus "Introduction to PenWindows", Available at http://tawny.cs.nott.ac.uk/AMI/apprentice/INTERNAL/tech08.htm Architectural overview of the PenWindows / Windows for Pen Computing system from Microsoft: hedit and bedit controls, Pen Extensions, normal and inking mode, RC handwriting recognition manager, PenWin.dll [Cubictechcorp95a] cubictechcorp "LabView User Manuals", cubictechcorp.homestead.com/Manuals.html, fetched 2009 LabView manuals from early 1990's for visual programming language. Shows slide switches, manuals for sale? [DardaillerD95a] Dardailler, Daniel "Mobile GUI on the Web", Presentation and paper, World Wide Web Conf. (WWW4), Boston USA, December 1995. Suggest "X" windows as solution to lack of control over exact appearance of HTML documents: Pixel-level control. Extends to remote execution (X-client/server). Suggest LBX (Low Bandwidth X) protocol to reduce latency/bandwidth problems. Similar presenation to "X Over the Web", WWW3, Darmstadt, Germany april 1995. [DarwinI95a] Darwin, Ian; Quercia, Valerie; and O'Reilly, Tim "X Window System User's Guide - OPEN LOOK Edition", Open-Source publication of manuscript originally for O'Reilly publishers, 1995 Open Look windows management system / user interface. Drag-Right-Distance to select sub-menu. MoveThreshold, MouseDamping system properties. Compare with press-and-hold



[DeTarJ94] DeTar, Jim "NCR, Scriptel push cordless digitizer - NCR Microelectronic Products and Scriptel Holding Inc. develop WriteTouch cordless electrostatic digitizer", Electronic News, Nov 14, 1994, Available on http://findarticles.com Electrostatic / capacitive digitizer from Scriptel, can sense finger input as well as stylus [DrummDE94a] Drumm, Donald E. "Computer Input Device using an Orientation Sensor", United States Patent 5,297,119, February 15, 1994 Donald Drumm "Orb" project from Wang Freestyle: pointing device for accessibility, uses head-mounted tilt sensor to push cursor position, simple voice detection circuit that detects fricatives "s" sound versus vocalized sounds. [ECMA95a] ECMA/European Computer Manufacturers Association "Application Programming Interface for Windows", Standard ECMA-234, December 1995. www.ecma.ch Standardization functional specification of Microsoft Windows 3.1 application programming interface. WS_EX_TRANSPARENT creates transparent window. Also GetBkMode TRANSPARENT parameter for drawing. Transparent windows are included in windows hit testing, transparent windows can be buttons/scrollbars. [EatonJK94a] Eaton, John K. and Eaton, Laura "LabTutor: A Friendly Guide to Computer interfacing and LabVIEW Programming", Oxford University Press, 1994 (hardcopy book) LabVIEW graphical programming language. [EliensA95a] Eliens, A. "Hush -- a C++ API for Tcl/Tk", The X Resource, Issue 14, April 1995, O'Reilly & Ass., pp 111-155 Refers to mapping of events to mouse buttons in user interface. Compare to press-and-hold. [EloTouch95] Elo TouchSystems, Inc. "DOS and Windows Driver Guide, version 2.0", Elo TouchSystems, Inc. 105 Randolph Road, Oak Ridge, Tennessee 37830. Copyright 1987..1995 Contains description of touchscreen implementation of mouse menus for DOS application which are not "mouse-aware". See also Microsoft documentation on MENU.COM and mouse menus. Tablet drivers for mouse emulation in DOS and Windows. [EssexD95a] Essex, David "Rewriting Handwriting Recognition", BYTE Magazine, June 1995 Refers to reduced expectations on handwriting recognition accuracy, not to read human-illegible handwriting. Reference to Doonesbury commentary, use of point-and-tap selections and digital ink. Shows modified Graffiti character set (neography), Lexicus Cursive Recognition [FarkasDK95a] Farkas, David K. and Poltrock, Steven E. "Online Editing, Mark-Up Models, and the Workplace Lives of Editors and Writers", IEEE Trans. on Professional Communication, Vol 38 No 2, June 1995, pp. 110..117 Review of products and the problems of collaborative on-line editing tools for editors and writers. Suggests sticking with "traditional" mark-up symbols: easily learned, familiar, well developed for the problem domain. States pop-up or embedded comments awkward. Change tracking does not do well showing extensive formatting changes to users. Recommends digital ink (electronic ink) with gesture commands (the mark-up symbols), but commands not executed until approved by author. PenEdit (Advanced Pen Technologies), MATE, Red Pencil. [FaughnanJ95a] Faughnan, John "Bar coding journal articles - DataGlyphs", Posting to ftp.org/mailman discussion list, December 29, 1995 Proposal to use a DataGlyph (two-dimensional barcode) to having indexable/scannable information at the start of published articles. Compare to Silverbrook patents of 2004, and to Anoto. Note: there was a similar technology for a 2D barcode date for articles already in place in the 1980's? [Fenwick95] Fenwick, Daniel J. "Generation of Interdependent Font Characters Based on Ligature and Glyph Categorizations", United States Patent 5,412,771, May 2, 1995, assigned to Signature Software Inc., Hood River, Oregon Creating of a "handwriting-looking" font from samples of someone's handwriting: refers in claims to automatic classification/categorization based on slope and other characteristics of the characters [FischerAM95a] Fischer, Addison M. "Method and apparatus for creating supporting and using travelling programs", United States Patent 5,390,247, February 14, 1995 Travelling software program: interpretive code (P-code: compare mobile Java objects) moves from user to user to handle PKI cryptographyic signing and verification. Includes code-signing of program segments, transmit the certificate segment ahead of the program. [FischerAM95b] Fischer, Addison M. "Computer system security method and apparatus having program authorization information data structures", United States Patent 5,412,717, May 2, 1995 Signed authorization tokens (program authorization information) stating what capabilities (rights/permissions) a program may exersise. Tokens may be co-signed by multiple permission-granting authorities. Include checking timestamp and signature of program getting permission. Compare with DRM tokens? [FishkinK95a] Fishkin, Ken and Stone, Maureen C. "Enhanced Dynamic Queries via Movable Filters", Proc. CHI



1995, pp 415-420 Magic Lens project: application to database systems -- e.g. user requests map with information (from database) as to properties to rent. Magic Lens "filter" in window of GUI tool to change how information is display / what information is displayed. [FitzmauriceG95a] Fitzmaurice, George W.; Ishii, Hiroshi; Buxton, William "Bricks: laying the Foundations for Graspable User Interfaces", Proc. CHI 1995 GraspDraw project: Physical devices coupled to "virtual" objects via position sensors: blocks (bricks) that may be moved on a position-sensing surface. Refers to multiple-value input being either time-multiplexed (one mouse or stylus) or space-multiplexed (two hands). [FrankEH94a] Frank, Edward H.; Naughton, Patrick J.; Gosling, James Arthur; and Liu, John C. "Method and apparatus for presenting information in a display system using transparent windows", European Patent Application EP 0605945 A1, July 13, 1994 Transparent windows in a GUI, using alpha blending of pixel data. Example show menu boxes with buttons overlaid on other windows. [FrankishC95a] Frankish, Clive; Hull, Richard; Morgan, Pam "Recognition Accuracy and User Acceptance of Pen Interfaces", CHI '95 Proceedings, available at http://www.acm.org/sigchi/chi95/Electronic/documents/papers/crf_bdy.htm Short study of Windows for Pen Computing handwriting recognition accuracy, compares to Neisser60, cites also LaLomia94 [Fried-Nielson94a] Fried-Nielsen, Lars K. "Symbol Browsing in an Object-Oriented Development system", United States Patent 5,339,433 IDE development environment with object, code browser, links reference to source code. No layout editor for GUI shown. [FriendJ95a] Friend, John; Freedman, Mike; Bricklin, Dan "Input Device with Deferred Translation", United States Patent 5,455,901 Delayed translation/Deferred recognition of electronic ink, part of Slate/Bricklin thinking of ink as a data type. Implemented for PenPoint and Pen Windows in PenApps, a Visual-Basic like language. Refers to leaving electronic ink displayed if handwriting not recognized, or handwriting recognition result not found in dictionary. User can also display ink instead of recognized text. Gestures? [FrischA95a] Frisch, Aeleen "Unix: Essential System Administration", O'Reilly and Associates, 1991, 1995 (hardcopy book) System administration manual for Unix. Discusses ACLs access control lists on AIX and HP-UX versions of Unix, with extended permissions, deny as well as grant permissions/rights. [FullerTH94a] Fuller, Thomas H. Jr. and Kimura, Takayuki D. "Supervised Competitive Learning", Jnl. Intelligent Material Systems and Structures, march 1994, pp. 232-246. Also publish as Report WUCS-93-45, Washington Univ. Dept. of Comp. Sci., October 1993 Machine learning / backpropogation neural net work recognizing handwritten characters and symbols (gestures): training-based recognizer. Multi-stage recognizer: "energy" dynamic features (velocity and accelleration), and 3x3 grid patterns. [GarfinkelS95a] Garfinkel, Simson "PGP Pretty Good Privacy: Encryption for Everyone", O'Reilly, 1995 (hardcopy book) Guide to PGP public/private key cryptography and PKI [Garnet94a] Brad A. Myers, Dario Giuse, Andrew Mickish, Brad Vander Zanden, David Kosbie, James A. Landay, Richard McDaniel, Rajan Parthasarathy, Matthew Goldberg, Roger B. Dannenberg, Philippe Marchal, Ed Pervin "The Garnet Reference Manuals: Revised for Version 3.0", CMU Computer Science report CMU-CS-90-117-R5, December 1994, and additional materials Collection of reference manuals and papers for Garnet IDE (see other refs) from CMU archive web site as a single publication/reference: includes directory listings. [GeisslerJG95a] Geissler, Joerg "gedrics: The Next Generation of Icons", Proc. 5th Int'l Conf. on Human-Computer Interaction (INTERACT '95), Lillehammer, Norway, June 27-29 1995, pp. 73-78 Critique of Pen Windows / W4PC as only encompassing double-click, not drawing of gestures. Gedrics: gesture-driven icon. Gestures (of single stroke, to be fast) are performed on an icon: tab to activate, right-left to open, single-stroke question mark for help menu, closed "X" to delete, press/hold or hold/drag for copy/paste/move-to-parent, etc. circle left and circle right. Implementation on Newton. Does not mention "marking menus". [GeneralMagic94a] General Magic "General Magic Phone Predates Smartphones", General Magic Corporation. Video available at http://www.youtube.com/watch?v=wko0m6XA9yg, Fetched 2011 Video presentation of General magic smart phone / PDA. Slate computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Calendaring, virtual keyboard, electronic ink for notetaking, Personal Link communication network services from AT&T for email, etc.



[GeneralMagic94b] General Magic "Video Telescript General Magic: A 1994 Vision of the Future", General Magic Corporation. Video available at http://www.youtube.com Fetched 2011 Video presentation of General magic smart phone / PDA. Slate computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Focus is on Telescript: early messaging/email service? [GeneralMagic95a] General Magic "Telescript Language Reference, October 1995", General Magic Corporation Telescript: Prototype-based programming language for pen-centric pen computing applications for General magic platform. [GeoWorks94a] GeoWorks "GEOS Concepts Version 3.0", Geoworks, Incorporated 1994 GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Includes discussion of electronic ink, handwriting recognition, API for gesture recognition. [GeoWorks94b] GeoWorks "GEOS Objects Version 3.0, Nokia 9000 Communicator SDK", Geoworks, Incorporated 1994 GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Class heirarchy and messages. [GerlachJD94a] Gerlach, John D. and Weiblen, Michael E. "System for Relocating a Multimedia Presentation on a Different Platform by Extracting a Resource Map in Order to Remap and Relocate Resources", United States Patent 5,317,732, May 31, 1994 Visual programming language / GUI for creating programmed multimedia presentations [GerpheideGE94a] Gerpheide, George E. "Methods and apparatus for data input", United States Patent 5,305,017, April 19, 1994 Capacitive/electrostatic grid tablet ("dipole electrodes"), finger affects capacitance at intersections. Coarse position is the grid crossing point with the highest value for change of capacitance. Interpolates between two adjacent grid crossing points for higher fine resolution to determine centroid of finger position. Proximity sensing. Says Rympalski measures changes in capacitance as conductive object comes near. Compare with multi-touch? [GesslerS95a] Gessler, Stefan and Kotulla, Andreas "PDAs as mobile WWW browsers", Proc. WWWF '94, Computer Networks and ISDN Systems, 1995. Available at ww.teco.uni-karlsruhe.de/~stefan/papers/wwwf94/www_pda.html Speculative (forward-looking) paper on web-browser for PDA Apple Newton: advantage is ubiquity and portability, disadvantage was high wireless data charges while mobile, small display (four-way scrolling particularly difficult) and small memory. Deferred fetching of images (low bandwidth through workstation/PC to dial-up Internet connection). No bibliography: does not mention Lynx browser [GlaeserG94a] Glaeser, Georg "Fast Algorithms for 3D-Graphics", Springer Verlag, 2004 Textbook/reference on 3D graphics algorithms: mostly curved surfaces, hidden surface removal, with emphasis on real-time performance (so nothing on ray tracing or luminosity). No masking. [Greanias95] Greanias, Evon C.; Verrier, Guy F.; Donaldson, Robert L. "Touch Overlay for Improved Touch Sensitivity", United States Patent 5,386,219, January 31, 1995 Capacitive grid digitizer, transparent on top of display [GreenbergS94] Greenberg, Saul "HCI Video Resources", Available at http://www.cpsc.ucalgary.ca/~saul/hci_topics/hci_resources/videos.html Extensive list of video tapes from ACM CHI, CSCW, and ACM SIGGRAPH Vieo Review relating to handwriting recognition, sketch input, signature verification, and pen-computing: 1983 to 1994. 1963 Video: Videotape of Sketchpad system by Sutherland 1985 Video: Software Control at the Stroke of a Pen - Pencept - CHI '85 1989 Video: The Paper-Like Interface, IBM Watson Research Center - CHI '89 1989 Video: Freestyle - Wang Labs - CHI '89 1992 Video: Combining Gesture and Direct Manipulation - CMU 1992 Video: Touchscreen Toggle Design - Univ. of Maryland 1992 Video: Tactile Manipulation on a Digital desk - Rank Xerox EuroPARC Video: Alternate Reality Kit - Xerox PARC Video: The Lapidary Graphical User Interface Design Tool - Univ. of Tennessee [GreysonAM95a] Greyson, Ann M.; Hokit, Jeffrey D.; Kaptanoglu, Marjory; Wagner, Annete M.; and Capps, Stephen P. "Method and apparatus for the manipulation of text on a computer display screen", United States Patent 5,442,742, August 15, 1995 Apple Newton: selected text in text editor can be copied/pasted by drag-and-drop: shaded image of text is what user drags around. Compare with floating icon for drag-and-drop in PenPoint? [GurskiMA95a] Gurski, Michale A. "Privacy-Enhanced Email (PEM)", Internet posting, 1995 Overview of PEM Privacy Enhanced Email PEM standard: certificate based encrypted and originatorauthenticated/signed email. File also contains RFCs. RFC 1421: Privacy Enhancement for Internet Electronic Mail: Part I: Message Encryption and Authentication Procedures (1993) RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management (1993) RFC 1423: Privacy Enhancement for Internet Electronic Mail: Part III: Algorithms, Modes, and Identifiers (1993) RFC 1424: Privacy Enhancement for Internet Electronic Mail: Part IV: Key Certification and Related Services (1993) [HP95a] Hewlett-Packard "HP OmniGo 100 PDA product information", Hewlett-Packard



Touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials. [HP95b] Hewlett-Packard "HP OmniGo 100 Organizer Plus pen-computer PDA", HP OmniGo 100, Hewlett-Packard Omnigo (physical device) Functional touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. (Screen is square, rotation is immediate.) Pop-up tools have window frame and draggable title bar. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials. [HP95c] Hewlett-Packard "Geoworks Organized Extras for the HP OmniGo 100", Hewlett-Packard, included with OmniGo devices Touchscreen PDA with full keyboard, swivel/convertible/clamshell display, penGEOS software. Front cover shows OmniGo 100 with both portrait and landscape screen orientation. Trademark on "hotsync" "hotsync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable). GeoWorks Bindery for electronic books with hyperlinks in text. IZL (Interpreted Zoomer Language) programming environment for GeoWorks. Septor wireless pager/news-alerts services . [HP95d] Hewlett-Packard "HP OmniGo 100 Connectivity Pack File Management Tool", Hewlett-Packard, included with OmniGo devices Windows software and serial cable for syncing and transfering between OmniGo pen PDA and a Windows PC. Trademark on "hotsync" "hot-sync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable). [HP95e] Hewlett-Packard "HP OmniGo 100 Quick Start and User's Reference Guide", Hewlett-Packard, included with OmniGo devices Pen GEOS operating system documentation. [HarrisonBL94a] Harrison, Beverly L.; Zhai, Shumin; Vicente, Kim J.; and Buxton, Bill "Semi-Transparent "Silk" User Interface Objects: Supporting Focused and Divided Attention", Cognitive Engineering Laboratory, Report CEL 94-08, University of Toronto, 1994 Looking for optimal trade-off of transparency between user attention being focused on one of two images, versus divided attention between the two (Stroop Test). Show semi-transparent windows, semitransparent pull-down menus, pop-up menus over graphics and over text. Also shows 100% transparent, with only menu labels / lines visible (in contrasting puke-green color) with shadowed outline for contrast. [HarrisonBL94b] Harrison, Beverly L.; Ishii, Hiroshi; Vicente, Kim J.; and Buxton, Bill "Evaluation of a Display Design space: Transparent Layered User Interfaces", Report CEL 94-07, Cognitive Engineering Laboratory, Univ. of Toronto, Canada, 1994 See-through/transparent image over background image gives illusion of depth (depth multiplexing): superimposed windows, overlapping windows. Also changing physical depth of images (semi-silvered mirrors) in 3D. [HarrisonBL95a] Harrison, Beverly L.; Kurtenbach, Gordon; and Vincente, Kim J. "An Experimental Evaluation of Transparent User Interface Tools and Information Content", Proc. UIST 95, Pittsburgh, USA, November 1995 Show semi-transparent pop-up menus (including pie menus or context menus) semi-transparent or wireframe over object, so that user can still see context, underlying image/object not obscured. See-through UI tools. [HarrisonBL95b] Harrison, Beverly L.; Ishii, Hiroshi; Vicente, Kim J. and Buxton, William A.S. "Transparent Layered User Interfaces: An Evaluation of a Display Design to Enhance Focused and Divided Attention", Proc. CHI 95, May 7-11, November 1995, pp. 317..324 User interface with semi-transparent windows, menus, dialogue boxes, other objects with attention to "layered" interface objects, click-through tools. Cites to Knowlton on dynamically re-labelled buttons and keys. [HayesP94a] Hayes, P. "NameFinder: software that finds names in text", Proc. of RIAO '94, pp 762-774 (abstract only) Commercial product that finds literal names based on capitalization, and dictionary, plus knowledge engineering rules. Compare with regex expressions to find phone numbers. Finding names, combine with UI? [HechtDL95a] Hecht, David L.; Flores, L. Noah; Petrie, Glen W.; and Webster, Thomas M. "Random access techniques for use with self-clocking glyph code", United States Patent 5,449,896, September 12, 1995 Xerox data glyphs, 2-D encoding of information in small pattern within printed images. Diagonal pattern segments, 0 and 1 representations at right angles, but small enough that do not appreciably affect visual appearance. Made robust to rotation by encoding reserved pattern segments that are used to recognized the orientation. [HenckelJDD95] Henckel, Jonathan D. and Hospers, Paul A. "Data Processing System Graphical User Interface



which Emulates Printed Material", United States Patent 5,463,725, October 31, 1995 Gesture (?) for turning a page by touching edge of page in a simulated book, and moving animated page(s) to the left or right. Compare with flick gesture, PenPoint OS metaphor of notebook pages? [HenshawSF94a] Henckel, Jonathan D. and Hospers, Paul A. "Translucent Windows: Dragging an Image without Obscuring the Desktop", IBM Technical Disclosure Bulletin, Vol 37 No 10, October 1994 Translucent/semi-transparent window on OS/2 so as not to obscure image underneath (touchscreen/mouse). Blending for transparency is done by checkerboard of BITBLT, so that background image with colors appears visible. [HinckleyK94a] Hinckley, Ken; Pausch, Randy; Goble, John C.; and Kassell, Neal F. "A Survey of Design Issues in Spatial Input", Proc. UIST '94, November 1994, Marian del Rey, California, pp. 213-222 Hybrid user interface: touchscreen UI can be combined with 3-D input device, because user can reach out to touch touchscreen with the 3-D input device in their hand. Relative vs. Absolute gestures: hard to perform gestures in absolute coordinates. Bi-manual/two-handed input advantageous over sequential single-hand input, because user can (e.g.) position a tool with one hand and use the tool with the other. Mentions using multiple pointing devices in a prototype (trackball and mouse, or two mice). User's spontaneous use two hands for combining translation/rotation/scaling gesture (direct manipulation). Compare with multi-touch. [HinckleyKP95a] Hinckley, Kenneth P.; Bathiche, Steven N.; Cauthorn, James H.; and Sinclar, Michael J. "Automatic Scrolling", United States Patent 6,975,306, December 13, 2005 Scroll to line, paragraph, or other document boundary. Scrolling initiated by minimum gesture movement, with second threshold to stop autoscrolling or scroll back if touch location has not moved on touchsensitive surface. [HoYL95] Ho, Yung-Lung, and Li, Hwok-Leung "Track Pad Cursor Positioning Device and Method", United States Patent 5,424,756, June 13, 1995 Touchpad controller for very small touchpad: uses (optical) sensing of velocity to position cursor faster if touch is moving fast. [HopkinsT95a] Hopkins, Trevor and Horan, Bernard "Smalltalk: An Introduction to Application Development using VisualWorks", VisualWorks Object-Oriented Programming Series, 1995 Introduction to Smalltalk language and environment. (Electronic file.) [HuJ95a] Hu, Jianying, Brown, Michael K., and Turin, William "Invariant Features for HMM Based On-Line Handwriting Recognition", in "Image Analysis and Processing", Braccini, C., DeFloriani, L. and Vernazza, G. (Ed.), Springer Verlag, 1995, pp. 588-593, Lecture Notes in Computer Science #974 Invariant handwriting features (independent of rotation, translation, scale) are ratio of tangents and normalized curvature, for Hidden Markov Model recognizer [HuJ95b] Hu, Jianying, Brown, Michael K., and Turin, William "Use of Segmental Features in HMM Based Handwriting Recognition", Proc. IEEE Int'l. Conf. on Systems, Man, and Cybernetics, Oct 22-25, 1995. Vol 3, pp. 2778-2782 Stochastic pattern recognizer, Hidden Markov Model (HMM) URL as of 5/9/2002: http://www.research.avayalabs.com/user/jianhu/index.html with additional information on Jianying Hu. States that using a sliding window in a recognizer reduces error rate by 50%, but does not give the point of comparison, or the description of how the error was computed (false positive, false negative, etc.) [HuangHC95a] Huang, Hung-Chih and Stein, Robert A. "Frontally Shielded Capacitive Touch Sensor System", United States Patent 5,457,289, October 10, 1995 Resistive-film capacitive touchscreen with front shield to protect from stray capacitance [HuangJ95a] Huang, Jianming and Knowles, Terence J. "Acoustic Wave Touch Panel for use with a non-active stylus", United States Patent 5,451,723, September 19, 1995 Acoustic wave touchscreen: hard stylus presses on top elastomeric layer, which then absorbs acoustic energy of wave propagating in layer underneath [HullR94a] Hull, Richard; Reynolds, Dave; and Gupta, Dipankar "Scribble Matching", HP Laboratories Bristol Technical report HPL-94-61, July 1994 indexing of electronic ink notes by matching scribble markings: states 97% successfully matching rate (compare with character recognition rates of 97%), Scribbles/blobs taken from fields: e.g. "To:" fields in notes/email. Feature points are knots/inflections and syntactic rules for sequences, compare with chain codes. [IBM94a] IBM ""Redbook" IBM Pen for OS/2 and PenDOS", IBM Form Number GG24-4217-00 Architecture of IBM Pen for OS/@ V1.0 and PenDOS from IBM, version 2.2: Lists gesture assignments by Session Type, Gesture Commands and Sessions [IBM94b] IBM "Simon Says "Here's How!", Simon Mobile Communications Made Simple, Users Manual", IBM Corp., 1994 IBM Simon user manual. Touch screen uses plastic stylus or finger. PredictaKey keyboard shows six most



likely characters on virtual keyboard, Other shows next six most likely. User can highlight numbers in messages and they are dialed from a pop-up menu. Physical slide switch p. 4 for On/Off / Resume/Suspend, p. 8 Simon is not completely off. If not off, touching screen turns unit on. Buttons with broken-line edge are not available: drag stylus across screen to highlight button, then lift stylus to select that feature. Some buttons display check mark on/off on touch. Folder-shaped buttons contain additional buttons. Has search function for address book p 35. Specifically states that operations happen "when you release the button" e.g. pen/stylus lift, not on pen/stylus down -- carry-over from mouse development? [IBM94c] IBM "AIX Version 4 Desktop Handbook", IBM Corp. Document Number GG24-4451-00, December 1994 Windowing system uses different control layout from Microsoft Windows. Drag-and-drop to printer icons. Style manager allows color palette for all windows to be updated. Three button mouse, pop-up menu button can be rightmost of leftmost button depending on handedness of user. [IBM94d] IBM "IBM Simon (First review) Video", Fetched from youtube.com 2014 Unnarrated demonstration of IBM Simon from BellSouth touchscreen smartphone, circa 1994 [IBM94e] IBM "IBM Simon Phone -CHI 2011 Video", Fetched from youtube.com 2014 Demonstration of IBM Simon touchscreen smartphone, circa 1994, from reporting on CHI 2011 conference. [IBM95a] IBM "Data Interpretation Techniques for a Pen-Based Computer", IBM Technical Disclosure Bulletin, Vol 35 No 9, September 1995 (partial copy) Pen/OS2 pen computer: every pen touch-down event on an object, object determines how stroke should be interpreted: e.g. mouse-mode in title bar, mouse+gesture mode on button, mouse+gesture+singlecharacter-recognition in list box, etc. User can override definitions manually, or by holding stylus sidebutton. Compare with "gesture buttons", ink interpretation in PP and PW? [IBM95b] IBM "Pen for OS-2 Version 1.03 readme and file listing", IBM, 1995 Readme file (and file listing) of installation media for Pen extensions to OS-2 operating system, 1995. "Enable Edit Window" -- first point of contact (in one edit window) causes all other input to be directed to that window regardless of where. "Telepen" whiteboard application? Some incompatibilities with pen as mouse. [ImpedovoS94a] Impedovo, Sebastiano, editor "Fundamentals in handwriting recognition", Proceedings of the NATO Advanced Study Institute on Fundamentals in Handwriting Recognition, Chateau de Bonas, France, June 21-July3, 1993 Overview of problems in handwriting recognition, both OCR and on-line. Includes discussion of loop/cusp transition (i discussion on segmentation of script characters) [InfoWorld94a] InfoWorld "More tools available for Power Mac programmers", InfoWorld, April 4, 1994, page 23 CodeWarrior by Metroworks, Object Master, MacAPP, Workshop. Development environment: gives 1994 or earlier date for MacAPP and CodeWarrior [InfoWorld94b] Barney, Doung "WinPad OS for hand-helds held up", InfoWorld, September 5, 1994, page 6 Microsoft WinPad palm-top/hand-held pen computer, high price and release delays resulting in Compaq not releasing Mobile Companions. (WinPad later named Microsoft At Work for Handhelds.) [Inforite94] Inforite "MP100 Writing Pad, Rite-Expressions annotation software for handwriting", BYTE Magazine product announcement, May 1994 Appears to have no relation to Inforite82 product, separate company? Several companies have used the name "Inforite"? [Intel95a] Intel "Intel386 EX Embedded Microprocessor MHT9000 Handheld Terminal", Intel Application Note AP719, August 1, 1995 Intel386 embedded processor, includes registers and MTL6560 digitizer controller for resistive touchscreen (digital voltage measurement), controller/co-processor for LCD [InterLink94a] InterLink Electronics "InterLink Electronics FSR Force Sensing Resistors -- VersaPad USB Integration Guide", InterLink Electronics Sensor Technologies, 1994 resistive touchpad using two sheets separated by an air gap, four-wire sensor. Based on contact resistance, also determines touch pressure (force). Tap-and-drag gestures. Drag Edge lock: if drag to edge, maintains drag gesture status for three seconds after release so that drag can be continued/extended with an additional drag/wipe gesture. Further gestures include right-click by tapping in upper right corner region of touchpad digitizer, right edged border scrolling for up/down scrolling. [IsaacsonP94a] Isaacson, Portia "The Power of 'E-INK'", OEM Magazine, February 1994, Page 93 States large hardware business for specialize electronic chips for PDAs with electronic ink, handwriting recognition coprocessor. Mentions immaturity (performance defects) of digitizer tablets. e-ink. [IsenseeSH95a] Isensee, Scott Harlan and Poston, Ricky Lee "Graphical User Interface Using Transparent Windows", European Patent Application 95303142.4, Pub No 0684542A1, November 29, 1995 User interface, set windows to transparent so that user can see the information they have selected. [Ishigaki94] Ishigaki, Kawasaki Kazushi "Handwriting Entry Processing System Using Keyboard and Mouse Interfaces Requiring No Modification of an Application Program", United States Patent 5,361,310, November 1,



1994

Handwriting recognition UI characters translated to keyboard input: no references to Sklarew/Linus or Ward/Pencept? [IshiiH94a] Ishii, Hiroshi; Kobayashi, Minoru; and Arita, Kazuho "Iterative Design of Seamless Collaboration Media", CACM, Vol 37 No 8, August 1994, pp. 83-97 TeamWorkStation-1/ClearBoard: Whiteboard systems: Combined videoconference image in shared drawing space on workstation display. Using transparent display, other user appears in video on through transparent display, writing reversed. Cites to Media Space shared drawing space from 1988. "Transparent digitizer sheets" -- does not say what touchscreen technology used. [JacobRJK94a] Jacob, Robert J.K.; Sibert, Linda E.; McFarlane, Daniel C.; and Mullen, M. Presont Jr. "Integrality and Separability of Input Devices", ACM Trans. Computer-Human Interaction, Vol 1 No 1, March 1994, pp. 3-26 Describes mouse as absolute position device if not lifted from pad (sic), thus a three-dimensional digitizer is equivalent for two dimensions. Refers to device-independent API interfaces such as ACM CGS. But in practice, practical attributes of input devices are not separable from types of user interfaces and interactions. [JanikCM94a] Janik, Craig M. "Flexible Wearable Computer", United States Patent 5,285,398, February 8, 1994 Wearable computer, a PC (with standard PCMCIA cards, etc.) with components distributed in pockets in a "utility belt", cables/connectors for system bus etc. running through belt (or suspenders). Refers to foldable/portable input/output device for pen-based LCD (or keyboard/touchscreen). [JohnsonGW94a] Johnson, Gary W. "LabVIEW Graphical Programming: Practical Applications in Instrumentation and Control", McGraw-Hill, Inc. 1994 (hcarcopy book) LabView graphical programming language for software simulation and control of virtual laboratory instruments, describing dataflow programming. Text refers to RUN switch in examples (p84/92). Introduction gives history of LabVIEW to 1986. User interface: notes that scroll bar on MacIntosh and Windows effectively halts asynchronous/multi-tasking operation. [KabbashP94a] Kabbash, Paul; Buxton, William; and Sellen, Abigail "Tow-Handed Input in a Compound Task", Proc. CHI '94, Boston, Massachusetts, April 1994. pp. 417-423 Toolglass two-handed/bimanual user interface: non-dominant hand positions an object or tool, dominant hand uses or manipulates the object or tool. [KajlerN95a] Kahler, Norbert and Soiffer, Neil "A Survey of User Interfaces for Computer Algebra Systems", J. symbolic Computation, Vol 11 no 1, 1995 Survey of mathematical expression editing, mostly from a keyboard (and mouse?). Section 4.10 talks about pen computing user interfaces: two-dimensional input is good, but large number of symbols used is problematic for handwriting recognition [KanoS94a] Kano, Shigeru; Nakada, Yasuhiro; and Usuda, Yutaka "System for editing characters and graphics by pen point entry", UK Patent Application GB 2274186 A, July 13, 1994 User interface with touchscreen, electronic ink, gesture commands, and handwriting recognitions. System determines how to interpret user input based on context such as size relative to known characters, location of input on/over/awa-from existing ink or characters, whether the input document already exists, and relative priority of gestures. [KaplanJ94a] Kaplan, Jerry "Notes file: Startup: A Silicon Valley Adventure", various: Wired magazine, Slate.com, etc. Additional materials in file: winster.com, etc. historial notes on PenPoint, concerning Jerry Kaplan [KaplanJ95a] Kaplan, Jerry "Startup: A Silicon Valley Adventure", Penguin Books, ISBN 0-395-71133-9, ISBN 0 14 02.5731.4, 1995 (hardcopy book) History of GO Corporation and PenPoint, experience with IBM and Microsoft, from point of view of CEO of GO. Compare to account of same events by Marlin Eller in "Barbarians led by Gates" about PenWindows. Additional materials in file: winster.com, etc. historial notes on PenPoint [KasselRH95a] Kassel, Robert Howard "A Comparison of Approaches to On-Line Handwritten Character Recognition", Ph.D. Thesis, Dept. of EECS, M.I.T., May 12, 1995 Study of testing handwriting recognition algorithms using some 87,000 characters from 150 writers. Supported by Microsoft Pen Computing Group: Greg Slyngstad and Sunh Rhee. (Cites variability models of Ward and Kuklinski, but does not derive implications about 97,000 being a small sample size.) [KelleyAJ94a] Kelley, A.J. and Salcudean, S.E. "MagicMouse: Tactile and Kinesthetic Feedback in the HumanComputer Interface using an Electromagnetically Actuated Input/Output Device", Technical Report, Dept. of E.E., Univ. of British Columbia, October 19, 1993 Haptic mouse with force feedback, integrated with X windows. Electromagnetic actuator in mouse, feedback provided external to host CPU because control loop through host application too slow or too low in bandwidth. Turns mouse into absolute position device? [Kempf94a] Kempf, James "HRE API: A Portable Handwriting Recognition Engine Interface", Available at



http://playground.sun.com/pub/multimedia/handwriting/hre.html Proposed API for handwriting recognition systems, using pen strokes optimized for point vectors in the X window system, but otherwise independent of OS or windowing system Incremental translation/recognition of handwriting support for context as for Japanese and Cursive English API for character prototype files and dictionaries Data structures for gestures and arbitrary objects Data structures correlations between recognized objects and pen strokes [Kempf94b] Kempf, James "Preliminary Handwriting Recognition Engine Application Program Interface for Solaris 2", Sun Microsystems 1993, 1994. Available at http://playground.sun.com/pub/multimedia/handwriting/hre.html Handwriting recognition system for pen-based computing for Solaris: cited to PenWindows/Microsoft Windows for Pen Computing (MWPC), PenPoint, Apple Newton for pen-computing. Solaris API based on PenWindows API and Penpoint. Not integrated tightly with windowing system: multiple window toolkits on Solaris (XView, OLIT, Motif, COSE, Interviews). Stateful implementation: can acquire context information from previous strokes. Recognition subsets of gestures, mathset / math symbols, Kanji (JIS1 subset only), Katakana, hiragana, etc. Support for delayed strokes vi correlation. [Kim94] Kim, Joonki; Teibman, George J.; and Tappert, Charles C. "Method and Apparatus for Improving Prototypes of Similar Characters in On-line Handwriting Recognition", United States Patent 5,285,505, February 8, 1994 Averaging out the prototypes in adaptive handwriting recognition system [KingmanJA95a] Kingman, John A.; Larky, Steven P.; and Vanover, Michael T. "Window specific control of overlay planes in a graphics display system", United States Patent 5,469,541, November 21, 1995 Transparent overlay windows in a user interface [KirkD95a] Kirk, David ed. "Graphics Gems III", CalTech AP Professional Press, 1995 Third volume in "Graphic Gems", collection of papers: Image Processing (bitmap compositing), Numerical and Programming Techniques, Modeling and Transformations, 2D Geometry and Algorithms, 3D Geometry and Algorithms, Ray Tracing and Radiosity, Rendering. [KnabeK95a] Knabe, Kevin "Apple Guide: A Case Study in User-Aided Design of Online Help", Proc. of CHI '95, Short Papers On-line help, with hypertext linking and search for related topics by text [KnappAG94a] Knapp, Alan G. "Fingerprint sensing device and recognition system having predetermined electrode activation", United States Patent 5,325,442, June 28, 1994 Capacitive grid/matrix of conductors used for touchpad sensor for fingerprint scanning. Compare to grid arrangement of Rekimoto. [KnasterB94a] Knaster, Barbara "Presenting Magic Cap: A Guide to General Magic's Revolutionary Communicator Software", Addison-Wesley, 1994 (hardcopy book) OS for hand-held personal information manager: Telescript language for application implementation. Touch-sensitive touchscreen, stylus or finger. Refers to targetting "halo" for objects. (Compare with Slate PenApps "gravity"). Tap/Touch gesture terms used interchangeably. Desktop/Office/filing-cabinet/hallway metaphor. Electronic ink note-taking. On-screen keyboards: brings up "smart guess" keyboard (numeric, etc.) based on syntax of input. [KnowltonKC94a] Knowlton, Kenneth C. "Computer Apparatus and Method for Graphical Flip Book", United States Patent 5,283,864, February 1, 1994 Simulation of indexed flip book / rolodex in Wang Freestyle, with small edges of pages shown. [KodoskyJL94a] Kodosky, Jeffrey L, Truchard, James J.; and MacCrisken, John E. "Graphical Method for Programming a Virtual Instrument", United States Patent 5,301,336, April 5, 1994 Feature in LabView visual programming language for virtual laboratory instruments. Fig. 5 , text page 70 refers to slide switch: compare to Apple slide-to-unlock? [KohlJT94a] Kohl, John T.; Neuman, B. Clifford; and Ts'o, Theore Y. "The Evolution of the Kerberos Authentication Service", IEEE Distributed Open Systems, 1994, pp. 78-95 History and technical description of Kerberos authentication system through version 5, at M.I.T. Project Athena. Single sign-on: user login gives ticket used by all application. Uses secret-key (single key) cryptography, version 5 addressing interoperability with public-key cryptography: cites to RSA digital signatures. See also earlier 1991 publication. [KontarinisDA95a] Kontarinis, Dimitrios A. and Howe, Robert D. "Tactile Display of Vibratory Information in Teleoperation and Virtual Environments", Presence: Teleoperators and Virtual Environments, vol. 4 no. 4, pp. 387-402, 1995 Vibration haptic feedback on remote-manipulation display, combined with/without force feedback. Illustration is two finger "waldos" to prevent user pressing too hard. [KramerA94a] Kramer, Axel "Translucent Patches -- Dissolving Windows", Proc. UIST '94, November 2-4, 1994, Marina del Ray, California, pp. 121-130 Overlapping translucent windows (or non-rectangular patches), so that users can annotate on top of other



windows. Implementations: re-drawing (non-pixmap?) on top of other drawings after lightening lower window, changing color of lower pixels by translucency of pixel above (alpha blending). Interesting approach on sketching, electronic ink. Patch-related drawing gestures: line into (pearl), zig-zag clear (erase), zig-zag over border (cross-out/delete), small circle for pop-up, clockwise/ccw spirals. Gestures vs. ink by side-button on stylus/pen tablet (compare Pencept recognition macros). Overlapping windows may cause lower patches to render differently (e.g. not show control elements, etc.) (cites Toolglass). [KreegarJW95a] Kreegar, Jeffrey W. "Non-modal Method and Apparatus for Manipulating Graphical Objects", United States Patent 5,396,590, March 7, 1995 User interface to select objects using selection rectangle (objects may intersect edges), the selected a tool: Scale, Skew, Rotate, etc. Moving rotate tool changes/sets pivot/rotation point. [KunoY95a] Kuno, Yoshinori; Tanabe, Noboru; and Hasebe, Koichi "Portable display device with at least two display screens controllable collectively or separately", United States Patent 5,467,102, November 14, 1995 Dual-display device, two displays can be used as one (two pages of a book), or used separately. Compare with devices like Refalo? Uses term "electronic pen" for touchscreen. May be used as add-on display to desktop computer. Thumbnails of document pages on one display, full page on other. Can display virtual keyboard on one display or portion. [Kurtenbach94] Kurtenbach, G.; Moran, T.P.; and Buxton, W. "Contextual Animation of Gestural Commands", Computer Graphics Forum, Vol. 13 (1994), No. 5 pp 305-314 Use of pop-up radial menus to train users to "gestures" which match the motion used to select from a radial menu. Also display of images of gestures in menus, and pop-up text giving name/meaning of gesture recognized. Circle/lasso gesture around an object brings up a pop-up menu of actions that may be applied to the object. Also shows pig-tail gesture for delete/erase, scroll-up and scroll-down (flick?), lasso gesture for select. Gestures may be recognized using velocity: e.g. Slow flick moves to next page, fast flick to end of document. press-and-hold/"pressing and waiting" [Kurtenbach94b] Kurtenbach, G.; and Buxton, W. "User Learning and Performance with Marking Menus", Proc. CHI '94, pp. 258-265 Marking menus on MacIntosh/Apple. (pop-up pie menus when user hold stylus still, selectable without first lifting pen). Zig-zag gestures for navigation. press-and-hold / press-and-wait for 0.3 second. [KusumotoH94a] Kusumoto, Hiroki and Hasuhara, Hiroshi "Image drawing apparatus for displaying input image on display means", Unite States Patent 5,311,207, May 10, 1994 Pressure sensitive tablet digitizer using two conductive sheets and conductive rubber [LaLomiaM94a] LaLomia, Mary "User acceptance of handwritten recognition accuracy", Conference on Human Factors in Computing System, Boston, Massachusetts, 1994, pp 107-108 Cited along with Neisser (93% recognition success for humans without context), that users not satisfied with 97% correct character-to-character handwriting recognition success. [LambAC94a] Lamb, Arthur C. Jr. "Pop-up Keyboard System for entering Handwritten Data into Computer Generated forms", United States Patent 5,276,794, January 4, 1994 pop-up virtual keyboard in GridPad computer, in a particular form application. Compare to Soft input panel SIP? [LammingM94a] Lamming, Mik; Brown, Peter; Carter, Kathleen; Eldridge, Margery; Flynn, Mike; Loutie, Gifford; Robinson, Peter; and Sellen, Abigail "The Design of a Human Memory Prosthesis", Xerox EuroPARC Technical Report EPC-1993-119, 1993. Also The Computer Journal, Vol 7 No 3, 1994, pp. 153-163 Basically a PDA/smartphone/always-with-you-computer with notes, reminders, calendar, video diary that is also sensitive to user's context (especially location). Intended to help look up information, or recall information for user. Cites to Memex as collection of user's notes and documents. Memory aids: e.g. easier to remember who was at meeting if other aspects of meeting are recreated / reminded. Cites to Amstrad PenPad PDA, EO 440 Penpoint, Newton Messagepad, etc. Main example is Xerox PARCtab, touchscreen, wireless/infrared, active badge (fine-grained location). [LandmeierWL95a] Landmeier, Waldo L. "See-through Digitizer with Clear conductive Grid", United States Patent 5,381,160, January 10, 1995 Front-mounted electromagnetic digitizer (CalComp) with a display: gives long description of Scriptel frontmounted electrostatic display, discussion of problems of rear mounting. Uses term "pen-driven computing". Two layers of grid are fabricated on two layers of transparent mylar. [LeeJ94] Lee, Jack "Pen-based form computer", United States Patent 5,347,477, September 13, 1994 Forms-based inputting gestures to control recognition, insert, delete scroll -- compare with GO Refers to smoothing (and averaging?) to obtain the non-distortion point on stylus input filtering? [LeeY95a] Lee, Yenwei and Dong, Yu "Method and Apparatus for Character Recognition Interface", International Patent Application WO 97/18526, May 22, 1997 On-screen toolbar for selection one of several on-screen virtual keyboards or handwriting input, toolbar can be disabled: handwriting recognition shows multiple candidate results, user picks correct one. Toolbar



can be repositioned on screen: compare with Pen Windows and PenPoint? [LeedhamCG94a] Leedham, C.G. "Historical perspectives of handwriting recognition systems", IEEE Colloquium on Handwriting and Pen-Based Input, 1994, pp 1..13 (abstract only) history of handwriting recognition (?) back to 1950's. [LewisJR94a] Lewis, James R.; Allard, David J.; and Hudson, Harold D. "Predictive Keyboard Design Study: Effects of Different Word Populations, Number Of Displayed Letters, and Number of Trigraph Tables", IBM Technical Report TR54.846, May 31, 1994 Study of digraph/trigraph dictionaries in virtual keyboard typing correction: a single trigraph dictioary not as useful as multiple digraph/trigraph dictionaries with different word populations / vocabulary for appropriate domains. [Lexicus95] Lexicus "Lexicus' Cursive Recognition", BYTE Magazine, June 1995: available at http://www.byte.com/arg/9506/img/501026j0.htm Lexicus neural network handwriting cursive recognition: 25,000-word dictionary and statistical methods (compare with Viterbi algorithm?) [LiebermanH94a] Lieberman, Henry "Power of Ten Thousand: Navigating in Large Information Spaces", Proc. UIST '94, November 2..4, 1994, pp. 15-16 When zooming on graphical image/map, show birds-eye or world or greater context view as translucent overlay on window, so that user can maintain visual context. Zooming on file system heirarchy zooms into files, without explicit opening of file. Cites to Bier for "see-through" interface (transparent pixels instead of translucent overlay). [LoganJD94a] Logan, James D. and Evans, Blair "System and Method for Emulating A Mouse Input Devices with a Touchpad Input Device", United States Patent 5,327,161, July 5, 1994 Microtouch touchpad digitizer: press-and-drag distance is a minimum distance threshold before user input does drag automatically by simulating mouse button action. Dwell/press-and-hold for minimum distance threshold. [LoomisJM94a] Loomis, Jack M.; Golledge, Rignald G.; and Klatzky, Roberta L. "Personal Guidance System for the Visually Impaired using GPS, GIS, and VR Technologies", Proc. Assets 94 1st ACM Conf. on Assistive Technologies, 1994, pp. 85-91 GPS aid for visual impaired/blind, giving information about surrounding environment as well as location. Output to user by means of artificial auditory beacons located at correct/appropriate binaural location relative to user. [LoprestiDP94a] Lopresti, Daniel P. and Tomkins, Andrew "On the Searchability of Electronic Ink", 4th International Workshop on Frontiers in Handwriting Recognition, Taipei, Taiwan, December 1994 Avoid need for handwriting recognition when indexing documents by approximate matching of hand-drawn pictograms. Mentions difficulty of handwriting recognition of (Asian) languages. Compare with Slate DayTimer/PenScheduler for ink indexing, Windows for Pens NoteTaker. [LoprestiDP95a] Lopresti, Daniel P.; Aref, W.G.; Kamel, I. "On handling Electronic Ink", ACM Computing Surveys, vol 27, no 4, December 1995, pp. 564..567 Searchability of electronic ink; approximate matching of hand-drawn pictograms, not just comparing with character recognition results to index handwritten text [LouieJ95] Louie, Jim "An Analysis of a Pen-Based Tool for Acquiring Engineering Design Information", Master's Thesis, University of Toronto, 1995 Review of pen-based computer (PenPoint) with applications for engineering: notetaking, engineering design. Applications written in Slate PenApps. Generally, paper fared better than electronic notebook, in part because notebook was not perfect writing surface and was a bit small (6" by 9"). Notes that engineers dislike doing documentation, in part because words are not the natural medium of expression -diagrams/sketches are preferred. Cites several other electronic/virtual notebook systems, but most not portable. [LoweryG94a] Lowery, Gwen "Managing Projects with Microsoft Project 4.0", John Wiley & Sons, Inc., 1994 Project planning/project management application. Tasks, Milestones, assignment of resources and personnel, project tracking. Supports critical path method CPM. Views include PERT charts, Gantt charts, Detail charts, Calendar views. [LucenteM95a] Lucente, Mark and Galyean, Tinsley A. "Rendering Interactive Holographic Images", Proc. SIGGRAPH 95, Los Angeles, Aug. 6-11, 1995, pp. 387-394 Computational rendering and display of holographic (stereo vision: full 3D possible) images in "patient" real-time (2 seconds). Can also render a real-world scene captured from a linear array of cameras. [LucenteSAM95a] Lucente, Samuel A.M.; Balasubramanian, Peruvemba S.; Sapper, Richard F.; and Lee, Nathan J. "Rotatable pen-based computer", United States Patent 5,432,720, July 11, 1995 Pen computer can be rotated so that screen orientation is portrait or landscape, for right-handed or lefthanded user. Mechanical button to select orientation.



[LuciwWW95a] Luciw, William W; Capps, Stephen; and Tesler, Lawrence G. "Method and apparatus for deducing user intent and providing computer implemented services", United States Patent 5,390,281, February 14, 1995 Determine user intent by applying a pattern template to user interactions in a pen user interface, and then carry out an operation automatically. Also check Capps file [MacKenzieIS94a] MacKenzie, I. Scott; Nonnecke, Blair; and Riddersma, Stan "Alphanumeric entry on pen-based computers", Int'. Jnl. of Human-Computer Studies, No 41, pp. 775-792, 1994 Speed of entering text on pen-based (PDA/tablet) computer: on-screen/soft keyboard with QWERTY basest, then hand-printing recognition, on-screen/soft keyboard in two-row Alphabetic arrangement slowest. [MacKenzieIS95a] MacKenzie, I. Scott "Movement Time Preduction in Human-Computer Interfaces", Readings in Human-Computer Interaction, 2nd ed., 1995, pp. 483-493 (Baecker95a)(reprint from 1992) Application of Fitts' law to 2D and 3D user interfaces. [MacWeek95a] MacWeek "Digital Ocean to make a more rugged Newton", MacWeek, vol 9 no 2, January 9, 1995 Product announcements on Tarpon, ruggedized Newton MessagePad with wireless modem (Ardis network), GPS, cellular phone. [MackayWE95a] Mackay, W.E.; Pagani, D.S.; Faber, L.; Inwood, B.; Launiainen, P.; Brenta, L.; and Pouzol, V. "Ariel: Augmenting Paper Engineering Drawings", Proc. CHI '96, May 7-11, 1995, Denver CO, pp. 421-422 Description of Wellner-DigitalDesk-like interactive project for capturing handwritten (paper) updates to CAD drawing. Overhead Projection of virtual paper onto physical paper, location of physical paper tracked optically. Live video communication between users. [MagicLink94a] Sony "Video: Magic Link Sony PDA PIC-1000 running Magic Cap OS (General Magic)", Sony circa 1994: Video fetched from YouTube 2011 Shows pen/stylus based PDA with office desk / office room and hallway metaphor. Appears to be a resistive touchscreen, looks like it might be a Data Rover 840. [MankeS94a] Manke, Stefan and Bodenhausen, Ulrich "A Connectionist Recognizer for On-line Cursive Handwriting Recognition", Proc. ICASSP-94, 1994 IEEE Int'l. Conf. on Acoustics, Speech, and Signal Processing, Vol 2, pp. 633-635 Dynamic time warping / Time Delay Neural Network for single-character recognition, using a 3x3 or 20x30 bitmap local image Claims 99.5% recognition writer-dependent, between 91.5% and 99.5% on writerindepedant: expansion to larger writer-independent database gives about 75% [MarcusA95a] Marcus, Aaron; Smilonich, Nick; and Thompson, Lynne "The Cross-GUI Handbook For Multiplatform User Interface Design", Addison-Wesley/Unisys, 1995 (hardcopy book) GUI frameworks comparisons: Macintosh Desktop, OSF/Motif Desktop (with three-button mouse operations BSelect, BTransfer, BMenu right button), NeXTSTEP Desktop, IBM CUA Workplace Shell, Microsoft Windows Desktop Environment. Does not cover PenPoint, page 191-ff covers Microsoft Pen Windows extensions: press-hold-drag is a selection operation, PenWindows 1.0 gestures Extende Backspace, etc. Direct manipulation. Pop-up menus on page 34 and 202. Describes drag-and-drop and pop-up menus as OOUI, object-oriented user interface, not GUI. [MartinD95] Martin, David A. "Interactive Display System", United States Patent 5,448,263, September 5, 1995 Projection whiteboard system with LCD project panel, overhead projector, digitizer. See other references to MartinD. Describes alignment/calibration method for display and projection vs. tablet, similar to that of PenCept/CIC/etc. for paper versus tablet. [MartinT95a] Martin, Teresa A. "The Tablet Newspaper: A Vision for the Future", Proc CHI '95 "Mosaic of Creativity", Video presentation, May 7..11, 1995, p. 425 Knight-Ridder Information Design Lab: electronic newspaper on a pen computer, with pen/touch user interface [Maryland95] Maryland, Universtify of at College Park "DOCBIB through 1994: Document Understanding Bibliography", http://documents/cfar.umd.edu/biblio/DOCBIBpre95.ps, fetched 1998 Bibliography of papers relating to document understanding, starting about 1980. Sections include: On-line character recognition; Gestures and Sketches; Script; Foreign optical character recognition; Recognition of mathematics and Formulas; Line drawings; Signature verification; Writer Identification; Synthetic data; Forms processing; Shorthand; Music recognition [MasuiT95a] Masui, Toshiyuki; Kashiwagi, Kouichi; and Borden, George R. IV "Elastic Graphical Interfaces for Precise Data Manipulation", Proc. CHI '95, May 7-11, 1995, pp. 143-144 scrollbar slider: user can control speed of scrolling (for finger control) based on pulling a virtual elastic line farther or less (or changing direction of pull). Cites to AlphaSlider. Note: cited in at least one scroll/bounce patent by Apple. [MaulsbyD94a] Maulsby, David "Instructable Agents", Ph.D. Thesis, Univ. of Calbary, Canada, June 1994 (partial copy) Chapter 5: Interactive learning / Concept learning system Cima, Turvy, and Moctec. Example is teaching



system to find phone numbers in notes, in different contexts and in various canonical forms, and second component that learn so interact with users and application programs, (dialing numbers) [McDanielG94a] McDaniel, George "IBM Dictionary of Computing, Tenth Edition", McGraw Hill, 1994 (hardcopy book) Technical dictionary [McKeehanJ94] McKeehan, Julie and Rhodes, Neil "Programming for the Newton: Software Development with NewtonScript", AP Professional Press, division of Academic Press, 1994, ISBN 0-12-484800-1 (hardcopy book) Newtonscript Language [McLeanJ95a] McLean, J., Pickover, C.A. and Winarski, D. "Access/Control Icons (Icon Keys)", IBM Technical Disclosure Bulletine, Vol 38 No 4, April 1995, pp. 407-410 (partial copy) Slide an icon onto a position to unlock: slide to unlock? [McQueenJC95a] McQueen, J. Craig; MacKenzie, I. Scott; and Zhang, Shawn X. "An Extended Study of Numeric Entry on Pen-based Computers", Proc. of Graphics Interface '95, Toronto, pp. 215-222 Pie-pad pop-up menu for numeric input, pop-up circular menu with digits. Marking menus. Mentions Goldberg 93 paper on unistroke. [MeyerA95] Meyer, Andre "Special Feature: Pen Computing - A Technology Overview and a Vision", ACM SIGCHI Bulletin, July 1995. Also available at http://www.amug.org/amug/sigs/newton/nanug/PenReport/References.html Includes reviews of several experimental systems of the early 1990's, and a lengthy bibliography: several German references included. Almost all the references are from 1991 to 1995, very little before that. Unistrokes. Jotted notes: implying electronic ink as a data type. PenPoint, Windows for Pen Computing. Cites Alan Kay for early vision of handwriting computer (although no such device was ever built?). Cites early paper by Ward, but not the on-line bibliography (1995). Rotatable orientation of pen computers: landscape/portrait. [MicroTouch95a] MicroTouch Systems Inc. "MicroTouch Touch Controllers References Guide, 1995", MicroTouch Systems Inc., Methuen Massachusetts touchscreen/tablet/digitizer: mentions dual sensing of stylus/pen or finger or both, with positions of each. [Microsoft94a] Microsoft "Tap and Tap-and-Hold Pointer Events", Article 86921, formerly Q86921, support.microsoft.com: Microsoft Corporation, 1995, last reviewed January 9, 2003 Tap and hold (TapNHold) gesture: tap analogous to clicking left mouse button: TapNHold click tip of pen on tablet and holding for a specified amount of time. German edition of (discontinued) support article documents Windows for Pen Computing 1.0, prior to Windows95. Tap/press-and-hold on Pen Windows [Microsoft94b] Microsoft "Information At Your Fingertips", Transcript of Bill Gate's Keynote Speech at Fall/COMDEX Nov. 14, 1994: from microsoft.com 2008 Windows for Pen Computing as portable information at your fingertips: reference "vision" video of Information at your Fingertip in 2005, with face-to-face video in mobile settings, pen computing, video conferencing. [Microsoft94c] Microsoft "Windows for Pen Computing User's Manual 1.0", User manual for Toshiba TC 200 80 pen computer with Pen Windows, 1994 Windows for Pen Computing version 1.0 user manual. Pen tools group includes Pen Pallete (soft input panel for writing/keyboard), on-screen virtual keyboard. Sample form has boxed handwriting input, unboxed input with timeout for word segmentation.. Checkmark gesture to edit text (similar to PenPoint), pigtail gesture to delete, space-insert gesture with hotpoint. Tap and hold gesture (press-and-hold) to start selection stroke. Newline, copy (sideways pigtail), paste (inverted V), delete, delete word (right-leftcombined), extend, tab gestures. Cursor automatically turns to pen for handwriting input in application when cursor is in a text-input field. User-defined gestures are circled letter gestures. Trainable recognition (note that Toshiba unit ships with CIC recognition). Control panel settings for double-tap gesture time and area settings, press-and-hold delay. Press-and-hold is effectively a separate gesture recognized by distance threshold and time threshold: moving before time threshold starts a gesture or ink (with recognition). Handedness setting changes location of menus. Control panel rotate setting to switch orientation of display landscape/portrait. Notebook application: barrel button on pen to display ink eraser; deferred recognition: select ink (by tapping) then select text tool. Recognized text optionally sized according to ink dimensions. Shape recognition (circles, squares). Equations (= at end) automatically calculated; outline indexing of notes. Chart of recommended printing styles. [Microsoft94d] Microsoft "Accessibility Products for Microsoft Windows and Microsoft Windows NT", "MS_ACCES.TXT", Microsoft, 1994 List of third-party accessibility extensions for Windows and Windows NT. Screen reader / screen review, screen enlarger, on-screen keyboard / virtual keyboard. Vendors include Magic Deluxe / Microsystems Software, MAGNUM Deluxe / Arctic Technologies, MagniPORT / Microsystems Software, Number Nine (hardware pan and zoom in display card), Vista / Telesensory Inc., Zoom Text / Ai Squared, JAWS, outSPOKEN . On-screen keyboards include word prediction, click or dwell mode / AutoClick. Screen



Keyboard (SKB) in Windows for Pen Computing. [Microsoft94e] Microsoft "Microsoft Windows "Chicago" Reviewer's Guide: Beta-1", Microsoft, 1994 Microsoft Pre-release feature description for Chicago / Windows95. Plug-and-play, Toolbar (for cut-andpaste, etc.) in MS-DOS command window. Support for multiple pointing devices (mostly for accessibility: alternative head mouse, etc.). Nothing on SIP Soft Input Panel, PenWindows or tablets. [Microsoft95a] Microsoft "The Windows Interface Guidelines - A Guide for Designing Software", Microsoft Corporation, Feb. 1995 (partial copy) UI guidelines include keyboard, mouse, and Windows for Pen Computing / PenWindows gestures: General circle-around-UC-letter gestures, has circle-M/M-circle pop-up "context menu gesture" / "pen menu gesture". Fundamental behaviors defined for a Pen are Pressing/Tapping/Double-tapping/Dragging. Gestures leave ink. Lasso-tap (circle-tap) selection gesture. Tap on text box brings up text-input handwriting window, with alternative word list or alternative character list from recognizer. Partial print copy in physical file -- see electronic copy at http://www.ics.uci.edu/~kobsa/courses/ICS104/coursenotes/Microsoft_WindowsGuidelines.pdf (fetched 2010) stylus or finger, pen can perform equivalent operations of mouse. [Microsoft95b] Microsoft "WD: Known Issues Using "Windows 95 for Pen Computing"", support.microsoft.com, article Q164505 / 164505, last Review November 5, 2000 (1995) Defect in tablet/pen driver in Windows for Pen Computing: circle-m context menu gesture does not emulate right mouse button, other problems with barrel button on stylus [Microsoft95c] Microsoft "The Windows Interface Guidelines for Software Design", Microsoft Corporation, 1995 (partial copy) Similar topic to Microsoft95a -- includes section on special design considerations of Accessibility. Some sections on pen input. Guidelines on targeting of electronic ink to fields (but not built-in? Compare with Slate PenApps "gravity"). Lasso-tap gesture to select. Need larger tapping (double-tapping) target with stylus than with mouse, similar considerations for targeting of lasso-tap selection gesture on text. Pen menu (context menu) gesture is circle-m. Describes activate-on-lift of mouse being applied for pen: "Do not activate the window unless the user releases the mouse button (the pen tip is lifted) in that window." Press-hold-drag for select instead of barrel button. Auto-scrolling based on velocity. Mentions accessibility "Screen Review Utilities" (screen readers) for the bling, on-screen keyboards. Pop-up/context menus synonymous: refers to contextual interfaces such as pop-up menus, property sheets, and OLE drag and drop. pop-up (context) menu handlers. Press-and-hold for select, p. 55. Partial print copy in physical file -electronic image file. [Microsoft95d] Microsoft "penwin.h Pen Windows functions, types, and definitions Version 1.0", Microsoft Corporation, 1995 #include definitions file for Windows for Pen Computing Version 1.0 -- 1992? Pen tablet data OEMPENINFO include pressure (force), height (proximity), angle X/Y and Z, barrel rotation. Standard gestures for clear, extendSelection, undo, copy, cut, paste, clearword, kkconfer; AnsiGestures (ASCII?) backspace, tab, return, space; application gestures circle a-Z and circle A-Z; shapes line, ellipse, rectangle, min and max See also penwin.h for version 2.0 [Microsoft95e] Microsoft "Accessibility Products for Microsoft Windows: On-Screen Keyboard Utilities", Microsoft Corporation, 1995 Accessibility on-screen visual/virtual keyboard systems for accessibility: compares to single-switch Morse code. Eek!; Microssytems Software HandiKey; My-T-Mouse; SAW Switch Access to Windows; SoftType; WiViK2 Windows Visual Keyboard; Features include word prediction, dwell-mode pointing, access grids, mouse emulators (head-mouse, etc.), keyboard macros See also similar Microsoft technical notes from other years [Mikan94] Mikan, Peter J. "Computer Mouse Simulator Device", United States Patent 5,376,946, December 27, 1994 Device to convert absolute co-ordinates of touchscreen to mouse relative-motion coordinates, and having "buttons" recognized on the touchpad for the mouse button: prior art include MicroTouch Unmouse cited 1990 Compare to Schumer patent on tablet controller, where driver functions are claimed if located in an external controller for a tablet (or mouse?) [MinasM95a] Mina, M. and Viehstaedt, G. "DiaGen: A Generator for Diagram Editors Providing Direct Manipulation and Execution of Diagrams", Proc. of VL Visual Languages 95, Darmstadt, Germany Sketch/diagram editor employing constraints, and dealing with inconsistent state during direct manipulation editing: flowchart, hypergraphs [MinskyMDR95a] Minsky, Margaret Diane Rezvan "Computational Haptics: The Sandpaper System for Synthesizing Texture for a Force-Feedback Display", Ph.D. Dissertation, M.I.T, June 1995 Force-feedback joystick, with multiple force actuators. Software to create feel of textures, virtual spring and inverse spring forces, etc. Includes reference manual for Sandpaper software.



[MoranTP95a] Moran, Thomas P.; Pedersen, Elin R.; McCall, Michael K.; and Halasz, Frank G. "Time-Space Object Containment for Graphical User Interface", United States Patent 5,404,430, April 4, 1995 When selecting objects by selecting a range (drag selection rectangle?), take into account time ordering of when objects were created: user is likely to want to select objects created by the user at about the same time. Uses a white board with rear projection. [MoranTP95b] Moran, Thomas P.; Chiu, Patrick; van Melle, William; and Kurtenbach, Gordon "Implicit structures for pen-based systems within a freeform interaction paradigm", Proc. CHI '95, (preprint) Whiteboard system (Xerox Live-Board Tivoli / MeetingBoard on Windows PC) with free-form interaction -primarily electronic ink scribbling: recognized rough layout of lists, text, tables, and indented outlines in electronic ink, based on informal structure or "visual grammars". Mentions aha! InkWriter which formats/indexes ink based on handwriting recognition, but does not compare with van Raamsdonk which edits directly electronic ink. Selection via various bracket and corner gestures, insert via caret gesture. Has "table clean-up" function to align slanted writing lines of electronic ink. [MorticeKern95a] Mortice Kern Systems "MKS Toolkit User's Guide", Mortice Kern Systems, 1995 (hardcopy book) Port to Windows of Unix Programmer's Workbench: compare with cygwin [MorticeKern95b] Mortice Kern Systems "MKS Toolkit Reference Manual", Mortice Kern Systems, 1995 (hardcopy book) Port to Windows of Unix Programmer's Workbench: compare with cygwin [Mosaic95] Mosaic Input Technologies, Inc. "Gesture Mosaic User Guide", www.amtelcom.com/Mosaic/UserGuide.html Simplified unistroke-like gestures, as substitute for writing full characters. User writes on a 12-segment outline of characters, outline in shape of figure eight, similar to seven-segment display. Punctuation and some other characters input via pop-up keyboard. For Apple Newton. (See also AlphaPad00) [MossNN95a] Moss, Nicolas, N and Marianetti, Ronald II "Movement and Redimensioning of Computer Display Windows", United States Patent 5,434,964, July 18, 1995 Radius Inc. Display: rotating display between landscape and portrait orientation results in all windows being rotated, windows may also automatically layout GUI elements. [Motif94a] X Window System, Open Source Edition "The Definitive Guides to the X Windows System, Vol. 6a: Motif Programming Manual for Motif 2.1, Open Source Edition", www.opencontent.org/openpub Motif Window Manager: 1994 version, updated as of 2001. resource XmNsnapBackMultiple sets snapback/bounce distance if use moves mouse cursor too far outside of bounds of scrollbar widget. Automatic Layout of GUI elements? XmNselectThrehold: click and drag minimum distance that pointer must move to select text on a drag. [MozartM94a] X Window System, Open Source Edition "First Smartphone the Simon Cell Phone New in 1992 by Mike Mozart of TheToyChannel on YouTube", www.youtube.com/watch?v=q658HGEcOZ8, posted 2012 Video demonstration of IBM Simon touchscreen smartphone: games, apps, scrabble, text messages, data plan [MyersBA94a] Myers, Brad A. "Challenges of HCI Design and Implementation", Interactions: ACM Trans. on Inf. Systems, Jan. 1994, pp. 73-84 in Garnet system papers: Prototyping user interfaces can be difficult, because test-subject users may not behave like real users, may even have a hidden agenda. Coupling of user-interface to application (via application call-backs) is a nightmare in maintainability. Shows dynamic menus. [MyersBA94b] Myers, Brad A.; Guise, Daria A.; Mickish, Andrew; and Kosbie, David S. "Making Structured Graphics and Constraints Practical for Large-Scale Applications", Carnegie Mellon University Report CMU-HCII-94100 Garnet system: Constraint toolkit for UIMS, including object management in user interface [MyersBA94c] Myers, Brad A.; Dario Giuse; Andrew Mickish, Brad Vander Zanden, David Kosbie, James A. Landay, Richard McDaniel, Rajan Parthasarathy, Matthew Goldberg, Roger B. Dannenberg, Philippe Marchal, Ed Pervin "The Garnet Reference Manuals: Revised for Version 3.0", Carnegie Mellon University Report CMU-CS-90-117-R5, December 1994 Garnet system: Constraint toolkit for UIMS, including object management in user interface [MyersBA94d] Myers, Brad A.; Giuse, Dario; Mickish, Andrew; Vander Zanden, Brad; Kosbie, David; McDaniel, Richard; Landay, James; Goldberg, Matthew; and Parthasarathy, Rajan "The Garnet User Interface Development Environment: Video Abstract", Proc. CHI '94, April 24-28, 1994, Boston, pp. 457-458 Garnet system: Direct manipulation user interface and IDE with Interactor objects for mouse, rotating, text editing, gesture recognition. Include Lapidary to create widgets or objects, Marquise tool to demonstrate/animate user interface behavior of applications. Copy of presentation video in electronic files. [MyersBA94e] Myers, Brad A. "User Interface Software Tools", CMU School of Computer Science, Technical Report CMU-CS-94-182, August 1994 Garnet system: supersedes CMU-CS-92-114. States gesture UIs used since 1970's in commercial CAD tools



(probably Applicon) [MyersBA94f] Myers, Brad A. "Overview of the Garnet System", CMU School of Computer Science, December 1994 Overview to Garnet System 3.0, gesture-based/direct-manipulation user-interface project. Introduction to full set of Garnet documentation, "The Garnet Reference Manual: Revised for Version 3.0". Extensive bibliography. Garnet: "Generating An Amalgam of Real-time, Novel Editors and Toolkits". IDE which allowed developers to "draw" the user interface using graphical objects: user-interface styles could be applied to all user-interface/objects (look-and-feel independent). Slots (methods): compare with PenBasic from Slate. "X" gesture to delete (using tablet?). [MyersBA95a] Myers, Brad A. "User Interface Software Tools", ACM Trans. on Computer-Human Interaction, March 1995, Vol. 2 No. 1, pp 64-103 in Garnet system papers: describes X-Windows lack of agreement on widgets, therefore Xt intrisics so support. 3-D GUI toolkits: Brown Animation Generation System, Silicon Graphic's Inventor toolkit. See also Myers 93 reference, same title. [MysliwiecTA94a] Mysliwiec, Thomas A. "FingerMouse: A Freehand Computer Pointing Interface", Vision Interfaces and Systems Laboratory Technical Report VISLab-94-001, Univ. of Ill. at Chicago, October, 1994 FingerMouse: Hands-free/Glove-free pointing input: downward-looking cameras (stereo) recognizes pointing finger when held above keyboard. Shift key used for mouse click (one-button mouse). Concept originated with Francis K.H. Quek. [NIST94a] Barkley, John (ed.) "Security in Open Systems", National Institute of Standards and Technology, NIST SPecial publication 800-7, June 1994 Practical guide to Security features and practicies in POSIX (Unix): MAC Mandatory Access Control, DAC Discretionary Access Control, ACLs, Information labeling. Public key cryptography PKI. SQL and network communications security (e.g. weakness of TFTP trivial file transfer protocol), X.400/X.500 standards, APIs for certificate storage and access (compare CryptoAPI, CryptoKI), certificate revocation lists. Mentions ECMA standard 138, "Security in Open System" with attribut certificates. No list of references? (e.g. Multics) [NabeshimaS95a] Nabeshima, Shinji; Yamamoto, Shinichirou; Agusa, Kiyoshi; and Taguchi, Toshio "MEMO-PEN: A New Input Device", Proc. CHI '95, May 7-11, 1995 Denver CO, pp. 256-257 Optical pen stylus digitizer that takes CCD images of physical ink trail of pen near tip, whole image and writing dynamics can be reconstructed from the sequences of images. 80% (40/50) success rate on reconstructing whole images. Difficulty reconstructing long straight lines. Stress sensor (force sensor) for detecting when writing. Sampling/data rate 10 snapshots/second. Compare with Anoto. [NagaokaM95a] Nagaoka, Makoto and Sugiyama, Soka "Information Input Device", United States Patent 5,440,080, August 8, 1995 Transparent X/Y digitizer/touchscreen using transparent electrode wires of ITO indium tin oxide. Stylus/pen uses a plastic film coating to get appropriate writing friction on a metal core stylus. [NationalInstruments94a] National INstruments "LabVIEW Graphical Programming for Instrumentation: Student Edition (Software+Documentation)", National Instruments Corporation, 1994(?)(hardcopy book)(Data CD) LabVIEW Graphical programming language and software for Windows 3.1: Date approximate. Contains Wells book. [NeXT95a] NeXT Computer "The Project Builder Application", NeXT Computer, 1995 Part of NeXT Computer SDK. Overview of GUI tool for building/making applications. There may be an earlier version: this appears to be 3.X circa 1995. refers to bundles (which can be applications or collections of resources), and projects contained within other projects. [NeXT95b] NeXT Computer "The Interface Builder Application", NeXT Computer, 1995 Part of NeXT Computer SDK. Overview of GUI tool for building GUI user interfaces. There may be an earlier version: this appears to be 3.X circa 1995. See also "Working with Interface Builder": build UI from palette of icons/views/components. [NicolsonEJ94a] Nicolson, Edward John "Tactile Sensing and Control of a Planar Manipulator", Ph.D. dissertation, Decpt. of EE/CS, U. California at Berkeley, 1994 Tactile pressure/force sensing skin on cylindrical fingers, and spacing and depth of sensors to "feel" shape of objects with fingers: multi-touch. Strain values used to determine radius of curvature of indentations in elastomeric/rubber skin. Compare with M.I.T. "Teddy"? Review of capacitive, conductive rubber, piezoresistive, electrorheological, and ultrasonic skin sensors that can be constructed in arrays. Design useds capacitive sensors measuring changes in plate separation. [NovakJL94a] Novak, James L. and Wiezer, James J. "Non-contact Capacitance Based Image Sensing Method and System", United States Patent 5,281,921, January 25, 1994 Electrostatic/mutual capacitance imaging proximity sensor for objects: single pair of electrodes to sense distance, pair of electrodes moved over object in a known plane (or otherwise)



[OSHistory94a] www.os-history.de "OS-History: Die Geschichte von Microsoft", www.oshistory.de/betas/winpad.html, fetched 2014 History and screenshots fro WinPAD Beta, precursor to Windows CE. Stylus on handheld computer, electronic ink annotation. Manual change of orientation/rotation. [PLDOS95] www.pldos.pl "PC Dos 2000 Details", www.pldos.pl/pcdos General information on PC DOS and PenDOS: mentions "mouse emulation" in PenDOS, separate from tablet/stylus driver: limited use of Gestures in pen-aware Mouse applications. (Web site partially in Polish) [Panasonic94a] Panasonic "User's Guide CF-V21P Series Personal Computer", Matsushita Electric Industrial Company, 1994 Pen Computer for Windows for Pen Computing. Special VGA display drivers (for inking functionality?) for screen rotation at 0,90,180,270 degrees orientation. [PaulLG95a] Paul, Lauren Gibbons "Keeping secrets", Ziff-Davis Publishing Company, PC Week Press report on trade-secret theft lawsuit between VMI and Autodesk, concerning Otto Berkes, later director of UMPC/Origami project at Microsoft [PenComputing95a] Pen Computing Magazine "Gesture Mosaic: Newton Software Review", Pen Computing Magazine, Issue #6 August/September 1995 Review of Mosaic Input, Zone-based handwriting recognition using figure-eight overlay. Awkward to learn, vendor claims 40 words per minute input rate with practice. [PenComputing95b] Pen Computing Magazine "IBM ThinkPad 730TE: Product Review", Pen Computing Magazine, Issue #7 November/December 1995 Pen tablet PC using Wacom digitizer. Mentions "Dynapad" by Alan Kay: Mistaken reference to Dynabook? Provided with choice of PenPoint, PenDOS, Pen for OS/2 Warp, PenRight!, or Windows for Pen Computing 1.0. Uses ThinkWrite handwriting recognition instead of Microsoft MARS recognizer. Windows has option in Control Panel to set screen orientation, but off-screen controls more suitable to portrait mode orientation. [PenComputing95c] Pen Computing Magazine "Casio DT-9000 Product Review", Pen Computing Magazine, Issue #7 November/December 1995 Casio PDA: does not mention pen or handwriting input. "handy terminal" vendor description. Comparison with Panasonic JT-900PT. Includes printer. [PenComputing95d] Pen Computing Magazine "Pen Computing and Windows 95", Pen Computing Magazine, Issue #7 November/December 1995 Windows for Pen Computing version 1.0 not easily upgradeable to Windows 95 -- now refereed to as pen service Windows for Pen. Windows for Pen Computing development environment: Visual Basic with version 2.0 pen services penctrl.vbx updated to pen2ctrl.vbx. [PenComputing95e] Pen Computing Magazine "Programming Visual Basic for Pen Input", Pen Computing Magazine, Issue #6 September 1995 Mentions HEDIT and BEDIT (boxed edit) handwriting input controls shipped with Visual Basic. Windows for Pen Computing development environment: Visual Basic with version VB3.0/Pro with provided HEDIT and BEDIT controls. [PenComputing95f] Pen Computing Magazine "PenRight! for Windows", Pen Computing Magazine, Issue #5 June/July 1995 PenRight by Philip Shoemaker: MSDOS, MSDOS DPMI, and MSWindows pen-enabled platforms. Includes cross-platform support for Graffiti. Windows for Pen Computing development environment: PenRight! for Windows with integrated graphical environment, update to earlier PenRight! Pro SDK. Cross-platform to GEOS (MSDOS based), Casio Z-7000 and Tandy Zoomer PDAs. States that Palm Computing has a handwriting recognizer for MS Windows (Graffiti), but not shipping. [PenComputing95g] Pen Computing Magazine "HOPE! Development Environment", Pen Computing Magazine, Issue #7 December 1995 HOPE! for porting applications from Visual Basic to a PDA that does not run Windows: emulated Windows user interface. Pen Computing development environment: supports Casio Z-7000 PDA and Casio PA100 pen computing products, which run GEOS OS. [PenComputing95h] Pen Computing Magazine "First Look: Newton OS 2.0", Pen Computing Magazine, Issue #7 December 1995 Description of Newton OS NOS ("Dante") 2.0. Mentions selection and dragging of one icon, or of multiple icons. Feature also in Windows desktop? [PenComputing95i] Pen Computing Magazine "Motorola Forte Mighty pen computer with tightly integrated wireless capabilities", Pen Computing Magazine, Issue #7 December 1995 Windows for Pen Computing with integrated wireless radio. Handwriting recognition by ART, or cursive recognition by Lexicus. [PenPoint94a] PenPoint "Video: 1994 Demo 1 PenPoint for ATT/EO", wanderbookman.com, posted September 25 2008 on YouTube.com (three parts)



1994 Video: Demonstration of PenPoint / EO unit. YouTube comments mention direct manipulation and gesture UI, also demonstration of convertible tablet computer before Microsoft did a convertible. Shows landscape mode versus portrait orientation/rotation [PenWindows95a] PenWindows: Microsoft Corporation "Pen Computing and Windows 95", Pen Computing, Number 7, December 1995 Windows for Pen Computing version 1.0 versus version 2.0, boxed edit control, hedit handwriting edits Lisa Sampfli, PenFact Inc. in Boston Massachusetts [PenWindows95b] PenWindows: Microsoft Corporation "Programmer's Guide to Pen Services for Microsoft Windows95", Microsoft Press, ISBN 1-55615-835-1 (electronic book) Gestures p. 61: Circle gestures with letter: Lasso: p. 481 shows list of circle gestures by shape, using mnemonic characters -- compare to "recognition macros" of PenCept; HPENDATA object holds chunks of electronic ink data, separate from mouse messages. WM_LBUTTONDOWN / IsPenEvent() / DoDefaultPenInput: electronic ink data is associated as extra data (HPENINFO) attached to a mouse event. Defines stroke pen data between two pen tip transitions, hence touching strokes and in-air/proximity strokes. (What about out of proximity?) Context menu in bedit (boxed handwriting edit) controls. Single tap and double tap on character bring up a (context) menu/list of alternative characters/words. Page 41 portrait/landscape rotation of orientation mode [PetrieH95a] Petrie, Helen; Morley, Sarah; and Weber, Gerhard "Tactile-based Direct Manipulation in GUIs for Blind Users", CHI '95, Companion Proceedings, May 7-11, 1995, pp 428-429 (video presentation) Tactile output of GUI for the blind: two-line Braille Display can be moved around screen image in discrete steps (no mouse), image on screen translated to Ascii/Braille characters: brackets, etc. Routing button "drops" mouse cursor at current location of Braille display in window. [PierK94a] Pier, Ken and Landay, James A. "Issues for Proximate User Interfaces", Xerox PARC Technical Report, April 19, 1994, available via internet User-interface for large surfaces needs UI without static-placed element (e.g. pull-down menus). Pop-up pie menus, hierarchical menus, pop-up handwriting recognition pads, gestures, vocal user interface. Problems studied in example of Xerox Liveobard whiteboard system. Cites pop-up menus to movable menus of Wiseman, Xerox Alto 1976. PenPoint, GO, PenWindows, Momenta. [PiwetzC95a] Piwetz, Christian; Eiffert, Frank; Heck, Helmut; and Mueller-Clostermann, Bruno "An Adjustable User Interface Providing Transparent Access to Application Programs for the Physically Disabled", ACM SIGCAPH Newsletter, January 1995 Graphical/virtual/synthetic keyboard on-screen keyboard on Apple MacIntosh, usable with mouse without physical keyboard. Also includes predictive type-ahead. Main use is as accessibility technology, combined with dwell-mouse for handicapped use. [PoonA95a] Poon, Alex; Weber, Karon, and Cass, Todd "Scribbler: A Tool for Searching Digital Ink", Companion Proc. CHI '95, May 7-11, 1995, pp. 252-253 Wacom tablet integrated into MacIntosh (?). Indexing of electronic ink notes (automatic concordance) by ink matching of stroke groups separated by distance, then matched by dynamic time warping. Both text (handwriting) and special symbols (weather maps) -- no reference to indexing of sketch input. Used as indexing/annotation method in Marquee video logging. [Poyner94] Poyner, Rick "Wintab Backgrounder", LCS/Telegraphics, Cambridge Massachusetts, 1994 Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages. Wintab also mentioned in Windows95/W4PC/TabletPC documents. Also exists in version from about 2003 with minor updates from Wacom, later version 2010 (see file) [Primax95a] Primax Electronics Ltd. "Primax DataPen Personal Text Scanner", DataPen, Primax Electronics Ltd. (physical device) DataPen hand-held text scanner. Functional, in original packaging. Compare with Anoto pen. [Primax95b] Primax Electronics Ltd. "DataPen User's Guide", Primax Electronics Ltd. Cited in Arai 1997 paper for hand-held optical digitizer: appears to be a text scanner. No copy, file is alternative references. [Psion94a] Psion Teklogix "Workabout Pro MC hand-held computer Quick Start Guide", Psion/Teklogix, 1994 Hand-held computer -- format is like hand-held terminal, running Windows CE, touchscreen used as mouse replacement. [QuekF95a] Quek, F.; Mysliwiec, T.; and Zhao, M. "FingerMouse: A Freehand Pointing Interface", Proceedings of the International Workshop on Automatic Face- and Gesture-Recognition, Zurich, Switzerland, June 26-28, 1995, pp. 372-377. Also VISLab Report: VISLab-95-02 (abstract only)



Optical image detection of finger pointing instead of mouse, using color mapping for detection. Mouse click by tapping shift key on the keyboard. Pointing mode for finger tracking by recognizing "pointing gesture" of hand. [Quinnell95] Quinnell, Richard A. "Cover Story: Touchscreen Technology", EDN Magazine, November 9, 1995. Available from http://www.end.com/archives Technology review of touchscreens: flexible resistive film (a la Elographics), Capacitive, Capacitive zone, SAW surface acoustic wave or SAW guided acoustic wave, IR infra-red touchscreens, strain gauge. Refers to problems with capacitive touchscreens. Mentions that acoustic SAW screens can sense multi-touch more than one point of contact. Capacitive proximity sensing (Microtouch zones). Does not use the word tablet. Cites Scriptel/Symbios electrostatic tablet with proximity sensing. Describes fabrication expense of rearmounted electromagnetic digitizers. Electronic ink in windows 95. [RSA95a] RSA Laboratories "PKCS #11: Cryptographic Token Interface Standard, Version 1.0", RSA Laboratories Technical Note, Version 1.0, April 28, 1995 PKCS11 API standard Version 1.0 for smartcards and other cryptographic tokens: single-user, mostly portable devices. Object-based in "C". [RankinR94a] Rankin, R. "Accessing the Internet by E-mail", UseNet FAQ, 1994 What it says: publics FTP servers on internet accessible by eMail. [RedmayneDV94a] Redmayne, Derek Vincent "Extended Touch Screen", PCT Patent Application WO 94/22069, February 25, 1994 Touchscreen sensor extends past edges of display for fixed buttons or keyboard. Touch sensor can be opaque on edges/margins. Compare with Wang Freestyle "The Guide"? [RegisterDS95a] Register, David S. "Stylus Operable Computer with Wireless Keyboard in Storage Bay", United States Patent 5,440,502, August 8, 1995 Handheld computer with slide-out/detachable keyboard: Mentions Momenta [RubinSM94] Rubin, Steven M. "Computer Aids for VLSI Design, Chapter 10: Human Engineering", Web reference at www.rulabinsky.com/cavd/text/chap10-6.html Bibliography only: contains reference to Ledeen Recognizer in Newman/Sproull textbook, other interactive editors (without character recognition): Application GUI using Ledeen recognizer [RuggedWare94a] RuggedWare " Using Pen on the Road: Apparel Version 2.0 Sales Automation Software", RuggedWare Ltd., Denver Colorado, 1994 Sales Automation application using Windows-based pen computer. Input into "letter guides", boxed character recognition input, makes distinction between character cells which are empty, and those which contain a space. Compare with Ward patent on simulated keyboard input using handwriting input. [RusD94a] Rus, Daniela and Subramanian, Devika "Designing structure-based information agents", Report SS9403-012, Working Notes of 1994 AAAI Spring Symposium on Software Agents, pp. 79-86 Automatic software agents recognize text, but also figures, tablet, paragraphs of interest, via detectors and add information links (actions). [SachGM95a] Sach, Gary M., Bromage, Bruce K. and Purpura, Don F. "Touch Control Panel", United States Patent 5,398,045, March 14, 1995 Touch panel displaying different pop-up menus depending on where it is touched: entry is selected by separate button when desired item is displayed. [SakamotoH95a] Sakamoto, Hiroyuki "Handwriting input apparatus for inputting handwritten data from unspecified direction", United States Patent 5,389,745, February 14, 1995 Rotatable handwriting input widget/panel so that user can write/input from various directions, separate from orientation of touchscreen display. Compare with SIP panel? [SakamotoK94a] Sakamoto, Kohichiro; Murashima, Yoshiyuki; Nishida, Hiroshi; and Shibata, Yukihiro "Data Processor with Rotatable Display", United States Patent 5,329,289, July 12, 1994 Display can be rotated to laterally elongated position (landscape mode) or vertically elongated (portrait mode), control unit sets display format for windows. Saves orientation information when turned off. Compare with Radius Technology, Masimo 93? [SantiniS94a] Santini, Simone; Sanz, Jorge; and Del bimbo, Alberto "Shape Matching by Elastic Deformation", Tech. Rp. UILU-ENG-94-2212, March 1994 Dept. of Elec. and Comp. Eng., U. Illinois at Urbana-champaign Elastic matching (degree of one-dimensional deformation) with template for OCR optical character recognition. Mentions real-world preprocessing problems of difficult binarization, difficult separate from background. Samples taken from standard NIST OCR database. [SatoF95a] Sato, Fumitaka and Kishida, Ken "Data processing apparatus for editing image by using image conversion", United States Patent 5,428,721, June 27, 1995 Trapezoidal/rotational editing of part of a picture by defining four points of a trapezoid, and moving the four corners. [SatohY95a] Satoh, Yasuhiro; Kiyosawa, Yoshiyaki; and Kimura, Yuichi "Pen Input Device", United States Patent



5,444,193, August 22, 1995 Optical mouse in pen/stylus form: light sensor in stylus, two colors of lights (or two filters), surface has grid of horizontal lines and vertical lines of different colors so that stylus/mouse can detect movement in X and Y by counting lines. States that can be used for writing (unlike mouse) because can detect movement in proximity, like many tablets. [SaundE94a] Saund, Eric and Moran, Thomas P. "A Perceptually-Supported Sketch Editor", Proc. UIST '94 (preprint) Parsing of informal structure in electronic ink sketching in an electronic blackboard (whiteboard) system. [SavoyRL94a] Savoy, Robert L. and Hill, Winfield "Keyboard having touch sensor keys for conveying information electronically", United States Patent 5,341,133, August 23, 1994 Touch sensors on keys of keyboard: user gestures (combinations of swipes) on space bar to enter/leave graphics/pointing mode for other mode. Mentions using two pointing keys for two hands: compare to Buxton bi-manual systems, multi-touch. Handicapped use by operating sensors with tongue. Grid array of sensors, scanned by rows and columns: optical touch sensor measures proximity of reflected light from small emitters: interpolation of signal intensity for higher resolution. [ScattolinP95a] Scattolin, Patrice "Recognition of Handwritten Numerals using Elastic Matching", Master's Thesis, Concordia University, Montreal Canada, November 1995 Elastic matching (see Tappert) applied to OCR of handwritten numerals, after thinning and line tracing. [SchneidermanR94a] Schneiderman, Ron "Wireless Personal Communications: The Future of Talk", IEEE Press, 1994 (hardcopy book) Business tutorial on wireless, cellular, and other technologies: includes notes on EO/Hobbit pen computing, IBM Simon, PDAs with wireless communications [SchneierB94a] Schneier, Bruce "Applied Cryptography: Protocols, Algorithms and Source Code in C", John Wiley and Sons, 1994 (hardcopy book) Textbook, historical reference and tutorial on cryptography: includes C source code on MD5, IDEA, DES, etc. Listing of patents p. 447 for RSA and Diffie-Hellman key exchange [SchomakerL95a] Schomaker, L., Nijtmans, J., Camurri, A., Lavagetto, F. Morasso, P., Benoit, C., Guiard-Marigny, T., Le Goff, B., Robert-Ribes, J., Adjoudani A., Defee, I., Muench, S., Hartung, K., Blauert, J. "A Taxonomy of Multimodal Interaction in the Human Information Processing System: A Report of the ESPRIT Project 8579, in "Multimodal Integration for Advanced Multimedia Interfaces" (MIAMI project)", http://hwr.nici.kun.nl/~miami/taxonomy/taxonomy.html [email protected]: long bibliography of pen/handwriting input and other GUI technologies [SchwarzPM94a] Schwarz, Peter M. and Shoens, Kurt A. "Managing Change int he Rufus System", Proc. 10th Intl' Conf on Data Engineering, IEEE, 1994, pp. 170-179 Rufus language: Indexing of document (for retrieval) by parsing for implicit or semantic structure in the document: d.g. Jimmy Carter in Title, rather than Author or mentioned in passing. [Scriptel95a] Scriptel Corporation "WriteTouch: Digital Pen and Touch Technology Overview", Scriptel Corporation, 1995 (date approximate) Electrostatic/capacitive digitizer with cordless stylus or finger touch: mentions shielding between LCD and sensor (front-mounted), greater optical transmissivity due to optical properties of ITO shielding compared to touchscreens. Cordless stylus requires battery. References Windows 95 with Pen Extensions (Pen Windows). Joint product with NCR? (Date approximate) [Scriptel95b] Scriptel Corporation "Scriptel Digitizer for Pen-Based Computers", Scriptel Corporation, 1995 (date approximate) Electrostatic/capacitive digitizer with cordless stylus or finger touch. Describes capacitive sheet digitizer as "non-contacting resistive digitizer". Has good simplified description of resistive sheet tablets, ratiometric / normalizing coordinates. (Date approximate) [Scriptel95c] Scriptel Corporation " Symbios Logic And Scriptel Announce WriteTouch, The Only Cordless Digital Pen Product Enabling Both Pen And Finger Touch Input; Write Touch Allows Computer Manufacturers to Add Pen And Touch Input Capabilities to Portable Systems; WriteTouch and Windows 95 Expected to Enable Robust Pen Computing for Horizontal and Vertical Markets", Scriptel Corporation, July 3, 1995 Symbios manufacturing hardware controllers for Scriptel transparent tablet digitizers: Cites advantages of proximity sensing, no electromagnetic interference, cordless stylus over other PDA touchscreen technologies. Simulationeous pen and finger-touch detection: multi-touch. [SeilerWJ94a] Seiler, William J. "Method and apparatus for brightness and contrast adjustment", PCT Patent Application WO 1994012971 A1, June 8, 1994 Pointer device (touchscreen/stylus) movement in different directions to adjust contrast up/down and brightness up/down. Compare with marking menus? Initiated by key, or active while a button is held down. [Sekendur95] Sekendur, Oral F. "Optical position determination", United States Patent 5,477,012, December 19,



1995

Optical digitizer on "smart paper", with a (passive) pattern printed on the paper -- Compare to Anoto technology. Appears to use a something similar to a circular barcode / ringcode: provenance not given, might be interesting to find. Digitizer tablet "smart paper" that puts a printed code on paper, position is read by optical sensor in the stylus tip. Compare with technology by Anoto et al. Shows circular barcode optical target, doubled 1d Barcode target, square matrix target for "smart paper" Two styli pictured, no parallax correction (?) to optical image in optical digitizer [SelkerEJ94a] Selker, Edwin Joseph "Tactile feedback mechanism for cursor control", European Patent Application 93120305.3, December 16, 1993 Haptic/tactile feedback to a mouse, joystick, or other pointing device based on storing the window handle or other information in the input device and generating the feedback locally: faster than basing it on the information sent to the host [SenooS95a] Senoo, Seiichi; Ueno, Yukihiko; and Hama, Kentaro "Word Processing Apparatus with Handwriting Input Functions", United States Patent 5,488,475, September 5, 1995 Handwriting recognition on tableet with editing: editing symbols == editing gestures for text. Handwriting edit gestures of down-left, pigtail, etc. -- compare with PenPoint and Pen Computing gestures. [SerranoJA95a] Serrano, J. Artur "The Use of Semantic Constraints on Diagram Editors", Proc. of VL '95, September 1995, Aarmstadt, Germany Visual programming: Semantic constraints may interfere during editing of diagrams / sketch editing / Visual languages, as editing may leave state inconsistent: proposes incremental validation of constraints. [ShaferD94a] Shafer, Daniel G. "The Power of Prograph CPX", Prograph International, 1994 (hardcopy book) Prograph graphical programming language. [Sharp94a] Sharp.co.jp "Sharp Electronic Organizer Model IQ-8900/IQ-8920 Operation Manual", Sharp, 1994 Electronic organizer (PDA) with full keyboard, touchscreen. Fax modem. Touchscreen four-corner calibration. Touchscreen used as mouse (menus etc.), electronic ink. Some touchbuttons duplicate keyboard functionality e.g. "Store" button same as "Enter" key. Pop-up onscreen numeric keypad. VT-100 terminal emulation. Mentions emulation of PDA on PC display. [Sharp95a] Sharp.co.jp "Sharp IQ-9200 PDA product information", Sharp IQ-9200 product information -- multiple sources Sharp IQ-9200 PDA, dates approximate (1995)? has full keyboard in clamshell design, touchscreen with stylus for sketch input. [ShipmanFM95a] Shipmann, Frank M. III; Marshall, Catherine C; and Moran, Thomas P. "Finding and Using Implicit Structure in Human-Organized Spatial Layouts of Information", Proc. CHI '95, pp. 346-353 "spatial parser" to attempt to recognize implicit structures in how people lay out information: example is paper notecards, organized as horizontal and vertical lists, stacks, composite (list with additional "title card"), heaps. Compare with visual languages and 2D/two-dimensional parsing of mathematical expressions. [SiioI95a] Siio, Itiro "InfoBinder: A Pointing Device for a Virtual Desktop System", Proc. HIC '95, July 1995, pp. 261-264 In a Wellner-type optical/augmented desk / touchscreen, a tag with a button and an LED flasher. The LED position can be determined in the camera image, the LED is modulated to send a recognizable ID, and the button can be used as user input. Used to tag physical objects so that they can have augmented virtual capabilities (e.g. tag on phone ties it to on-line dialer director). [SiliconGraphics94a] Neider, Jackie; Davis, Tom; and Woo, Mason "OpenGL Programming Guide -- The Official Guide to Learning OpenGL, Release 1", Addison-Wesley Publishing Company, first printing 1993, Copyright 1994 First (?) version of OpenGL graphics library: includes subpixel drawing, alpha blending for transparency., Screen-door transparency == cheesy transparency [SilverbrookK95a] Silverbrook, Kia "Method and apparatus for providing transparency in an object based rasterized image", United States Patent 5,428,724, June 27, 1995 Using pixel data to render transparent/overlay images in windows and objects, according to Z level of the windows or objects. [SinclairMJ95a] Sinclair, Michael J. "Video based 3D tactile reconstruction input device having a deformable membrane", United States Patent 5,459,329, October 17, 1995 Touch sensor that makes 3D images of surfaces: rather than a thin sensor membrane, uses a tank or container filled with a liquid with a conformable membrane on top: pressed on an object, membrane with liquid conforms to shape of surface, video camera captures image of reflected light depending on angle of incidence with the (conformed) surface. [Sklarew94a] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 5,297,216, assigned to AST Research, Inc., Irvine, California, March 22, 1994 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and



learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" Forms entry in handwriting recognition user interface, where a data-entry form field window pops up for entry, after entry of a pre-determined command using command/gesture symbols [Sklarew94b] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 5,365,598, assigned to AST Research, Inc., Irvine, California, November 15, 1994 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" Sklarew/Grid very specific to resistive digitizer, no proximity sense (?) [SmithAR95a] Smith, Alvy Ray "Image Compositing Fundamentals", Microsoft Research Technical Memo 4, August 15, 1995 Tutorial on pre-multiplied and not pre-multiplied alpha blending, producing a shaped image with partial transparencies. Treatment of "over" operator (transparency) in Porter paper of 1984. Alpha is a fourth value "transparency" or "opacity" in addition to RGB values. Discussion of pixels modeled as points, or as square windows to which an object may partially contribute by crossing part of the square. [SmithAR95b] Smith, Alvy Ray "A Pixel is Not a Little Square and a Voxel is Not a Little Cube", Microsoft Research Technical Memo 6, July 17, 1995 A pixel is a point sample, not a square or rectangle. Must represent contribution of all graphics that are behind it: may be overlapping circle with region of adjacent pixels, or of broader nearby pixels for reconstruction of image. Argument applies also to scanning and printing. Should not refer to shape of pixels (e.g. rectangular, non-square) but to non-square/non-uniform spacing of pixels. [SmithAR95c] Smith, Alvy Ray "Alpha and the History of Digital Compositing", Microsoft Research Technical Memo 7, August 15, 1995 Alpha blending: integral alpha, premultiplied alpha, digital printing, graphical compositing, matte / masking / matting blending. Traces history to analog-film compositing using a physical matte to combine images. History of blue-screen/green-screen matting, Petro Vlahos. Digital compositing an obvious simulation of known analog-film techniques, but digital matting (using alpha blending) not obvious. [SmithJR95a] Smith, Joshua Reynolds "Toward Electric Field Tomography", Master Thesis, M.I.T. Media Arts and Sciences, August 1995 General review of electrostatic/capacitive sensing, from simple proximity detection up to 3D and beyond. How to determine position using sensors in any particular configuration -- basically calculating or determining a correction table or formula to get the position of the object from the mix of sensor values, for whatever configuration you know the sensors are in. No references to commercial products e.g. Scriptel, academic references. [SmithRB95a] Smith, Randall B.; Maloney, John; and Ungar, David "The Self-4.0 User Interface: Manifesting a System-wide Vision of Concreteness, Uniformity, and Flexibility", Proc. OOPSLA '95, Austin Texas, 2005, pp. 4760 Self-4.0 user interface: object-oriented programming originated in attempt to make software model realworld behaviors, need to apply those principles more closely. reification of layout constraints (automatic layout) for user interface, composition through embedding. Software can be modified while running. Slots for actions on an object. Graphical objects referred to as morphs. Cites to Visual Basic, but not to PenApps by Slate. [SmithWR94a] Smith, Walter R. "The Newton Application Architecture", Proc. 39th IEEE Computer Society International Conf., San Francisco 1994, pp 156-161 Description of Newton application and OS architecture: slots, packages, interpreted bytecodes, no separate address spaces for different applications. Describes Newton as simplified version of "Self", frames take place of Self objects. Defines View system (Smalltalk 80), and MacApp. Newton applications delivered in packages, containing parts dispatched to the appropriate part of the OS when installed: e.g. application part, font use by application, special communications driver. Pages in virtual memory are compressed, and decompressed when loaded. Picture from Newton Toolkit: cites MacApp for views. [SmithWR94b] Smith, Walter R. "SELF and the Origins of NewtonScript", PIE Developers magazine, July 1994 Discussion of NewtonScript for the Newton pen-computing PDA, based on prototype-based inheritance language SELF. Note: NewtonScript adapted from SELF's "parent slots". Self based on SMALLTALK. [SmithWR95a] Smith, Walter R. "Using a Prototype-based Language for User Interface: The Newton Project's Experience", SIGPLAN Notices, Vol 30 No 10, Oct 1995, pp 61-72 NewtonScript prototype-based language for Newton development: describes historical antecedents. Visual Pen development environment for Newton Toolkit: view system and collection of slots. Browser weak at supporting non-UI code, really just a "frame hierarchy" editor. Inheritance (packages?) inspired by HyperCard. Also cites to Bay-Wei Change, 1993, Smalltalk-80, Self [SochaJ94] Socha, John and Hall, Devra "teach yourself ... Visual Basic 3.0", MIS Press, 1994 (hardcopy book) Tutorial on Visual Basic language and development environment



[SoftwareIndustryReport95] SoftwareIndustryReport "New cordless digital pen product enables both pen and finger touch input", Software Industry Report, Millin Publishing, July 24, 1995. Available at www.allbusiness.com/technology WriteTouch electrostatic/capacitive digitizer from Scriptel supports either finger or stylus input: can do multi-touch? Announcement of cordless electrostatic digital pen using Symbios Logic mixed signal design with Scriptel. [Sony94a] Sony Corporation "Sony Magic Link PIC-1000 product information", Sony Corporation References, Product information on Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator": hands-free cellphone option, touchscreen, but virtual keyboard only, no handwriting recognition. Magic Cap OS from General Magic. [Sony94b] Sony Corporation "Sony Magic Link User's Guide PIC-1000 Personal Intelligent Communicator", Sony Corporation Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator" [SoukoreffRW95a] Soukoreff, R. William and MacKenzie, I. Scott "Theoretical Upper and Lower Bounds on Typing Speed Using a Stylus and Soft Keyboard", Behaviour and Information Technology, Vol 14, 1995, pp. 37-379 Study and statistical model (based on frequency digrams) of virtual keyboard typing speed on touchscreen with stylus. States more accurate and faster (overall) than handwriting recognition. QWERTY keyboard. Notes slipperiness of touchscreen, visual parallax. No citations to commercial pen computing systems (PenPoint, PenWindows, etc.) [SpeeterTH95a] Speeter, Thomas "Intelligent Work Surfaces", United States Patent 5,479,528, December 25, 1995 Force/touch touchpad sensors on desktop/tabletop, may be used to create virtual input devices (virtual keyboard, menu, tablet, etc.), sense location of physical objects (phicons?). Identify/authenticate user by force-image/biometrics of palmprint, etc. Instrument/augment objects such as locks, floor. [SpencerD94a] Spencer, Donald "Webster's NewWorld Dictionary of Computer Terms, Fifth Edition.", Macmillan, 1994 (hardcopy book) Technical dictionary of computer terms 1994 [StahovichTF95a] Stahovich, Thomas F. "SketchIT: A Sketch Interpretation Tool for Conceptual Mechanical Design", Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, Massachusetts, May 1995 Sketch recognition for mechanical design: uses models of physical/mechanical constraints to produce actual designs from sketch. States modifies sketch until a design that functions in a specified manner. Cited in Arvo00 [SteinRA94a] Stein, Robert A.; Logan, James D.; and Woringer, Charles A. "Position Sensing Computer Input Devices", United States Patent 5,365,461, November 15, 1994 Electrostatic resistive sheet touchscreen digitizer, can sense both a stylus by a particular signal and a finger touch by capacitance at the same time: e.g. write with stylus while selecting menu with finger. Gives formula for shift and scale correction/calibration, but not quadrilateral calibration correction. multi-touch capable? [StoneMC94a] Stone, Maureen C.; Fishkin, Ken; and Bier, Eric A. "The Movable Filter as a User Interface Tool", Proc. CHI '94, pp. 306-312 Semi-transparent "filter" viewer window can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text. [StoneMC95a] Stone, Maureen C. and DeRose, Anthony "Method for operating on objects in a first image using an object-based model data structure to produce a second contextual image", United States Patent 5,467,441, November 14, 1995 Semi-transparent "filter" viewer window has VOR "Viewing Operation Region", can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text. [StoneMC95b] Stone, Maureen C.; Bier, Eric A.; Fishkin, Kenneth P.; and DeRose, Anthony "Method and apparatus for producing a composite second image in the spatial context of a first image", United States Patent 5,479,603, December 26, 1995 MagicLens: visual transformations of part of a lower window when a second window is over it. Gives the effect of the second window being a lens: e.g. shade the color, magnify, show hidden lines/wirefram, show streets on a map, etc. [StoneTL94a] Stone, Thomas L. "Stylus switch status determination in a digitizer having a cordless stylus", United States Patent 5,369,227, November 29, 1994 Cordless digitizer pen, sense stylus buttons by button changing resonance frequency of stylus. [StreitzNA94a] Streitz, Norbert A.; Geissler, Joerg; Haake, Joerg M.; and Hol, Jeroen "DOLPHIN: Local and Integrated Meeting Support across Remote Desktop Environments", Proc. ADM CSCW '94, October, 1994, Chapel



Hill NC, pp. 345-358 Remote whiteboard touchscreen system combining LiveBoard and networked personal computers. [SunSoft94a] SunSoft / Sun Microsystems "OpenWindows Reference Manual, release 3.4", Sun Microsystems / SunSoft, 1994 Open Windows / X Windows reference manual: includes vkbd(1) virtual keyboard/soft function keys display manager: display labels of function keys on screen with application-dependent labels automatically -- can use physical function keys or on-screen virtual keys input area. [SunSoft95a] SunSoft / Sun Microsystems "Solaris X Window System Developer's Guide", Sun Microsystems / SunSoft, 1995 Open Windows / X Windows reference manual [TamaruH95a] Tamaru, Hideshi; Togawa, Kazuo; Sakano, Akio; and Yoshida, Kimiyoshi "Touch Panel Apparatus", United States Patent 5,414,413, May 9, 1995 Optical touchscreen, LED light emitters and photodetectors are arranged diagonally instead of X/Y. Light intensity reduced on corner emitters to reduce reflections. [TapiaMA95a] Tapia, Mark A. and Kurtenbach, Gordon "Some Design Refinements and Principles on the Appearance and Behavior of Marking Menus", Proc ACM UIST '95, pp. 189-195 Marking menus with stylus: animation feed back to user if mark (inadvertently) triggers a command gesture. Compare with press-and-hold, and stroke if do not wait. [TaylorR95a] Taylor, Robert "Requirements and Availability of Application Programmer's Interfaces for VirtualReality Systems", UNC Technical Report TR95-009, April 1995 Aviary system: Re-mapping of UI events (mouse button clicks) to other application events (or system events). Compare to press-and-hold event mapping in later Windows version. [Tognazzini94a] Tognazzini, Bruce "The "StarFire" Video Prototype Project: A Case History", Proc CHI '94, Boston Massachusetts, pp. 99-105 Report on StarFire video project, showing speculative multi-touch user interfaces. Primary concerned with virtual video prototyping by means of "special effects" dramatized video production, which forces one to consider actual use cases in the course of building a shooting script for a film. [Tognazzini95a] Tognazzini, Bruce "Tog on Software Design", Addison-Wesley, Pub. December 1995 (hardcopy book) Essays on user-interface design, mentions 1994 "StarFire" video speculating on future touch-input portable computers, including a demonstration of a multi-touch (bimanual/two-hand) zoom gesture. Discussion on handwriting only mentions Graffiti by Palm. On touchscreens, states that poor accuracy (or visual parallax) causes users to slow down in an attempt to be more accurate: speculates on fiber-optic coupling plate (?) to reduce visual parallax. [Toshiba94a] Toshiba Corp. "Dynapad T200 Series Pen Computer User's Manual", Toshiba Corp., 1994 Hardware user's manual for Toshiba T200 and T200CS pen computers -- mentions screen rotation, handwriting, etc. but does not name the operating system (PenPoint or Windows for Pen Computing). [Toshiba95a] Toshiba Corp. "Toshiba T200 80 PenWindows 1.0", T200 PenWindows, Toshiba Corp., 1995 (physical device) Physical operational unit of Toshiba pen computer running Windows for Pen Computing 1.0 on top of MS/DOS 6.22. Photographs of screen rotation between landscape and portrait orientation, on-line gesture tutorials (with examples), Pen Palette pop-up handwriting bar for text entry, virtual keyboard in multiple configurations. SideKick 2.0. Base is Windows 3.1, with pen extensions. [Toshiba95b] Toshiba Corp. "Toshiba T200 CS 80 PenWindows 1.0", Toshiba PenWindows, Toshiba Corp., 1995 (physical device) Similar unit to Toshiba T200 80 Pen Windows 1.0, but color screen (better contrast on physical device and in photographs) [TowRF94a] Tow, Robert F. "Methods and Means for Embedding Machine Readable Digital Data in Halftone Images", United States Patent Patent 5,315,098, May 24, 1994 DataGlyphs: see other references. Compare to Anoto. Shows square matrix optical target, with asymmetric halftone dot patterns in cells of a digital halftone image [VanHulzenJA94a] van Hulzen, J.A., Chair "Annual Report of De Stichting Computer algebra Nederland 1994", De Stichting Computer Algebra Nederland Mentions papers and work direct manipulation / pen / stylus interfaces for mathematical input and editing (pp 27-30) [VenoliaD94a] Venolia, Dan and Neiberg, Forrest "T-Cube: A Fast, Self-Disclosing Pen-Based Alphabet", Proc. CHI94, April 1994, Boston, Massachusetts. pp. 265-270 Marking menu: get a pop-up of a rotary/pid menu with letters/characters, arranged concentrically in multiple rings, or set of characters is magnified in the direction you move the stylus from the center. Hand occlusion / obscuring of menu dealt with by showing pop-up menu at an offset from pen tip. Audio



feedback (clicks) on the flick gestures at start and end. Visual feedback of selected character and idealized form of the flick gesture. Target can also be displayed continually, like a floating graphical keyboard. [VogelL94a] Vogel, Les "Newton Program T-Cube submission", Internet posting 1994: available 2010 from welovegod.org/macos_files/macos_files/Sounds_of_Music/t-cube-10.hqx.desc Installation instructions for Apple T-Cube pie-menu (unistroke handwriting) text input. [WantR95a] Want, Roy; Schilit, Bill N.; Adams, Norman I.; Gold, Rich; Petersen, Karin; Goldberg, David; Ellis, John R.; and Weiser, Mark "The PARCTAB Ubiquitous Computing Experiment", Xerox PARC Technical Report CSL95-1, March 1995 ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. Uses Unistroke character recognition, click/tap gestures on heirarichal menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for left- or right-handed operation. [WantR95b] Want, Roy; Schilit, Bill N.; Adams, Norman I.; Gold, Rich; Petersen, Karin; Goldberg, David; Ellis, John R.; and Weiser, Mark "An Overview of the ParcTab Ubiquitous Computing Experiment", Xerox PARC Technical Report CSL-95-1, March 1995 ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. "Tab" refers to "small tablet computer" -- touchscreen computer. Uses Unistroke character recognition, click/tap gestures on heirarchical menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for leftor right-handed operation. [WatanabeT95a] Watanabe, T. and Fukui, S. "A method for controlling tactile sensation of surface roughness using ultrasonic vibration", Proc. 1995 IEEE Conf. Robots and Automation, Vol 1 pp. 1134-1139 (abstract only) Ultrasonic vibration of a few micrometers amplitude makes surface feel slick, less frictio, due to small air film (hypothesized) under finger. Haptic feedback of stickiness, roughness, small protrusion for touchscreens, other surfaces. [WeberK94a] Weber, Karon and Poon, Alex "Marquee: A Tool For Real-Time Video Logging", Proc. CHI '94, pp. 58-64 Real-time electronic ink annotations while video is being made, uses as keys to video. Intent is to eliminate having to review the video later to log/annotate/index it. User takes notes, can circle a "keyboard" in the notes to use as index. Compare with Anoto smart-pen note taking (combined audio/electronic ink recording). Horizontal scratch-out gesture (not confused with M/W in handwriting recognition): keywords translated later manually by user typing? [WellandR94a] Welland, Robert; Seitz, Greg; Wang, Lieh-Wuu; Dyer, Landon; Harringotn, Tim; and Culbert, Daniel "The Newton Operating System", Proc. 1994 IEEE Computer Conference, San Francisco, 1994 Newton OS, much is on details of memory management Memory space is a single (large) linear address space but mapped in pages, protection among tasks: because not context mapping among multiple address spaces, thread/task switching is very fast, object IDs are just their linear address. (Novel at the time?) downloading of system updates, other software. [WellnerPD94a] Wellner, Pierre David "Interacting with Paper on the DigitalDesk", Univ. of Cambridge Technical Report UCAM-CL-TR-330, ISSN 1476-2986, March 1994 Mentions Freestyle, and asserts that people "like" paper compared to electronic documents. Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Mostly a userinterface experiment in combining electronic simulation with real paper documents. Image subtraction and motion detection to locate hands etc. (Longish discussion of problem using a standard "C" image processing library from Modula, due to garbage collection). [WellsLK94a] Wells, Lisa K. "LabVIEW Graphical Programming for Instrumentation: Student Edition User's Guide", National Instruments Corporation, 1994 (hardcopy book) LabVIEW Graphical programming language "G" for virtual instruments: has simulated mechanical action of boolean switches, both switching and latching (when pressed, when released, until released). Example of Global Switch. [WexelblattA94a] Wexelblatt, Alan "A Feature-Based Approach to Continuous-Gesture Analysis", Master's Thesis, M.I.T. School of Architecture and Planning, Cambridge Massachusetts, June, 1994 About hand and body gestures of the form "put-that-there" gesticulation, rather than handwritten gestures as part of a computer GUI: refers in contrast to pen-based gestures with hand-held objects, instead of open-hand gestures (topic of thesis) [WexelblattA95a] Wexelblatt, Alan "Bibliography of Natural-Gesture Related Research", web.media.mit.edu/~wex/home-gesture.html Bibliography of gesture recognition research: primarily about hand and body gestures of the form "putthat-there" gesticulation, rather than handwritten gestures as part of a computer GUI [WexelblattA95b] Wexelblatt, Alan "Summary of CHI'95 "Gesture at the User Interface" Workshop",



web.media.mit.edu/~wex/CHI95-workshop-writeup.html Panel discussion on gesture recognition research: primarily about hand and body gestures of the form "put-that-there", rather than handwritten gestures as part of a computer GUI. Edward Altman, Jelle Coutax, Sidney Fels, Wen Gao, Nobuo Hataoka, Bernard Hibbits, Volker Kuehn, Christoph Maggioni, Axel Mulder, Polly Pook, Alan Wexelblatt. [WexelblattA95c] Wexelblatt, Alan "An Approach to Natural Gesture in virtual Environments", ACM Trans. Computer-Human Interaction, Vol 2 No 3, 1995, pp. 179-200 Gesture recognition for natural gestures (whole body gesticulation, empty-handed gestures/gesticulation), rather than pen-input gestures or handwriting recognition [WhitakerRD95a] Whitaker, Randall D.; Longinow, Nicholas E.; and McNeese, Michael D. "Group Interfaces: A Profile and a Prototype", Interim Report AL/CF-TR-1995-0069, April 1995, Armstrong Laboratory, Air Force Material Command, Wirte-Patterson AFB, Ohio, Multi-user information system, to use cordless pens, simultaneous tracking of multiple pens: 2..3 users enough in practice. Prototype uses resistive film with corded pen. Text handwriting recognition using CIC MacHandwriter. Wall mounting of prototype more awkward than original MacHandwriter hardware with desktop tablet. multi-touch with styli. [WhittakerS94a] Whittaker, Steve; Hyland, Patrick; and Wiley, Myrtle "Filochat: Handwritten notes provide access to recorded conversations", Proc. CHI 94, pp. 271-277 Simultaneous voice and electronic ink / handwriting recording in portable device: indexing of handwritten notes to audio recording. Compare with Wang Freestyle (cited) and to Anoto SmartPen/LiveScribe (later). [Wikipedia95a] en.Wikipedia.org "Dylan (programming language)", en.wikipedia.org/wiki/Dylan_programming_language: reference fetched in 2009 Competing language and IDE to NewtonScript for the Newton PDA: IDE released as technology release in 1995 [Wired94a] Wired Magazine: Steven Levy "Bill and Andy's Excelent Adventure II: General Magic", Wired Magazine, 1994. Available at wired.com General Magic development of pen-based PDA/pocket digital communicator, using Telescript programming language to implement software agents. Compared to Apple Newton. Bill Atkinson, Andy Hertzfeld. [WolfeA95] Wolfe, Andrew and Barrett, Gary L. "Touch Panel", United States Patent 5,438,168, August 1, 1995 Resistive film digitizer, voltage gradient. Correction for edge / pincushion distortion by using a variety of connectors at the edges, with resistance corrections on the connectors. [WooTYC94a] Woo, Thomas Y.C.; Bindignavle, Raghuram; Su, Shawowen; and Lam, Simon S. "SNP: An Interface for Secure Network Programming", Proc. USENIX Summer Technical Conference, June 1994 Secure network sockets: Secure Network Protocol in distributed security, predecessor to SSL key exchange and public-key encryption [WroblewskiDA94a] Wroblewski, David A.; Hill, William C.; McCandless, Timothy P. "Computer Display Unit with Attribute Enhanced Scroll Bar", United States Patent 5,339,391, August 16, 1994 Scrollbar display highlight marks corresponding to where text had been found in a document. Compare with Russian patent to Gorbunov on the similarl feature. [Xerox94a] Xerox Corporation "The Xerox PARCTAB", Available from nano.xerox.com Xerox PARCTAB palm-sized mobile computer / PDA, worn like a pager. Three function buttons and stylus user interface on display touchscreen. [YeeB94a] Yee, Bennet "Using Secure Coprocessors", Ph.D. thesis, Carnegie-Mellon University, May 1, 1994: also Technical Report CMU-CS-94-149 Secure processors: (co)processors with secure code in ROM, processor packed in epoxy with self-destruct lines, checksum on all loaded code. Compare with smartcards. See also Dyad system (secure DRM). [Zenith95a] Zenith "Zenith CruisePAD", Zenith Corp, 1991-1996 Integrated touchscreen/display device, connects wirelessly to Windows PC (Windows 95, Windows 3.1, PenWindows?). Image from billbuxton.com, 2012. Electronic file has additional references. [ZettsJM95b] Zetts, John. M "Recognizing the Cessation of Motion of a Pointing Device on a Display by Comparing a Group of Signals to an Anchor Point", United States Patent 5,404,458, April 4, 1995 Recognize dwell/press-hold in stylus and distinguish between mouse / gesture / handwriting input mode: press-and-hold [ZhaiS94a] Zhai, Shumin; Buxton, William; and Milgram, paul "The "Silk Cursor": Investigating transparency for 3D target acquisition", Proceedings of CHI '94, April 24-28, 1994, Boston, pp.458-464 Silk Cursor using "silk stocking" effect: partial transparency to simulate (non-continuous) difference in depth instead of stereoscopic views. [ZhaoR95a] Zhao, R.; Kaufmann, H.J.; Kern, T.; and Mueller, W. "Pen-based Interfaces for Engineering Environments", Proceedings of HCI 6th Human-Computer Interaction International, Tokyo, July 9-14. Elsevier Handi: handsketch-based diagram editor in Express-G language, user-defined an pre-programmed



unistroke gestures: one-stroke "T", circle, square, semicircle left/right gestures [ZimmermanTG95a] Zimmerman, Thomas G.; Smith, Joshua R.; Paradiso, Joseph A.; Allport, David; Gershenfeld, Neil "Applying Electric Field Sensing to Human-Computer Interfaces", IEEE SIG CHI May 1995 (Preprint copy) Human touch sensing on touchpad and other objects (furniture, location in room) using capacitance: field shunted to ground by human body, and field transmitted through conductive human body to stationary receivers. General remarks on capacitive sensing in general. Used to implement capacitive touchscreen ("finger-pointing mouse"). Can distinguish multiple persons (multi-touch) biblio93 (Prev page)

Main Page







Main Page


[AgrawalaM97a] Agrawala, Maneesh; Beers, Andrew C.; Froehlich, Bernd; Hanrahan, Pat; McDowall, Ian; and Bola, Mark; "The Two-User Responsive Workbench: Support for Collaboration Through Individual Views of a Shared Space", Proc. SIGGRAPH '97, pp. 327-332 Head-tracking 3D display virtual reality environment, single-finger data glove, projection ensures that when two users have fingertip pointing at same point in virtual environment, their fingers touch at same physical location. Makes comparison to whiteboard systems. [AkamatsuM96a] Akamatsu, Motoyuki and MacKenzie, I. Scott "Movement characteristics using a mouse with tactile and force feedback", Int'l. Journal of Human-Computer Stides, 1996, Vol 45, pp. 483-493 Mouse with tactile and force (haptic) feedback: slight improvement in targetting when mouse has "drag" force in addition to touch sensing ("drag" force by magnetic coupling) [AllardD97a] Allard, David J; Canova, Francis J. jr.; Johnson, Debra A.G.; Lanier, Charles S.; Lewis, James R.; Tiller, Byron K.; Villafana, William; and Yee, Raymond L. "Personal Communicator Having Improved Zoom and Pan functions for Editing Information on Touch Sensitive Display", United States Patent 5,615,384, March 25, 1997 Pan and Zoom in single gesture (with one finger, not multi-touch) with touch input on handheld device. Used "last contact" (stylus life / finger lift / button up) method to let user move off of wrong button with no effect. Features are of IBM Simon? [AllardDJ98a] Allard, David J; Canova, Francis J. jr.; Johnson, Debra A.G.; Lanier, Charles S.; Lewis, James R.; Tiller, Byron K.; Villafana, William; and Yee, Raymond L., et al "Apparatus and method for marking text on a display screen in a personal communications device", United States Patent 5,815,142, September 28, 1998 IBM Simon: select text (e.g. phone number) by press-hold gesture at start of text: enters selection mode, drag (stroke) to end of selection. Audible beep haptic/acoustic feedback when enters selection mode, and when lift. On lift, pop-up menu for possible actions, such as calling a phone number that has been selected, or searching for text in an index. Compare with marking menus, press-and-hold in PenPoint and Windows for Pen Computing for selection. Press-and-hold gesture. [AltmanD97a] Altman, Dan et al "System and Methods for Spacing, Storing, and Recognizing Electronic



Representations of Handwriting, Printing and Drawing", United States Patent 5,613,019, March 18, 1997 Editing of electronic ink with automatic insertion of spaces: references van Raamsdonk [AmantRS98a] Amant, Robert St. and Dulberg, Martin S. "An Experiment with Navigation and Intelligent Assistance", Proc. IUI 98, San Francisco CA, pp. 171-178 Experiment in showing suggestions from an intelligent assistant in a tree-view / heirarchy. [AndoH97] Ando, Haru and Kitahara, Yoshinori "Display system Capable of Accepting User Commands by use of Voice and Gesture Inputs", United States Patent 5,600,765, Feb. 4, 1997 GUI using touch input with both gesture and direct manipulation: cites to Put-That-There [AnsticeJ96a] Anstice, Jamie; Bell, Tim; Cockburn, Andy; and Setchell, Martin "The Design of a Pen-Based Musical Input System", Sixth Australian Conference on Computer-Human Interaction, 24-27 November, 1996, pp 260267. IEEE Press Presto: Handwriting input for musical scores, including editing gestures for pop-up menus, add tails, add dots, etc. based on context of information. Scribble/scrub mark for deleting. Shows writing styles and stroke order for musical notes and stems. Notes that this is easier than normal musical writing, and faster than OCR of music using 1995 technology. [Apple96b] Apple Computer Inc. "Newton 2.0 Programmer's Guide", Addison Wesley, 1996, ISBN 0-201-48838-8 Newton gestures and handwriting recognition. Intelligent Assistant component may be used in applications (must be programmed in explicitly?): automates scheduling to calendar, reminding to to-do list, mailing, faxing, printing, calling: user can write "call" "phone" "ring" or "dial" to make a phone call, extensible to new tasks (by user?). Passes current text selection or most recent text entry to assistant. Assistant attempts to associate input string templates and dictionaries that classify the words as actions, targets, or unknown: matching done by lexical dictionaries: classification includes classifying as phone number. If Assistant cannot determine intended action, Please Picker pops up offering more options. Slot "protoOrientation" allows screen/display orientation to be changed between landscape and portrait. [Apple96c] Apple Computer Inc. "Newton MessagePad 120", Newton, Apple Computer Inc, 1996 (physical device) Apple Newton (working unit), copyright and default text in unit indicates 1996. Photo file shows screen rotation, with rebuilding for landscape vs. portrait orientation. [Apple96d] Apple Computer Inc. "Newton Programmer's Reference 2.0", Apple Computer Inc, 1996 Programmer's reference manual for Apple Newton software development. ReorientToScreen message sent to applications when orientation changes, to re-layout their views when display has been rotated horizontal/vertical landscape/portrait. [Apple96e] Apple Computer Inc. "The NewtonScript Programming Language", Apple Computer Inc, 1996 Software programming language for Newton: see other references to NewtonScript [Apple96f] Apple Computer Inc. "Newton Solution's Guide V1: Software, Peripherals and Accessories for Newton PDAs", Apple Computer Inc, 1996 Third-party software and accessories for Apple Newton PDA: NovaModem 288 PCMCIA Modem for digital cellular phone service, wireless communications, Equate spreadsheet application with screen rotation landscape/portrait, Newton (physical) keyboard [Apple96g] Apple Computer Inc. "Newton Solution's Guide V2: Software, Peripherals and Accessories for Newton PDAs", Apple Computer Inc, 1996 Third-party software and accessories for Apple Newton PDA: FreeStyle (Paragraph) handwriting recognition, AppGen toolkit application generator, [Apple96h] Apple Computer Inc. "AppleScript Language Guide: English Dialect", Apple Computer Inc, 1996 Scripting language for Apple MacIntosh, scripting of other application objects, scripting within application (e.g. word processing/text edtiro). Contemporaneous to NewtonScript. Compare object inheritance with JavaScript. Syntax similar to "human" languages (hence "English dialect"). Compare with HyperTalk scripting for HyperCard. [Apple97a] Apple Computer Inc. "Newton Programmer's Guide: 2.1 OS Addenum", Apple Computer Inc, 1997 Additional Newton APIs: SetScreenOrientation for landscape/portrait orientation [Apple97b] Apple Computer Inc. "Newton 2.1 OS: Miscellaneous features", Apple Computer Inc, 1997 Has section with screen orientations. [Apple98a] Apple Computer Inc. "AppleWorks 5 for Windows 95, Windows NT, Mac OS", Apple Computer Inc, 1997 (hardcopy book)(Data CD) AppleWorks productivity suite (Word and Excel spreadsheet equivalents) [Apple98b] Apple Computer Inc. "Hypercard Installation and New Features", Apple Computer Inc, 1998 Hypercard 1998 URL link to internet (browser), Quicktime video and virtual realility: search of text tracks / subitles in scripting, synthesized text-to-speech voice output. [AppleNewton96a] Apple "Newton: Apple MessagePad Handbook, 1996", Apple computer, Inc. 1996 Apple Newton: describes various gestures and UI features, including "heavy mark" selection gesture: press-and-hold in text? Describes rotate button to change screen orientation



[AppleNewton96b] Apple Computer Inc. "Apple Newton Press User's Guide for Windows: Reference for Newton 2.0", Apple Computer Inc, 1996 API/programming documentation and devlopement system for Apple Newton. NTK Newton Tool Kit inspector/browser window. protoOrientation class for UI changes for horizontal/vertical landscape/portrait orientation views. Standard button in user interface to change orientation: may be multiple orientations. Transparent (invisible) windows. Automatic API support for virtual on-screen keyboards: auto re-draw on shift and control keys. Automatic keyboard positioning not to obscure location of key-receiving cursor. Scrub/rubout, highlight, tap double=-tap, caret, and line gestures. [AppleNewton96d] Apple Computer Inc. "Newton Press User's Guide for Windows", Apple Computer Inc, 1996 Apple Newton eBook application for windows: mentions gestures and UIs of Apple Newton, including "heavy mark" (press-and-hold gesture?) [AppleNewton96e] Apple Computer Inc. "Newton 2.0 User Interface Guidelines", Apple Computer Inc, Addison Wesley, 1996, ISBN 0-201-48838-8 (hardcopy book) Newton user interface (touchscreen computer). Views, list pickers, date and time pickers. Line gesture to insert new (open) line in text, scrub/erase/rubout gesture, tap-and-a-half (tap, then hold two strokes) to drag/copy instead of drag/move, backwards-L to insert newline, V gesture to join, lasso selected, various select gestures, caret insert gestures, extended caret to insert multiple blank lines. Pop-up dictionary and word list picker menu. Transparent views and windows. Newton gestures and handwriting recognition pp 6-13 ff. Printed (block), cursive, mixed handwriting. Also shapes and sketches. Scrubbing (delete) gesture may be made in any orientation: delete shows animation (puff of smoke) for delete status. Gesture caret, caret plus long line. Rotate button on message pad to change screen orientation. [AppleNewton96f] Apple Computer Inc. "Newton OS 2.1 Engineering Documents", Apple Computer Inc., October 6, 1996 Newton touchscreen computer: Collection of various API definitions, graphical tools (Arrow, Text, etc.), Text and Graphical views, utilities, etc.. eMate 300 multi-user -- actually multiple alternative logons, not simultaneous multiple users. [AppleNewton97a] Apple Computer Inc. "AppGen application generator for Apple Newton", www.archive.org, Tucows Inc., April 28, 1997 Application generator for data collection applications for Apple Newton -- see also Apple file. [AppleNewton97c] Apple Computer Inc. "Newton Toolkit User's Guide for Windows", Apple Computer Inc, 1997 User's manual for Apple Newton. Mentions drag to select via "heavy mark" gesture (press-hand-hold gesture) [AppleNewton97d] Apple Computer Inc. "Apple Newton MessagePad 2000", Apple Computer Inc, 1997 Functional Apple Newton unit [AppleNewton97e] Apple Computer Inc. "Apple Newton MessagePad 2000 User's Manual", Apple Computer Inc, 1997 Apple Newton MessagePad 2000: deferred translation (converting ink to text). Rotating the screen landscape/portrait mode -- applications that cannot be run in landscape mode do not rotate, message that display will remain in portrait orientation. Assist applet for automatically sending faxes, email, dialing when user types/writes request word or synonym (simplified natural language user interface). Automatic dialing via telephone modem or touch-tone tones from speaker. Scrubbing/scratch-out gesture. Double-tap gesture on a word displays alternative spellings/recognition. Select text and also select drawing graphics by highlight/mark gesture. Caret gestures (vertical and horizontal) to insert text or blank lines. Upstroke/downstroke gestures for capitalization. User-definable abbreviations in text automatically expanded. Sketch/shape recognition. Selectable printing/cursive handwriting recognition. User selectable which forms/shapes/writing styles for individual characters, but not trainable. User's manual for Apple Newton. Handwriting recognition, gestures, "Assist" function that reads a word and selects possible actions to be performed based on what the user wrote (or at least, what the Newton *thinks* the user wrote). File contains additional material: Newton Works Word Processor User's Manual, Connection Utilities for Windows and MacIntosh, Internet Enabler. Paper file has additional user documentation on connectivity software, syncing software for Windows and MacIntosh. [AraiT97a] Arai, T. et al. "PaperLink: A Technique for Hyperlinking from Real Paper to Electronic Content", CHI 97 Electronic Publications: Papers, March 22-27, 1997 (13 pages) VideoPen: rough laboratory prototype of small video camera mounted on a pen to capture documents as they are written, or as a hand-held OCR/image capture as documents are marked up: paper states that prototype not developed enough for realistic user studies. Cited in Jared01 re Anoto and similar technologies for optical tablet digitizer [ArefWG96a] Aref, Walid G.; Barbara, Daniel; and Lopresti, Daniel P. "Ink as a First-Class Datatype in Multimedia Databases", In: Database Systems: Issues and Research Direction, Springer Verlag, 1996 pp. 113-163 Electronic ink annotations as data-type: compare with Slate DayTimer pen scheduler. Indexing of ink blobs



as names/labels/pictographic names in file system without handwriting recognition, using similarity comparison / approximate ink matching, matches ink input chunk by chunk in for heirarchical search. Notes that ink annotation more flexible that ASCII handwriting recognition: special characters, graphical symbols, sketches, circuit diagram sketches, etc. Window-matching using simplified form of elastic matching (therefore dependent on visual position, rather than writing/drawing sequence?) [ArnottJ96a] Arnott, John "Rear Projection Display Apparatus", United States Patent 5,521,659, May 28, 1996 Rear-projection tabletop display with minimal parallax error because translucent projection surface layer is very thin. May be used with light pen or pressure-sensitive pen (stylus) digitizer. VisionMaker spin-off from Bill Buxton? [AscensionTechnology96a] Ascension Technology Corporation "Ascension Technology Corporation SpacePad Position and Orientation Measurement System Installation and Operation Guide, Rev. A", Ascension Technology Corporation, June 28, 1996 3D/6DOF DC magnetic position sensor, compare with Polhemus. Senses up to 144 inches from transmitter sensor for high accuracy, example of use is stand-up game pod [AtkinsCA97a] Atkins, Charles Agee "System for the operation and management of one or more financial accounts through the use of a digital communication and computation system for exchange, investment and borrowing", United States Patent 5,644,727, July 1, 1997 Financial transactions over wireless PDA/Smartphone with video camera and touchscreen [AtkinsonR96a] Atkinson, Randall "Intermediate Network Authentication", United States Patent 5,511,122, April 23, 1996 Two-way digital signature exchange to authenticate both ends of an internet connection (packets): compare with SSL 1.0? File contains additional materials, contemporaneous comments by inventor regarding internet standards. See also folder for Network Signatures. [AustD96a] Aust, Dietmar "Augmenting Paper Documents with Digital Information in a Mobile Environment", Diplomarbeit, Sep. 3, 1996, Universität; Dortmund, Germany "Smart Paper"-type project, optical video camera on colored highlighter. User can mark with highlighter on (paper) text, system recognizeds pattern of mark over text. Electronic bookmarks can link to internet resources (URLs). Cites to Digital Desk. Not pattern-based like Anoto. [Bagley96] Bagley, Steven C;, Kopec, Gary E. "Editing text in an image", United States Patent 5,548,700, August 20, 1996 Editing of (static) electronic ink without character recognition, by recognizing bounding boxes of individual characters. Did not cite van Raamsdonk, did cite Facsimile editing by Suenega [Bagley98] Bagley, Steven C. "Editing Scanned Document Images using Simple Interpretations", United States Patent 5,734,761, March 31 1998 Editing of (static) electronic ink without character recognition, by recognizing bounding boxes of individual characters. Refers to editing groups of graphical objects by pretending that they are a set of letters/words, but does not used specific character recognition, merely recognition of text-like elements [BagnasES98a] Bagnas, Emmanuel Salud "Darkened Transparent Window Overlapping an Opaque Window", United States Patent 5,805,163, September 8, 1998 Transparent windows in a GUI, so that other elements are still visible: cites to Bier See-through/Clickthrough tools. [BalakrishnanR98a] Balakrishnan, Ravin and Patel, Pranay "The PadMouse: Facilitating Selection and Spatial Positioning for the Non-Dominant Hand", Proc. 1998 ACM Conf. on Human Factors in Computing Systems (CHI'98), pp. 9-16 Marking menus using small touchpad on mouse, so that one hand can perform mouse/pointing function and also menu selection. Alternative is PadMouse with marking keys 10 keys mounted: compare to multibutton puck of Summagraphics and other tablets? Press-and-hold -- has flowchart of timing. [BallerTS96a] Baller, Theunis S.; Makinwa, Kofi A.A.; Polaert, Remy; Hafkamp, Antonius J.J.; Koolen, Gerardus j.K.M.; and Duwaer, Arne L. "Force Activated Touch Screen Measuring Deformation of the Front Panel", United States Patent 5,541,372, July 30, 1996 Touchscreen by mounting strain gauges directly on the corners of front glass of a CRT monitor display and measuring the deflection of the glass, computing single point. Correction for non-linear response under greater force. [BarrettGL98] Barrett, Gary L. and Wolfe, Andrew L. "Curvilinear linearization device for touch system", United States Patent 5,736,688, April 7, 1998 Resistive/conductive ink patterns to correct non-linear distortions of resistive digitizer [BartlettJF98a] Bartlett, Joel F. "The Itsy Pocket Computer", Slide Presentation, Second International Symposium on Wearable Computers, October 19, 1998 See other Itsy references for description. [Baudel97] Baudel, T., Buxton, Fitzmaurice, G., Harrison, B., Kurtenbach, G. and Owen, R. "Clickaround tool-



based graphical interface with two cursors", United States Patent 5,666,499, September 9, 1997 See also http://www.billbuxton.com Two-handed user interface using two input cursors or stylus, one hand positions a menu or tool with various buttons or action elements, the other operates on the tool and the object selected underneath [BauerM98a] Bauer, Martin; Heiber, Timo; Kortuem, Gerd; Segall, Zary "A Collaborative Wearable System with Remote Sensing", Proc. ISWC '98 2nd IEEE Int'l. Symposium on Wearable Computers, Washington DC, p. 10 (abstract only) Cited frequently for wearable computer with video camera and display [BaxterLK96a] Baxter, Larry K. "Capacitive Sensors: Design and Applications", IEEE Press Series on Electronics Technology, Wiley-IEEE Press, ISBN 978-0-7803-5351-0, September 1996 (electronic book) Reference work on capacitive sensors: chapters include Proximity Detector, Graphic input tablet, Camera positioner, Digital level. Measuring capacitance (frequency) vs. measuring impedance gives linearity effects for distance, micromachined accelerometers. Synchronous demodulation in standard components. Comparison with ultrasonic proximity sensors (6.2.4): variation of speed of sound varies with tempuratur, humidity, frequency (!), little effect from pressure. [BedersonB96a] Bederson, Benjamin B.; Hollan, James D., Perlin, Ken; Meyer, Jonathan; Bacon, David; Furnas, George "Pad++: A Zoomable Graphical Sketchpad for Exploring Alternate Interface Physics", Jnl. of Visual Language and Computing, 1996, Vol 7, pp. 3-31 Zoomable user interface, refers to "user-interface physics" to describe semantic zooming (showing more information) as contrasted to larger image (fatter pixels), see also Data Lens. [BedersonB98a] Bederson, Benjamin B. and Meyer, Jon "Implementing a Zooming User Interface: Experience Building Pad++", Software - Practice and Experience, Vol 28 No 10, Aug. 1998, pp. 1101-1135 Zoomable user interface: spatial vs. temporal indexing, user builds up a model in her or his head about the relationships of data. For faster response while zooming or roaming, may show a lower-resolution image, and then perform "refinement" to higher resolution. Similarly, show fake "greekified" text at first, during refinement render actual text. [Beernink97a] Beernink, Ernest H., Aguste, Donna M. and Meier, John R. "Method and Apparatus for Recognizing Handwriting of Different Users of a Pen-Based Computer System", United States Patent 5,666,438, September 9, 1997, assigned to Apple computer Inc., Cupertino, California Pop-up menu/keyboard for punctuation characters and editing commands: insert, delete, etc. Improves recognition non-adaptively, by letting user input setting for spacing of words, cursive/mixed/printed, timeout for end of character, and other preference settings. [BeerninkEH97b] Beernink, Ernest H.; Foster, Gergg S.; and Capps, Stephen P. "Gesture sensitive buttons for graphical user interfaces", United States Patent 5,612,719, March 18, 1997 Gesture on a GUI button: examples are tap, check-mark, X. [BelfioreJD97a] Belfiore, Josph D. et al "Auto-Scrolling during a Drag and Drop Operation", United States Patent 5,611,060 Auto-scrolling based on speed of mouse as well as limit of scrollable distance [BennettPW98a] Bennett, Paul William "Method of and system for updating dynamic translucent windows with buffers", United States Patent 5,764,229, June 9, 1998 Translucent/transparent windows, with management of multiple windows based on Z-order and combining translucency. [BermanAH96a] Berman, Andrea H. and Whitmore, Mihriban "Initial Usability Testing of a Hand-held Electronic Logbook Prototype for the Human Research Facility", NASA Contractor Report NAS9-18800, April 1996, LM31992 Apple Newton hand-held pen computer very awkward to use in microgravity environment, because it was hard to write and hold either the pen computer and/or the user still at the same time. Handwriting input rated "marginal". [BermanER98a] Berman, Eric Robert et al "Methods and apparatus for interacting with data objects using action handles", United States Patent 5,760,773, June 2, 1998 Context menu for an object by clicking/tapping on a handle on the object. Tap on handle brings up context menu and not press-and-hold. [BerquistDT97a] Berquist, Davit T.; Eisenberg, Peter M.; Grunes, Mitchell B.; Kenner, Martin A.; Kozak, Janelle; Kruse, John M.; Munson, Cindy L.; and Robins, Althea M. "Method for handling obscured items on computer displays", United States Patent 5,689,666, November 18, 1997 See-through windows. Combines images by "cutting a hole" (transparent) in the foreground window so that you can see what is in back. [BierEA96a] Bier, Eric A. and Buxton, William A. S. "User interface having movable sheet with click-through tools", United States Patent 5,581,670, December 3, 1996 See also http://www.billbuxton.com Simulated keyboard user-interface, other devices, positions on a movable menu of user-interface components (simulated selector, rotator, etc.) on a virtual sheet or grid



which can be positioned over the object for the action to be applied to. [BierEA97a] Bier, Eric A.; Buxton, William A. S. and Stone, Maureen C "User interface having click-through tools that can be composed with other tools", United States Patent 5,617,114, April 1, 1997 Variety of simulated devices on a digitizer/display: examples include simulated mouse, simulated keypad, simulated French curves or stencils (movable menu?), movable pie menus. Transparent windows? Notes that tool overlays can also be opaque and behind partially-transparent application image. See also http://www.billbuxton.com [BigelowSJ98a] Bigelow, Stephone J. "Understanding Pen Systems and Touchpads", Chapter 34, in "Troubleshooting, Maintaining, and Repairing PCs -- a Technician's Guide", McGraw-Hill, 1998 Simple overview of resistive digitizers, capacitive pen digitizers, capacitive touchpad digitizers, electromagnetic digitizers Has listing of Gestures for Pen Windows, and for PenPoint, including pigtail gestures and other gestures not used in later Tablet PC from Microsoft [BillinghurstM98a] Billinghurst, Mark "PutThatWhere? Voice and Gesture at the Graphics Interface", Computer Graphics, Nov 1998, pp. 60-63 Cites to multi-modal voice and hand-gesture input in 1992, multi-model allows user to perform multiple tasks. Title refers to "Put That There" multimodal interface with hand and hand-pointing gestures at M.I.T. Refers to context: microtemporal, macrotemporal, and context. [BinsteadRP98a] Binstead, Ronald Peter "Multiple Input Proximity Detector and Touchpad System", United States Patent 5,844,506, December 1, 1998 Capacitive touch-pad using array of sensors (keypad, or x-y coordinates) formed by intersecting overlapping grid, including discrete proximity sensing keys (multi-touch). One application is detection at multiple (adjacent) sensors to make more reliable detection of a touch. [BirkH97a] Birk, Henrik; Moeslund, Thomas B.; and Madsen, Claus B. "Real-Time Recognition of Hand Alphabet Gestures Using Principal Component Analysis", Proc. SCIA '97, 10th Scandinavian Conf. on Image Analysis pp. 261-268, 2007 Hand-gesture recognition for 25 (of 26?) international hand alphabet / ASL American Sign Language: Trained system, 1000 samples, 1500 test images. [BissetSJ96] Bisset, Stephen; Miller, Robert J.; Allen, Timothy P., and Steinbach, Guenter "Touch Pad Driven Handheld Computing Device", United States Patent 5,543,588, August 6, 1996 Capacitive touch-screen and display. Touch screen behind/rear display, multiple-finger gestures on touch screen (multi-touch). Cites Rympalski as working by altering transcapacitance coupling. [BissetSJ98a] Bisset, Stephen and Kasser, Bernard "Multiple Fingers Contact Sensing Method for Emulating Mouse Buttons and Mouse Operations on a Touch-Sensor Pad", United States Patent 5,825,352, October 20, 1998 Capacitive touch-screen and display, multi-touch input with interpolation, limited use for emulating mouse buttons. Could be interpreted as applying to gestures? Mentions human factors problems of emulating mouse clicks with single-touch digitizer without proximity sensing. [BlickenstorferCN97a] Blickenstorfer, Conrad H. "Windows CE 2.0: Where does it stand after Year 1? Slow start, revving up now", Rugged PC Review, November 9, 1997 Review of Windows CE 2.0. 4MB memory marginal, 6MB better. Backlight needed. Velo 1 (HP) using software modem, internet web browsing for several hours. Requires 8..12 MByte for practical use. Notes portability to non-Intel (e.g. ARM) processors. Voice recording (phone?). Complaints about adapted Desktop GUI on small (480x640) displays. Suggests handwriting recognition with stylus over small keyboard. (File contains additional material on Velo 1 Windows CE memory upgrade to 2.0, etc.) [BlonderGE98a] Blonder, Greg E.; Boie, Robert Albert; Jin, Sungho; and McCormack, Mark Thomas "Position Sensing Systems including Magnetostrictive Elements", United States Patent 5,834,709, November 10, 1998 Magnetostrictive tablet with grid of magnetostrictive wire conductors in X columns and Y rows. [BlonderGE98b] Blonder, Greg E. "Graphical Password", United States Patent 5,559,961, September 24, 1996 Graphical password, user select things to tap on in an image. Slide-to-unlock? [BoieRA98a] Boie, Robert Albert; Gottscho, Richard Alan; Kmetz, Allan Robert; Krukar, Richard H.; Lu, Po-Yen, and Morris, John Robert Jr. "Integrated Liquid Crystal Display and Digitizer having a Black Matrix Layer Adapted for Sensing Screen Touch Location", United States Patent 5,847,690, 1998 Integrate digitizer/capacitive touch screen with LCD display. Original touch tablet uses horizontal electrodes that are wedged/tapered shaped in opposite directions on alternation rows. Y ordinate determined from which row is affect, X ordinate from strength of signal (width of electrode). In integrating with LCD, the electrodes are transparent, wedges are step-wise matched so that the taper is stepped so edged run between rows of LCD pixels. A black matrix with holes for every LCD pixel is put over the electrodes, so that the electrodes and stepped gaps cannot be seen. Also mentions using same electrodes together as the conductive plan for the LCD pixel operation. [BolingD98a] Boling, Douglas "Programming Microsoft Windows CE: The definitive guide to programming the Windows CE API", Microsoft Press, 1998 (hardcopy book)



Windows CE: Electronic file excerpt describes SIP Supplementary Input Panel (not software input panel) for touch/handwriting (?) input: IsSIPInputMethod call [BorensteinNS96a] Borenstein, Nathaniel S. "CMU's Andrew Project: A Retrospective", CACM Vol 39 No 12, Dec 1996, pp. 298-311 Review of Andrew system UIMS/GUI, ATK Andrew Tool Kit. Notes that implementing databases as large single files was problematic because of file transfering in bulk, used multiple files instead. Componentware compare with Java and ActiveX, direct data exchange DDE. [BoritzJ98a] Boritz, James "The Effectiveness of Three Dimensional Interaction", Graduate Thesis, University of British Columbia Cites R.A. Bolt "Put that there" Gesture system from 1980 Primarily deals with visual feedback, human factors in three-dimensional pointing methods [BowyerK97a] Bowyer, Kevin "Case Study Resources for an Ethics and Computing Course", 1997 ASEE/IEEE Frontiers in Education Conference Otto Berkes, trade-secret ethics example, lawsuit between VMI and Autodesk concerning tablet computing. [BraveS97a] Brave, Scott and Dahley, Andrew "inTouch: A Medium for Haptic Interpersonal Communication", Proc. CHI '97, April 1997 Remote haptic communication (telepresence) between two users, using force-feedback haptics. Prototype is three rollers with position sensing: objects agree on common state (to overcome time-lags in communication, etc.) and actuators drive rollers to same state. Cites to teleoperated systems. [BricklinD96a] Bricklin, Daniel; Johnson, Eirc E.; Friend, John; Kelley, Winslow B; and Levin, Peter H. "Graphic Indexing System", United States Patent 5,539,427, July 23, 1996 Slate pen-computing DayTimer-like calendaring and notetaking application: data shown as electronic ink, lasso gesture to select for deferred/delayed recognition for indexing. Electronic ink may be indexed and shown in multiple views/pages: Month/Day/Week/other calendar pages. Press-and-hold: hold time out / user maintain a constant pen position for a certain amount of time. Algorithm for lasso gesture. [BricklinD98a] Bricklin, Daniel; Lynch, William T; and Friend, John "Method and Apparatus for Entering and Manipulating Spreadsheet Cell Data", United States Patent 5,848,187, December 8, 1998 Slate @Hand / At-Hand patent, pen-computing spreadsheet product. Targeting of ink to a cell/button that is smaller than the electronic ink gesture. Both electronic ink and recognition result are displayed and can be preserved. Refers to multi-tiered handwriting/gesture recognition. [BricklinD98b] Bricklin, Daniel; Lynch, William T; and Friend, John "Method and Apparatus for Entering and Manipulating Spreadsheet Cell Data", United States Patent 5,717,939, February 10, 1998 Slate @Hand / At-Hand spreadsheet patent. Targeting of ink to a cell/button that is smaller than the electronic ink gesture. Both electronic ink and recognition result are displayed and can be preserved. Refers to multi-tiered handwriting/gesture recognition. Original implementation on GO/PenPoint, ported to Pen Windows. [BrownA98a] Brown, Andrew R. "Composing by Pen: Exploring the effectiveness of a system for writing music notation on computers via pen input", Manuscript published on-line at at http://www.academo.qut.edu.au/music/browna/NoteWriter.htm Tested usability of NoteWriter software by Hamel, 1988 on undergraduate music students: main objection was poor recognition performance: subjects/users did not have much time to get accustomed to the system: described Hamel specifically as a unistroke input system unistroke handwriting input for music notation [BrownH98b] Brown, Heather; Harding, Robert; Lay, Steven; Robinson, Peter; Sheppard, Dan; and Watts, Richard "Active Alice - using real paper to interact with electronic text", Proceedings of 7th Internal Conference on Electronic Paper, Saint Malo, April 1998. Reprint available at www.globis.ethz.ch (abstract only) Digital Desk: optical video camera determines position of writing on a desktop instead of digitizer. Compare to Richard Oed 1980's. Bibliography of other papers by Robinson in paper file. [BrownMH96a] Brown, Marc H. and Shillner, Robert A. "The DeckScape Web Browser", Video presentation, Proc. CHI '96, April 13-18, 1996, pp. 418-419 Web browser, pages can be stacked in multiple decks on screen: no 3-D effect on desks -- compare with Freestyle documents. All pages in history save, not just single branch when user backs up. [BurgessD98a] Burgess, David; Felsenstein, Lee; Saunders, Steven E. "Personal Object Detector", Personal Object Detector Object detector / navigation aid for the visually impaired. Handheld (flashlight-like) sonar device, uses continuous swept-frequency, measures frequency difference (caused by phase shift from time delay) between reflected signal and internal signal. Two can be used for stereo signal, and left-right echolocation. Compare with Jeremy Ozer? [BusinessWeek98a] Business Week "Cell Phone meets Laptop: Design Exploration: Gold MoCCA Designers", Business Week, May 24, 1998



Mobile Computing and Communications Appliance MoCCA: Design aware for fusion of laptop computer, cellular phone, and handheld personal organizer (PDA), wireless videoconferencing. Dual side-by-side screens. [ButcherL96a] Butcher, Lawrence "Method of use of multiple input styli in a system of multiple computers", United States Patent 5,581,269, December 3, 1996 Multiple styli/pens on multiple tablets, each tablet sends an identifying signal so that the stylus knows which computer it is on: multi-touch. Signal reported by stylus. [Buxton98a] Buxton, W. and Bier, Eric A. "User interface having simultaneously movable tools and cursor", United States Patent 5,798,752, Aug. 25, 1998 See also http://www.billbuxton.com User interface using a digitizer tablet with a puck, the puck has user input buttons on it (like a traditional digitizer puck), but also reports the rotational orientation of the puck. Puck also contains a trackball. Shows tool being rotated to a desired angle. [Buxton98b] Buxton, W. and Fitzmaurice, G.W "System for editing time-based temporal digital media including a pointing device toggling between temporal and translation-rotation modes", United States Patent 5,790,769, Aug. 4, 1998 See also http://www.billbuxton.com (Six dimensions) User interface using a digitizer tablet with a puck, the puck has user input buttons on it (like a traditional digitizer puck), but also reports the rotational orientation of the puck. Mentions toggling (via button?) between temporal and positional input modes, [ByongK97] Byong, K. Ko and Yang, Hyun S. "Finger mouse and gesture recognition system as a new human computer interface", Science Direct / Elsevier, pp. 551-561, September-October 1997 Finger-mouse gestures in contrast to Stylus and Mouse gesture input devices. Data capture by video image capture, not by Data Globe etc. [CIC97a] Communications Intelligence Corporation "Handwriter MX FX Version 1.0 Users Guide", CIC Communications Intelligence Corporation, 1997 Half-sized tablet with handwriting recognition for use with a PC, similar to Pencept PenPad or CIC Handwriter. "What you write - Turns to type." Includes YPad note-taking app (text plus sketches plus electronic ink) and Personal Ink for annotating documents on Windows using OLE embedding. [CIC97b] Communications Intelligence Corporation "Handwriter MX handwriting recognition tablet", Handwriter, CIC Communications Intelligence Corporation, 1997 (physical device) (Original packaging, functional)Half-sized tablet with handwriting recognition for use with a PC, similar to Pencept PenPad or CIC Handwriter. [CNET98a] Kawamoto, Dawn "Microsoft pilloried in tell-all: Book to dish Microsoft dirt", CNET News, April 23, 1998 Review of "Barbarians Led by Bill Gates" by Marlin Eller: describes Microsoft as a matter of dumb luck, points to PenPoint/PenWindows history. [Camaisa98] Camaisa, Allan J.; Wadkins, E. Tracy; and Gayda, Karen M. "Interactive Visual Ordering System", United States Patent 5,845,263, December 1, 1998 Touchscreen sales terminal with menus (as in food) for restaurants [CampbellCE96a] Campbell, Charles E. "Frequency Scanning Capaciflector for Capacitively Determining the Material Properties", United States Patent 5,521,515, May 28, 1996 Capacitive sensor using swept frequency, determines permissivity profile/patter of material, identifies material by comparing to reference patterns for materials. From then-known permissivity, determine approximate distance by capacitance change. [CappsSP96a] Capps, Stephen P. "Method for selecting objects on a computer display", United States Patent 5,523,775, assigned to Apple Computer, Inc., June 4, 1996 Recognition of a selection gesture over objects on a display: makes use of overlap, displays pen path on display as electronic ink; selection can be shown before the gesture is complete and the pen stylus is lifted. Compare with lasso selection in PenPoint and Buxton? [CappsSP96b] Capps, Stephen et al "Method and apparatus for indicating a change in status of an object and its disposition using animation", United States Patent 5,544,295, August 6, 1996 Animating an icon or other object to show its change in status: e.g. crumping an object to show that is status is now as "deleted" [CappsSP96xa8] Capps, Stephen et al "Interface for a computerized database having card and list views", United States Patent 5,446,882 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xa9] Capps, Stephen et al "Method and apparatus for manipulating inked objects", United States Patent 5,465,325 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the



Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb0] Capps, Stephen et al "Method and apparatus for providing computer-implemented assistance", United States Patent 5,477,447 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb2] Capps, Stephen et al "Method and apparatus for setting a clock in a computer system", United States Patent 5,487,054 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb3] Capps, Stephen et al "Graphic editor user interface for a pointer-based computer system", United States Patent 5,513,309 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb7] Capps, Stephen et al "Interface for a computerized database having card and list views", United States Patent 5,544,358 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb8] Capps, Stephen et al "Resetting the case of text on a computer display", United States Patent 5,555,363 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xb9] Capps, Stephen et al "Method and apparatus for formatting a communication", United States Patent 5,579,467 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc0] Capps, Stephen et al "Pointing gesture based computer note pad paging and scrolling interface", United States Patent 5,581,681 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc1] Capps, Stephen et al "Method for deleting objects on a computer display", United States Patent 5,583,542 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc2] Capps, Stephen et al "Method and apparatus for setting a clock in a computer system", United States Patent 5,583,833 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc3] Capps, Stephen et al "Status bar for application windows", United States Patent 5,588,105 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc4] Capps, Stephen et al "System for executing different functions associated with different contexts corresponding to different screen events based upon information stored in unified data structure", United States Patent 5,588,141 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP96xc5] Capps, Stephen et al "Method for manipulating notes on a computer display", United States Patent 5,590,256 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the



Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97a] Capps, Stephen P. "Method and apparatus for indicating a change in status of an object and its disposition using animation", United States Patent 5,596,694 Animating an icon or other object to show its change in status: e.g. crumping an object to show that is status is now as "deleted" [CappsSP97b] Capps, Stephen P. "Graphical User Interface using Historical Lists with Field Classes", United States Patent 5,666,502 Tap on field with stylus, get recently-entered history values for that type of field to select from. see also PenApps history. [CappsSP97xc6] Capps, Stephen et al "Method and apparatus for computerized recognition", United States Patent 5,592,566 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xc7] Capps, Stephen et al "Method and apparatus for correcting words", United States Patent 5,594,640 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xc8] Capps, Stephen et al "System and method of reflowing ink objects", United States Patent 5,596,350 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd0] Capps, Stephen et al "Method for routing items within a computer system", United States Patent 5,596,697 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd1] Capps, Stephen et al "Method for deleting objects on a computer display", United States Patent 5,602,570 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd3] Capps, Stephen et al "Method and apparatus for deducing user intent and providing computer implemented services", United States Patent 5,621,903 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd4] Capps, Stephen et al "System and method for event parameter interdependence and adjustment with pen input", United States Patent 5,634,100 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd5] Capps, Stephen et al "Methods and apparatus for a selectable backdrop", United States Patent 5,634,102 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd6] Capps, Stephen et al "Method for locating and displaying information in a pointer-based computer system", United States Patent 5,644,657 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xd7] Capps, Stephen et al "Method and apparatus for providing implicit computer-implemented assistance", United States Patent 5,644,735 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC?



[CappsSP97xd9] Capps, Stephen et al "Method and apparatus for the manipulation of text on a computer display screen", United States Patent 5,666,552 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xe0] Capps, Stephen et al "Method and apparatus for formatting paragraphs", United States Patent 5,671,438 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP97xe1] Capps, Stephen et al "Boxed input correction system and method for pen based computer systems", United States Patent 5,682,439 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98a] Capps, Stephen; Beernink, Ernest H.; and Temkin, David T. "String inserter for pen-based computer systems and method for providing same", United States Patent 5,778,404, July 7, 1998, assigned to Apple computer Inc., Cupertino, California Pop-up menu/keyboard for punctuation characters and editing commands: insert, delete, etc. Uses "pseudo keyboard" for virtual/on-screen keyboard. Compare with Wang "Guide" patent on keyboard simulation? [CappsSP98xe2] Capps, Stephen et al "Method for correcting handwriting on a pen-based computer", United States Patent 5,710,831 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98xe3] Capps, Stephen et al "Method for searching and displaying results in a pen-based computer system", United States Patent 5,710,844 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98xe4] Capps, Stephen et al "Method and apparatus for tab access and tab cycling in a pen-based computer system", United States Patent 5,745,716 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98xe5] Capps, Stephen et al "Method for locating and displaying information in a pointer-based computer system", United States Patent 5,764,818 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98xe7] Capps, Stephen et al "Rotating toy with electronic display", United States Patent 5,791,966 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CappsSP98xe8] Capps, Stephen et al "System for computer with interface and scripting systems cooperating in interrelated fashion by sharing frame objects of common unified data structure stored in object system", United States Patent 5,805,869 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CardS97a] Card, Stuart "Information Visualization and Information Foraging", Cseriac Gateway, Vol VIII No 1, 1997, pp 9..10 Summary of talk on information visualization by Stuart Card at Xerox PARC: mentions Memex, mostly talks about zooming (including semantic/non-graphical) and linking (via Hypertext) [CardSK98a] Card, Stuart K.; York, William M.; and Robertson, George G. "System for moving document objects in a 3-D workspace", United States Patent 5,838,326, November 17, 1998 Flick and pointing gestures to move objects/pages in a user interface (including web browser). Objects shown in simulated 3-D three dimensional perspective in virtual bookshelves. Compare with PenPoint, "Bob", PerspectiveWall? [CardSK98b] Card, Stuart K.; York, William M.; and Robertson, George G. "3-D document workspace with focus,



immediate and tertiary spaces", United States Patent 5,847,709, December 8, 1998 UI showing objects/pages in a simulated 3-D three dimensional perspective view: book cases (at an angle) on the side, other objects in background/distance. Perspective lines shown for 3D reference. Compare with "Bob", PerspectiveWall, General Magic? [CardelliL97a] Cardelli, Luca "Everything is an Object", Digital Equipment Corporation Systems Research Center, May 30, 1997 Presentation on prototype-based languages. Cites to NewtonScript, and to Unger on Self, which was prototype to NewtonScript. Has references to other Self manuals. No SDKs listed. [CarrollDW96a] Carrol, David W. "Wearable Personal Computer System", United States Patent 5,555,490, September 10, 1996 Wearable computer, a PC (with standard PCMCIA cards, etc.) with components distributed in pockets on a vest, cables/connectors for system bus etc. running through vest. Refers to keyboard and touchscreen, figures do not show how they would be included. See also Carroll '960 2000 [CarrollTouch96a] Carrol Touch Products "Touch System Programmer's Guide", Carroll Touch, August 1996 Infrared (optical) touch systems, guided wave (acoustic SAW) touch systems. Refers to "beam averaging" (interpolation?) as distinct from non-contiguos (unintended second) touches, and "failed beam" on infrared touch tablet. [Casio96a] Casio Inc. "Casio NX-6000 Planeo", Casio Inc. Product information on NX-600 touchscreen PDA / digital diary. Includes sketch input, handwriting recognition. [Casio98a] Casio Inc. "Casio PV-200 Pocket Viewer", Casio Inc. Product information on PV-200 Pocket Viewer: touchscreen PDA. File contains additional information, including product manuals. [ChaikenD98a] Chaiken, David; Mayter, Mark; Kistler, Jay; and Redell, Dave "The Virtual Book", Digital Systems Research Center, SRC Research Report 157, November 11, 1998 Lectrice: virtual book tablet-sized pen computer optimized for book reading: platform for Lecturn Virtual Paper viewer software for PDF and HTML documents. Implemented as keyboardless X-Windows terminal. Wacom pen input. "Reminder" labels (of gesture alternative) in graphical user interface displayed next to physical buttons at edges of display. Gestures (using Graffiti or Rubine recognizer) general key-input sequences. Refers to electronic ink annotations as write-on-window mode. Cites to PenPoint, Dynabook, and Stanley Kubrick. Rotation/display orientation in landscape and portrait mode, requires physical buttons to be rotationally symmetric at all four sides. Documents may include audio and video. [ChanA96a] Chan, Albert "Hypermenu: Integrating Function Access and Work Object Representation in Graphics Applications", Masters's Thesis, Univ. of Texas School of Engineering Science, 1996 Show active elements in context (in a document or graphical object) that have a pop-up menu. Has review of pull-down/linear and circular menus. [ChattyS96a] Chatty, Stephane and Lecoanet, Patrick "Pen computing for air traffic control", Proc. of SIGCHI Conference on Human factors in computing systems, Vancouver 1996, pp. 87-94 Compare with Orr68, which was not cited Cites previous work on GRIGRI by Bothorel, Baudel, Dubocq and Lepied Study of a direct-manipulation input system for air traffic control, versus previous lightpen/keyboard systems. Mentions gesture recognition error rate, suggestion for use of application object context and other factors to improve recognition and input accuracy. [Chefalas96] Chefals, Thomas E. and Tappert, Charles C. "Silent training by error correction for on-line handwriting recognition systems", United States Patent 5,544,260, August 6, 1996 "Silent" training of any trainable handwriting character recognition system by displaying possibilities to user to make corrections to the training set. User writes directly over a character shown on the display. [ChiuP98a] Chiu, Patrick and Wilcox, Lynn "A Dynamic Grouping Technique for Ink and Audio Notes", Proc. UIST '98, 8/6/98 Dynomite prototype system: Modeless note-taking interaction for electronic ink and audio notes: mentions Apple Newton, Aha! InkWriter (Pen Windows) etc. but not Wang Freestyle. Audio delimited by periods of silence (200 msecs), electronic ink by minimum distance between strokes. Refers to whiteboard and sketching systems as related topics. [ChokSS98a] Chok, Sitt Sen and Marriot, Kim "Automatic Construction of Intelligent Diagram Editors", Proc. UIST '98, San Francisco CA pp. 185-194 Sketch recognition / sketch editor /Diagram Editor: context sensitivity / syntax in graph-based visual language (mathematical notation) [Cirque97a] Cirque Technology "GlidePoint Technology: How it works", www.cirque.com, 28 April 1997 Capacitive grid of sensor points in touch digitizer, finger near intersection of two electrodes modifies mutual capacitance. Centroid of finger contact area is determined from interpolation of readings of multiple touch points. Multi-touch capability, compare with Rekimoto.



[CitrinW96a] Citrin, Wayne and Gross, Mark D. "Distributed Architectures for Pen-Based Input and Diagram Recognition", Research Report, Dept. Of Electrical and Computer Engineering, Univ. of Colorado, Boulder CO USA, http://wallstreet.colorado.edu/projects/dds/Papers/AVI96.html PDA-based diagram/sketch recognition editor with single-stroke / unistroke gestures Co-operative drawing surface (whiteboard system using Apple Newtons). Part of SmartPad project: Electronic Cocktail Napkin [ClarkeJ97] Clarke, J. "Shorthand for the PalmPilot?", posting to newsgroup alt.comp.sys.palmtops.pilot, 09/13/1997, available at www.dejanews.com Review of Gregg and Pitman shorthand, and the fact that all experienced practitioners use special single/uni-stroke/unistroke forms for common words, phrases, pages of text Character handwriting recognition for short-hand: refers to single-stroke / unistroke codes,Computer Aided Transcription Mentions commercial companies by name [ClineTL97a] Cline, Troy Lee; Isensee, Scott Harian; Poston, Ricky Lee; and Werner, Jon Harald "Pop Up Scroll Bar", United States Patent 5,655,094, August 5, 1997 On-screen scrolling widget, user can scroll active/selected objects in any direction using widget. Widget looks like crossed scroll bars. [CloughWA97a] Clough, William A.; Ouelette, Daneil; De La Sablonniere, Serge "Portable Computer with Touch Screen and Computing System Employing Same", United States Patent 5,675,362, October 7, 1997 Portable slate pen-computer using touch screen (not proximity digitizer): Microslate, Inc. of Canada. Generic claim appears to be having an automatic application generator. Refers to drop-down pick list as a "library" of answers for touch-screen selection. [CohenPR97a] Cohen, Philp R, et al "QuickSet: Multimodal Interaction for Distributed Applications", Proc. Fifth Int'l Multimedia Conference (Multimedia '97), Seattle, WA, November 1997, pp. 31-40 Multimodal simulation of combat environment: mentions use of pen-gestures, marking for military map symbols: e.g. Routes, areas, mortars, tank platoon. alpha/pigtail delete gesture. Pointing hand-gestures triggered by voice recognition of diectic noun phrases (e.g. "this one"). [Computerwoche98a] Computerwoche "pdQ kommt 1999 - Qualcomm kombiniert PDA und Handy", Computerwoche, 23.09.1998 First smartphone: Qualcomm pdQ combines PDA and cellphone, using Palm OS. Other vendors using Symbian OS, not Windows CE. [ConstanbleF97a] Constanble, Frederick "Xerox sues USR over patents", posting to newsgroup alt.comp.sys.palmtops.pilot, 04/30/1997, available at www.dejanews.com Xerox sues US Robotics / Palm / Graffiti over unistroke character recognition, Goldberg patent [CoxDA98a] Cox, Donald A.; Chugh, Jasdeep S.; Gutwin, Carl; and Greenberg, Saul "The Usability of Transparent Overview Layers", Proc. CHI '98, 18-23 April 1998, pp. 301-302 Brief study of how easily 8 users can work with 2.5D overlaid semi-transparent images (pipelines in two layers, rather than GUI windows) -- works fine, except for some cases where user confused which layer they were working on. [CrossPad97a] A.T. Cross Pen Company "CrossPad", A.T. Cross Pen Company, www.cross.com (fetched 2011) File with product information on CrossPad, stand-alone digitizer clipboard with electronic recording of notes/ note-taking in electronic ink. Appears to be addressing similar applications to Anoto LiveScribe pen. Product introduced 1997, discontinued 2001. Off-line dynamic handwriting recognition applications provided by IBM. Could be used a mouse/digitizer with additional connection to PC. [CrossPad97b] A.T. Cross Pen Company "CrossPad Portable Digital Notepad Orientation Guide", A.T. Cross Pen Company Tablet sized digitizer, battery-powered pen, comes with IBM Ink Manager note-taking application. Compare with Anoto and Freestyle. Notes (including audio?) are stored in clipboard unit, transferred via cable to IBM PC. Paper is pre-formated with Keywords and Bookmark areas, but no special pattern (as in Anoto et al). [CrossPad98a] A.T. Cross Pen Company "CrossPad XP", A.T. Cross Pen Company. Fetched from research.microsoft.com, 2013 Clipboard-sized digitizer (electromagnetic?) with firmware for capturing electron ink, note-taking. Compare with Anoto? [CrossPad98b] A.T. Cross Pen Company "CrossPad Portable Digital Notepad", A.T. Cross Pen Company Sales brochurc: Clipboard-sized digitizer (electromagnetic?) with firmware for capturing electron ink, notetaking. *.INK format as self-executable (?) read-only file. [CullenJF98a] Cullen, J. F.; Penirs, Mark; and Hart, Peter E. "Computer User Interface for Manipulating Image Fragments, Using Drag, Drop and Merge Operations", United States Patent 5,732,230, March 24, 1998 User interface for editing image sections on a copier UI, using a touchscreen: user touches and drags, rotates. [CutlerLD97a] Culter, Lawrence D.; Froehlich, Bernd; and Hanrahan, Pat "Two-handed direct manipulation on the



responsive workbench", Proc. 1997 ACM Symp. on Interactive 3D Graphics, Providence RI USA, 1997 Two-handed 3D user interface using 6DOF stylus and PINCH gloves: bimanual symmetric and bimanual asymmetric actions, virtual dissection of a human body in Responsive Workbench. for PINCH glove, user can use multi-finger gestures (pinching to pick up, two-handed rotate gesture). Left hand positions/orients, right hand manipulates virtual tools. [DahburaAT96a] Dahbura, Anton T. "A Perspective on Ubiquitous Communications: the MoCCA Project", Slide Presentation, IEEE Boston Section, Multimedia Wireless at the turn of the century, October 24-25, 1996, Worcester, MA Newton MessagePad 130 as success, also Sharp Zaurus, HP OmniG 100, Casio Z-7000, USRobotics Pilot, Psion3A, but with limited communications/synchronization means. Prototype devices uses RoamAbout PC cards for wireless smartphone network. To include video capture. [DanielsPT96a] Daniels, Peter T. and Bright, William "Shorthand", "The World's Writing Systems", pp 807-ff, Oxford University Press, 1996 Examples of many short-hand handwriting systems, including Greek (1934), Tironian (1934), Bright's. Shows Medieval short-hand systems, including single-stroke / Unistroke systems. Willis' Stenography, Unistroke characters except for one "plus"-like two stroke from for "H", 1934. Early terminology for shorthand writing included symbolicals, arbitraries, wordsigns, tachyography, logographic, phonotypy [DankmanS96a] Dankman, Scott and Lastinger, Roc A. "Portable Electronic Platform System", United States Patent 5,491,609, February 13, 1996 Portable pen/touchscreen-computer with slide-in electronics modules for customization. TelePad. [DavidsonM97a] Davidson, Marc "Method and apparatus for adaptive touch recognition in a touch-sensitive user interface", United States Patent 5,627,567, May 6, 1997 For touchscreen targets / virtual keys/keyboard, accept near-misses to target based on nearest target, with a maximum expanded touch zone. Also a dead zone between the expanded touch zones. Expanded touch zone shapes may be modified to account for overlaps for different keys' zones. Notes that upstroke (lift stylus/pen from tablet) may be sloppy compared to location of downstroke (first touch/start of stroke). [DelBimboA97a] Del Bimbo, Alberto and Pala, Pietro "Visual image retrieval by elastic matching of user sketches", IEEE Trans. Pattern Analysis and Machine Intelligence, Vol 19 No 2, pp 121-132, February 1997 Cited in Arvo00 [DellaBonaMA98a] Della Bona, Mark A.; Dorfman, Jonathan; and Hamlin, Jay F. "Method and Apparatus for Contacting a Touch-Sensitive Cursor-Controlling Input Device to Generate Button Values", United States Patent 5,764,218, June 9, 1998 Activating UI buttons from a touchscreen digitizer based on gap intervals: how long the stylus/finger is pressed, and how log it is lifted. Compare with Kaplow and with Microtouch? press-and-hold? [DingeldeinD98a] Dingeldein, Dennis "Computer Graphics and Graphical User Interfaces", Available at http://www.zgdv.de/www.zgdv-uig/papers/gui/contents.html Informal research group on user-interface design: mentions work on single-stroke/unistroke gesture recognition by Rubine, others [DitzikRJ97a] Ditzik, Richard J. "Desktop Computer with Adjustable Flat Panel Screen", United States Patent 5,668,570, September 16, 1997 Portable computer (laptop) with fold-out screen, as contrasted to clamshell design. Mentions touchscreen input, rotation/orientation to landscape and portrait. [DoraiC98a] Dorai, Chitra; Wang, Gang; Jain, Anil K.; and Mercer, Carolyn "Registration and Integration of Multiple Object Views for 3D Model Construction", IEEE PAMI, Vol 20 No 1, Jan 1998, pp. 83-89 Registration (matching of similar regions) for single-camera views of object using ranging (distance) data, to construct 3D three dimensional model of surface. Distance from laser range scanner or digital interferometry. Compare with structure-through-motion. [DouglasSA97a] Douglas, Sarah A. and Mithal, Anant Kartik "The Ergonomics of Computer Pointing Devices", Springer Verlag, 1997, ISBM 3-540-19986-1 (partial copy) Joystick, including velocity joystick, slower and less usable than a mouse for pointing [DreifusH98a] Dreifus, Henry and Monk, J. Thomas "Smartcards: A guide to building and managing smart card applications", Wiley, 1998 (hardcopy book) Applications examples and deployment for smartcard applications, both with and without cryptography [DrewsPC98a] Drews, Paul C.; Held, James Pl; Kogan, Dan D. and Larson, James A. "Method and Apparatus for Redrawing Transparent Windows", United States Patent 5,831,615, November 3, 1998 Transparent tool windows over other windows: if underlying window is active, hide the transparent window (that is, hide it's toolbar and other markings); if transparent window is active, show toolbar and markings. Example is ink annotation transparent tool, when adding annotation show whole transparent window, when annotation complete show only the annotated ink but not the window markings. Compare with transparent ink overlay windows in Pen Windows? (not cited). Includes ink/sound editing (clean slate,



erase, back/undo) [DruinA97a] Druin, Allison; Stewart, Jason; Proft, David; Bederson, Ben; and Hollan, Jim "KidPad: A Design Collaboration between Children, Technologists, Educators", Proc. CHI 97, 22-27 March, 1997, pp. 463-470 User interface study with children using Pad++ drawing program with drawing tools: design suggestions from children included time-zoom (Zoom back in time). Touchscreen easier for children to use than mouse. Children saw semantic lens as an "X-ray" to see what was inside of a picture or text. See also NOOBIE of 1988, later video demonstration of graphical tools. [DubeyVN97a] Dubey, Venketeshwar Nath "Sensing and Control within a Robotic End Effector", Ph.D. Thesis, U. Southampton, UK, November 1997 Robotic actuator hand, includes photoelasticity touchpad/touch surface sensor: frequency-dependent delay in light when deflected/bent. Reviews also FSR, conductive rubber, piezo, and capacitive touchpad sensors: multiple pressure sensing sites. [DulbergM98] Dulberg, Martin S. and St. Amant, Robert "A Flying Click Gesture for Unary Selection and Activation", Technical Report TR-98-05, Department of Computer Science, North Carolina State University Flick / Flying Click mouse gesture faster than mouse click on a button [DymetmanM98a] Dymetman, M. and Copperman, M. "Intelligent Paper", Proceedings of joint conference EP'98/RIDT'98, March/April 1998. Available in "Electronic Publishing, Artistic Imaging, and Digital Typography", Springer Verlag LNCS 1375, pp. 392..406 Review of various forms of Intelligent Paper, linking paper data entry to electronic applications: unique page ID, input device, and communications to computers. Cites to digital desk (Wellner), contemporaneous handwritten memory notes as index to audio recording (lecture notes), digital data encoded in printed documents (SmartPaper, DataGlyphs), Active paper areas using electronic pen (Anoto? ), dynamic paper (electronic ink). Confetti: encoded tags stuck on paper documents (Compare with Accessibility labeling?) [Dynapro97a] Dynapro Systems Inc. "Dynapro Ergotouch Computer Model 3010 With Near Field Imaging Touch Screen User's Reference", Dynapro Systems Inc, Copyright 1997. www.dynapro.com Electrostatic/capacitive touchscreen digitizer, controller supports multiple simultaneous touches for safe operation (e.g. two simultaneous touches required to start a motor) or for mouse emulation (second touch to simulate mouse right-button activation for Windows context menus). TouchSurround: additional functional buttons on extended area of touchscreen display around display, programmable for the particular OS (Windows/OS2/Linux etc.) Mouse emulation in driver when touch is on display, keyboard emulation from icons/buttons outside the display. Capable of multi-touch? [Dynapro98a] Dynapro Systems Inc. "Dynapro NFI Near Field Imaging Touch Screen System, Software User's Guide", Dynapro Systems Inc, Copyright October 1998. www.dynapro.com TouchSurround touchscreen has active elements (fixed icons, etc.) in margins of touch digitizer outside frame of display: compare with Freestyle "Guide" hardware? [ElectronicProducts96a] Electronic Products "PCMCIA controller embodies Zoomed Video and 32-bit CardBus standards", Electronic Product, May 1, 1996 Databook Inc. PCMCIA card for use in portable computers (laptops, handhelds) to support videoconferencing. [EllerM98] Eller, Marlin and Edstrom, Jennifer "Barbarians Led by Bill Gates: Microsoft from the Inside", Owl Books, 1998 (electronic book) Marlin Eller's history of Microsoft tactics with GO Corporation, Pen/Windows, etc.: Contrast with "Startup" by Jerry Kaplan (of GO Corporation). Chapter "Pen Ultimate Warfare": quote describing Microsoft OS/APIs as a tax on a large market. Mentions initial contacts between GO Corporation and Microsoft prior to Pen Windows / Windows for Pen Computing. [EngberMS97a] Engber, Michael S. "Behind Bars: screen orientation and Button Bar", Newton Technology Journal, Vol 3 No 2, March 1997, pp. 1..6 Newton API for both sensing and setting screen rotation for any of four landscape/portrait modes. [FalconFD97a] Falcon, Fernando D. "Mouse driver arrangement for providing advanced scrolling capabilities to a conventional mouse", United States Patent 5,633,657, May 27, 1997 Inertial scrolling implemented in mouse driver, track mouse tail/cursor trail to determine direction of motion, acceleration, , generate discrete scrolling commands in specific directions. Coasting after mouse button is released, with decay in speed (similar to physical systems). User may use mouse for other input during coasting. Stop coasting motion with mouse button. Compare with Artificial Reality project? [FitzmauriceGW96a] Fitzmaurice, George W. "Graspable User Interfaces", Ph.D. Thesis, University of Toronto, Dept. of Computer Science, 1996 Time-multiplexed input user interface (mouse), space-multiplexed (multi-channel mixing panel). References Bricks, ActiveDesk, DigitalDesk. Wacom multi-stylus digitizer sensors used in prototypes, two-handed use of stylus and puck.



[FitzmauriceGW97a] Fitzmaurice, George; Baudel, Thomas; Kurtenbach, Gordon; and Buxton, Bill "A GUI Paradigm Using Tablets, Two-hands and Transparency", Proc. CHI '97, 22-27 March, 1997, Videos, pp 212-213 Puck, pen, and flipbrick on same multi-touch tablet (Wacom). Non-dominant (e.g. left) hand controls palette/tools such as rulers and french curves, dominant (e.g. right) hand holds stylus to draw. Also: Video file of related demonstration. [FitzpatrickGP96a] Fitzpatrick, Greg P. and Haynes, Thom R. "Overriding Action Defaults in Direct Manipulation of Objects on a User Interface by Hovering a Source Object", United States Patent 5,546,527, August 13, 1996 Similar to hold/dwell gesture with stylus, only user hovers (while dwelling) over object in proximity sensing of digitizer: dwell indicates different operations from the default operations. [FleckDC96a] Fleck, David C. and Tsuchida, Yoichi "Control method of a plurality of Locators and Control Device of Same", United States Patent 5,587,559, December 24, 1996 Electromagnetic or other grid tablet modified to work with more than one stylus or puck: multi-touch [FleckDC98a] Fleck, David C.; Stumpf, Richard M.; and Picard, Tom "Digitizer Eraser System and Method", United States Patent 5,793,360, August 11, 1998 touchscreen stylus has an eraser input on the back end. To erase, must first have selected text, then used eraser (with approximate gesture): then the actual "erase" operation generated (simulated keystroke input) depends on what kind of window you are on. Compare to recognition macros on Pencept. [FlemmingRW97a] Flemming, Richard W.; Wegen, Donald L. and Jenkins, Everett W. "Operator-controlled interactive communication device", United States Patent 5,638,427, June 10, 1997 Using a smartphone as a remote control unit for a device e.g. a copier. Compare with remote boot control, IBM Simon? [FlemmingTB96a] Flemming, Todd B. "Improving the Performance of the World Wide Web over Wireless Networks", Master's Thesis, Virginia Polytechnic Institute, Nov. 4. 1996 PDA/Wireless (intermittent, low-bandwidth) internet web browsing: connects to MHSP (Multiple Hypertext Stream Protocol) server supporting re-starts of connections, reduces bit resolution of images. [FlorenceJ96a] Florence, John; Hornsby, Kathleen; Egenhofer, Max "The GIS WallBoard: Interactions with Spatial Information on Large-Scale Displays", 7th Inter. Symposium on Spacial Data Handling (SDH '96), Delft, Netherlands, pp. 8A.1-15, August 1966 User interface for large white-board display using pop-up pie menus, data lenses. UI operations vary on user distance from wall display: arm's length (physical contact), spitting distance (gestures), within sight (external pointers and devices.) Prototype hardware is LiveBoard from Xerox [FoleyJD96a] Foley, James D.; van Dam, Andries; Feiner, Steven K.; and Hughes, John F. "Computer Graphics: Principles and Practice in C, 2nd Edition", Addison-Wesley, 1996 (hardcopy book) Computer graphics reference, 1996. Covers BitBlt (copyPixel/pixBlt/bit block transfer), optimization of offscreen bitmaps, mask to protect pixels/bits from copying: OR combines images rather than overwriting. BitBlt especially useful for overlapping windows GUI. Shading by halftone patterns/checkerboard. XWindows management of obscured windows by drawing to off-screen bitmap, so that when window is unobscured, copy/BitBlt of off-screen window image, rather than full re-draw. Interpolation in color space (alpha blending: raster subsystem), image compositing/blending -- cites to Porter-Duff mechanism 1984 for blending pixel colors. Terminology: locator devices, absolute/relative, direct/indirect, continuous (e.g. tablet)/discret. Precise positioning, Versatron footmouse, eye trackers. Cites to Lee '85 multitouch tablet (multiple finger positions) with indirect pressure/force sensing. 3D mouse, DataGlove -- not Polhemus? [Forcier96] Forcier, Mitchell D. "Script Character Processing Method and System with Bit-Mapped Document Editing", United States Patent 5,590,257, December 31, 1996 Editing of handwriting/electronic ink without recognition of characters Editing of electronic ink: word-wrap, inserting spaces, editing of handwritten lines of text van Raamsdonk possible related art? ... not cited. Press reports from 1999 of Forcier accusing Microsoft of infringement [ForsbergA98a] Forsberg, Andrew; Dieterich, Mark; and Zeleznik, Robert "The Music Notepad", Proc UIST 98, San Francisco, CA; pp. 203-210 handwriting recognition editor for musical notation. Uses stylus with two additional barrel buttons to select modes -- uses eraser end button of stylus for playback. Gestures similar to those of Buxton for music. Uses lasso and pigtail gestures for editing. Later ported to Microsoft Tablet PC for Microsoft. Uses length of time (press-and-hold) for shorter and longer periods to change duration of a note or to bring up editing menu. press-and-hold and threshold? On-line videos, also for Lineogrammer (Lineogrammar?) project [ForsbergAS98b] Forsberg, Andrew S.; LaViola, Joseph J. Jr.; and Zeleznik, Robert C. "ErgoDesk: A Framework for Two- and Three-Dimensional Interaction at the ActiveDesk", Proc. 2nd Int'l Immersive Projection Technology Workshop, Ames Iowa, May 11-12, 1998 ErgoSketch: tabletop touchscreen graphical editing prototype, with 2D pen input (dominant hand: stylus tip plus two side buttons) including stroke gestures, 3D tracker (non-dominant hand: hand-held device), and two-handed input. Cites to Rekimoto pick-and-drop.



[FoxA98a] Fox, Armando; Goldberg, Ian; Gribble, Steven D.; Lee, David C.; Polito, Anthony; and Brewer, Eric A. "Experience With Top Gun Wingman: A Proxy-Based Graphical Web Browser for the 3Com PalmPilot", Middleware '98, Proc. IFIP Conf. on Distributed Systems Platforms and Open Distributed Processing, pp. 407-424 Top Gun Wingman: proxy web-browser for small PDA PalmPilot, proxy middleware transforms web pages into simplified representation for PDA to process, thus off-loading PDA. [FrankEH97a] Frank, Edward H.; Naughton, Patrick J.; Gosling, James Arthur; and Liu, John C. "Method and apparatus for presenting information in a display system using transparent windows", United States Patent 5,651,107, July 22, 1997 Transparent windows in a GUI, using alpha blending of pixel data. Example show menu boxes with buttons overlaid on other windows. [FreeSoftwareFoundation97a] Free Software Founcation "Drag Start Distance", www.davidpilling.info/cd/DavidPilling/Maillist/1997_11.txt Technical support discussion on Drag Start Distance in open-source software. press-and-hold-and-drag distance. File contains additional discussion references to Drag Start Distance in KDE and other opensource software related to Linux O.S. [FreeSoftwareFoundation98a] Free Software Founcation "Drag Start Distance", http://140.116.2.3/Linux/CLE/CLE/devel/andrew/redhat/BUILD/kde-i18n/eo/messages/kdebase/kcminput.po Documentation of Drag Start Distance in open-source software. press-and-hold-and-drag distance. File contains additional discussion references to Drag Start Distance in KDE and other open-source software related to Linux O.S. [FreemanW97] Freeman, William T. and Weissman, Craig D. "Hand Gesture Machine Control System", United States Patent 5,594,469, January 14, 1997 Camera facing out from large display recognizes hand gestures. Appears similar to "Put-That-There" prototype at M.I.T. Compare with iPoint device from Fraunhofer Institut. [FrenkelEJ96a] Frenkel, Erik J. and Born, Jean-Jacques "Watch with Time Information via Silent Vibration", United States Patent 5,559,761, September 24, 1996 Accessibility Watch with (haptic) vibrations to indicate time for visual-impaired user. Also has tactile/embossed numeric points, tactile watch hands. [Fujitsu97a] Fujitsu Personal Systems "Fujitsu Point 510 Pen Tablet User's Guide", Fujitsu Personal Systems, 1997 Pen computer running Windows for Pen Computing. Includes note that rotating from landscape to portrait does affect tablet calibration, but re-calibration may be needed because viewing angle has changed (parallax). [Fujitsu98a] Fujitsu Personal Systems "Fujitsu Point 1600 Pen Tablet User's Guide", Fujitsu Personal Systems, 1998 Pen computer running Windows for Pen Computing. Includes note that rotating from landscape to portrait does affect tablet calibration, but re-calibration may be needed because viewing angle has changed (parallax). [FukudaR98a] Fukuda, Ryoji and Harada, Takashi "Data Processing Method and Apparatus", United States Patent 5,748,926, May 5, 1998 Recognize gestures in a handwriting recognition area on a tablet/touchscreen over an LCD display: compare with PenWindows? Gestures are mostly combinations of tap/press/drag. [FukumotoM97a] Fukumoto, Masaaki and Tonomura, Yoshinobu ""Body Coupled FingeRing": Wireless Wearable Keyboard", Proc. CHI '97, March 1997, Atlanta GA, pp. 147-154 Capacitive-base sensor for one-hand typing on any (grounded) surface: electrode sensors as wearable rings on each of five fingers, detected coupling through finger to ground when user touches a grounded surface, do five-finger chorded typing. [FukuzakiY97a] Fukuzaki, Yauhiro "Computer System having Multi-Device Input System", United States Patent 5,691,748, November 25, 1997 two digitizers (position pointing devices) on same tablet: mostly about user interface methods in a CAD system. (Wacom digitizer with two styli). Input device may be made integral / integrated with a flat display. [FurnessTA98a] Science News Daily "Virtual Retinal Display Wins 1998 Discover Magazine Technological Innovation Award", University Of Washington Science Daily, July 21, 1998, fetched 2013 www.sciencedaily.com Thomas A. Furness III, HIT Lab: retinal display paints image directly on human retina: used for persons with impaired vision. Full color image using RGB lasers. [GajskiDD96a] Gajski, Daniel D. "Principles of Digital Design", Prentice Hall, 1996 Undergraduate level digital design textbook. [GaschS96] Gasch, Scott "Alan Kay (biographical sketch)", http://ei.s.vt.edu/~history/GASCH.KAY.HTML Biographical sketch on Alan Kay: mentions BYTE 91 article, Kay had seen handwriting recognition system at RAND in 1968



[GeisWiki97a] GeisWiki "Omnigo FAQ for HP OmniGo 100 and 120", www.fortunecity.com/skyscraper/online/795/ogo/lite/faq.htm, 1995. Fetched 2013 from de.geos.wikia.com/wiki/OmniGo_FAQ FAQ information on HP OmniGo pen computer. PCMCIA slot, RS-232C, printer drivers for various printers (LasterJet, DeskJet, etc.) Graffiti handwriting recognition, small QWERTY keyboard, GEOS 2.1 Operating System OS. Screen viewable (UI rotates) for different orientations (landscape, portrait), drag-and-drop gestures. DOS mode (DOS 6.22 OS?) as a PC-XT. Supports digital cameras and Ethernet cards, radio modem (cellular network?). Sync to PC using serial cable at maximum speed. Photographs of internal hardware (touchscreen). [GeisslerJ98a] Geissler, Joerg "Shuffle, throw or take it! Working Efficiently with an Interactive Wall", Proc. CHI '98, pp. 265-266 "Throw" object on a large whiteboard using pick-and-throw or pick-and-drop: dwell/press-and-hold gesture to pick up an object, then short connected stroke away from target, then rapid flick stroke in direction of target: object is thrown distance corresponding to velocity of throw stroke. Does not describe simulated friction/acelleration? [GeisslerJ98b] Geissler, Joerg ("contact person") "AMBIENTE activity: InteracTable", www.darmstadt.gme/de/ambitente/activities/interactable.html, December 6, 1998 InteracTable touchscreen table (rear projection). Shows windows rotated on tabletop display for varying user perspectives around table. Objects may be rotated and shuffled on the surface for viewing by differing users, manipulation done by gestures using fingers or pens. Electronic ink and voice annotations. [GeneralMagic96a] General Magic "MagicCap: Magic Internet Kit Programmer's Guide", General Magic, 1996 Internet API and programming on General Magic Magic Cap OS: multithreading with actors -- object with Main that executes when object is created, until destroyed. RunNext call for cooperative multitasking (no preemption). [GeneralMagic97a] General Magic "DataRover 840 Getting Started Guide", General Magic, 1997 Basic user guide for DataRover 840 stylus-input PDA, using resistive touchscreen and stylus. [GerpheideGE96a] Gerpheide, George E. and Layton, Michael D. "Capacitance-based proximity with interference rejection apparatus and methods", United States Patent 5,565,658, October 15, 1996 Touchpad or touchscreen tablet with X and Y conductors, use measurement signal as reference signal for demodulating received signal to eliminate environmental noise: use pseudo-random number generator to generate signal for additional rejection of noise, since noise unlikely to be on same frequency with same modulation. Multi-touch capability? [GerpheideGE97a] Gerpheide, George E. "Apparatus and Method for Tactile Feedback from Input Devices", PCT Patent Application WO9718546, May 22, 1997 Touch tablet with texture feedback via haptic transducer. Regions may also be hard-labeled (indicia) on tablet surface. [GiangrassoD96a] Giangrasso, Dominic "Casio Planeo", Pen Computing Magazine, # 10 1 May/June 1996, www.pencomputing.com Review of Casio Planeo color PDA: pen taps on small displayed keyboard on touchscreen, sketch function for freeform electronic ink: no reference to character recognition? [Gillespie96] Gillespie, David; Allen, Timothy P.; and Wolf, Ralph "Object Position Detector with Edge Motion Feature and Gesture Recognition", United States Patent 5,543,591, August 6, 1996 Conductive digitizer responds to finger touch, plus recognition of tap, hop, zig-zag, push and dragging gestures (Unistroke?). Refers to compensating for unintended motion during gestures. Mentions multiplefinger gestures, such as tap+drag (two strokes), uses in-proximity timing, hop gesture: double-tap with distance between the two taps. Two-finger gestures by recognizing sudden shift in single reported position when a second finger is touched. Edge motion: rapid scrolling action if touch is in margins of the touchpad. Compare with press-and-hold? Cites surface-acoustic-wave SAW digitizers as related art to conductive digitizer. Cites Alps/Cirque GlidePoint with gesture recognition [GillickWG96a] Gillick, William G and Lam, Clement C. "Roller Mouse for Implementing Scrolling in Windows Applications", United States Patent 5,530,455, June 25, 1996 Hardware user interface, roller mouse with scroll wheel, in which scrolling continues dynamically after motion has stopped. Multiple input stroke accelerate scrolling, reverse stroke (on scroller wheel) decellerates/stops scrolling. Refers to dynamic scrolling, momentum scrolling, additional scrolling input changes momemtum or speed (decrease or increase) [GloorPA96] Gloor, P.A. "Elements of Hypermedia Design: Techniques for Navigation and Visualization in Cyberspace", Coopers and Lybrand, Switzerland, 1996 Hypermedia indexing and organization: links, etc. Mentions Mouse gestures for actions on nodes of a cone tree (3D tree) [Goldberg97] Goldberg, David "Unistrokes for Computerized Interpretation of Handwriting", United States Patent



5,596,656 assigned to Xerox Corporation, Stamford, Connecticut, January 21, 1997 Patent on "eyes-free" writing by constraining every character to a single stroke: many characters have no similarity to the customary handwriting style Subject of a patent dispute between Xerox and U.S. Robotics over the Palm Pilot character recognition "Graffiti", which used mostly single strokes Unistroke patent by Goldberg was re-examined: see hardcopy file under Goldberg, numerous references to shorthand and other writing stylusess for unistroke characters [GolovchinskyG98a] Golovchinsky, Gene; Price, Morgan N.; and Shilit, Bill N. "From Reading to Retrieval: Freeform Ink Annotations as Queries", Proc. SIGIR '99, August 1999, Berley, CA pp. 19-20 XLibris: electronic ink annotation, automatic indexing. Queries generated automatically from text that user selects/circles. Notes that users tend to print out large numbers of documents to search through, rather than narrowing down on-line, in part to be able to annotate / mark up: annotations aid in reading. Prototype uses Mutoh tablet/display on PC. Cites to linking by inking papers. [GonzalesGR98a] Gonzales, Gilbert Rene "Tactile communication device and method", United States Patent 5,719,561, February 17, 1998 6x4 array of tactile dots as tactile output device: compare with optacon for the blind. Dots are actuated in sequence to trace out shape of letters and numbers. Single character, example is wrist mounted. [GoodmanB98a] Goodman, Brian "Child-Friendly Collaborative Digital Environments", Thesis, Hampshire College, Division III, Fall 1998 Study of KiddyFace, Milekic touchscreen-based museum display, intuitive use of direct-manipulation interfaces with graphical objects with children (age 4 up). Coball prototype ball-in-hole game, Codrow graphical environment, Coview matching game, Cosound acoustic/audio objects discovered by touching, hidden hotspots, audio (not haptic) feedback . Cites to Ben Shneiderman on direct manipulation, but not to Buxton. Refers to multiple inputs (multi-touch), force/pressure on touchscreen, but input position reported is single midpoint: used to create collaborative Coball game for two kids/users. [GoslingJ98a] Gosling, James "A Brief History of the Green Project", James Gosling, 1998, blog posting. Fetched 2013 from www.anonymous-insider.net/java/research/1998/05-a.html Green Project at Sun Microsystems, produced Star 7 ("*7") PDA-like device with touchscreen and inertial scrolling. Original hardware project (set-top TV box) that initiated Oak/Java language. Animated user interface reminiscent of Microsoft "Bob". [GoughML97a] Gough, Michael L.; Venolia, Daniel S.; Gilley, Thomas S.; Robbins, Greg M.; Hansen, Daniel J. Jr.; Oswal, Abhay; and Tam, Tommy H. "Method and Apparatus for Displaying an Overlay Image", United States Patent 5,638,501, June 10, 1997 Semi-transparent overlay virtual keyboard image in pen computer (touchscreen). [GoughML97b] Gough, Michael L. and Holloway, Bruce V. "system for Entering Data into an Active Application Currently Running in the Foreground by Selecting an Input Icon in a Palette Representing Input Utility", United States Patent 5,603,053, February 11, 1997 Pop-up virtual keyboard, other tools available from a detachable palette or tray. Cites to Pen Windows and to PenPoint. Compare with Ward '495 patent about virtual devices and off-screen palette? [GrayBL96a] Gray, B.L. and Fearing, R.S. "A Surface Micromachined Microtactile Sensor Array", IEEE Int. Conf. Robotics and Automation, April 1996 Tactels (touch pixels), very fine 2D touch sensor array under rubber/elastomeric sheet to distribute force over nearby tactels. Mechanical capacitive sensors, measure change in capacitance as deformed. Does not investigate multi-touch aspect. [GreenbergS96a] Greenberg, Saul; Gutwin, Carl; and Cockburn, Andy "Using Distortion-Oriented Displays to Support Workspace Awareness", Research report 96/581/01, Dept. of Comp. Sci., U. Calgary, Calgary, Canada, November 2006 Fisheye transparent magnifying window viewer: compare to Zoomtext (not cited), Magic Lens, Tool Glass. [GribnauMW98a] Gribnau, Maarten W. and Hennessey, James M. "Comparing Single- and Two-Handed 3D Input for a 3D Object Assembly Task", Proc. CHI '98, 18-23 April 1998, pp 233-234 Mouse-like 3D input device with two buttons, and 6DOF three-dimensional electromagnetic sensor, one for each hand (bimanual) [Griepentrog96a] Griepentrog, Scott "Pen Services 2.0 - What is Microsoft doing?", Wiki posting at: twiki.stg.net Technical note on change in gesture and handwriting interface between Windows for Pen Computing Pen Services 1.0 (for windows 3.11) and 2.0 (for Windows95). Lists gestures under Windows 3.x: directional tab for space, newline, curlicue for cut and undo, sideways curlicue for copy, chevron for paste, diagonal up-down for delete and backspace, checkmark for edit text, etc. Replaced in Windows 95 Pen Windows with edit box and incomplete pop-up keyboard without arrow keys, others: some character recognition errors thus cannot be corrected. [Griepentrog96b] Griepentrog, Scott "Pen Windows Machine support page", Wiki posting at: twiki.stg.net Downloads for Windows for Pen Computing, Pen Services 1.0 drivers, also for Pen Services 2.0, WINTAB



driver for NT/95, PEN DOS drivers. [Griepentrog96c] Griepentrog, Scott "We have a friend in Bill ...", Wiki posting at: twiki.stg.net 1993(?) Interview with Bill Gates of Microsoft, he mentions using (or not using) handwriting recognition on his laptop -- mentions that it (Pen Windows) does not work very well [Griepentrog96d] Griepentrog, Scott "Word for Windows 95 Pen Support", Wiki posting at: twiki.stg.net Pen Annotation feature of Word 6.0 for Pen Windows not functional in Word 7.0 for Windows 95 [Griepentrog96e] Griepentrog, Scott "Pen Services for Windows 95", www.pencomputing.com Describes changes from Pen Services 1.0 to Pen Services 2.0 for Windows for Pen Computing: squiggle gestures of 1.0 replaced by under-arrow gesture, with circle around a letter, e.g. circle-S for insert a space. Macro definition of keysequence to circle-letter gesture dropped from 1.0. Alternative recognition: clicking on letter brings up display of alternative recognition results. (Note: check patent files). GRECO and MARS handwriting recognizers Many features removed vs. PenWindows1.0: on-screen keyboard has no function keys. Tab, enter other gestures replaced by "under arrow" below cursor: compare with on-screen pie menus and marking menus. [GrossMD96a] Gross, Mark D. and Do, Ellen Yi-Luen "Demonstrating the Electronic Cocktail Napkin: a paper-like interface for early design", Proceedings of CHI 96, available at http://depts.Washington.edu/dmachine/PAPER/CHI96/demo.html Trainable gesture recognition of multi-stroke glyphs, use of graphical context, collaborative drawing Cites earlier work on gestures and sketch recognition back to 1991: Rubine gesture recognizer Calligraphic drawing http://wallstreet.colorado.edu/Napkin [GrossMD96b] Gross, Mark D. and Do, Ellen Yi-Luen "Ambiguous intentions: A paper-like interface for creative design", Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology, page 183192, Seattle, Washington, 1996 Sketch/diagram recognition and editing: refers to "intelligent paper". Gesture recognition of glyphs by a positional chain code / zone recognizer algorithm using 3x3 path grids, plus detection of sharp corners. Straight lines and dots (anything small) by special-case algorithm. [GrossR97a] Gross, Ralph "Run-On Recognition in an On-line Handwriting Recognition System", Technical Report, June 1997, Carnegie-Mellon University and University of Karlsruhe NPen++ neural net character recognizer, use of heirarchical word dictionary for HMM word matching [GroverDL98a] Grover, Dale L.; King, Martin T.; and Kushler, Clifford A. "Reduced Keyboard Disambiguating Computer", United States Patent 5,818,437, October 6, 1998 Full-alphabet typing using touch-tone keypad: combination of spelling dictionary to look up likely words, give user a pick-list, and using multiple key strokes to get the character you want. "Adaptive". Multiple references (see file). Cites to Knowlton, Underkoffler. [HabibDM98a] Habib, Dennis M.; Engelberg, Mindy J.; McNutt, Kathryn A.; Kinser, Colby E.; Lynch, Charles E. II; and Williams, Chauncey L. "Help system with semitransparent window for disabling controls", United States Patent 5,825,356, October 20, 1998 Help system: text displayed in windows that are transparent, user can still see what is underneath. Figs. 5 and 5 show semi-transparent windows over other windows. See bottom of Col 16. [HakenL98a] Haken, Lippold; Tellman, Ed; and Wolfe, Patrick "An Indiscrete Music Keyboard", Computer Music Journal, Vol 22 No 1, pp. 30-48, Spring 1998 Multi-touch pressure (force)-sensitive music keyboard surface Multi-touch musical keyboard, tracks up to 10 fingers in X, Y, and Z (pressure/force). Describes several multi-touch (X/Y/Z) technologies. Hall-Effect sensor and magnets suspended on compressible medium for Z, conductive/resistive rubber strips for Y, separate strips (keys) for X. Musical multi-touch playing surface: Photoelastic playing surface light by polarized light from underside, greater pressure causes change in polarization of photoelastic material, creating visible rings around touch points. CDD camera(s) under surface detect location and number of rings for X, Y, and Z. Musical multi-touch playing surface: Conductive rubber sheet with high horizontal (X/Y) impedance/resistance, lower vertical (Z) resistance). High horizontal impedance allows touch points to be electrically isolated, array/matrix of detecting pads underneath rubber detect/measure voltage and current from top sheet over rubber. [HaleRD98a] Hale, Rodney D. "Gesture Recognition as a Means of Human-Machine Interface", Master's Thesis, Eng. and Appl. Sci., U. Newfoundland, July 1998 Study of recognition of hand gestures using flock-of-birds 3D digitizer/tablet (Ascension Technology Corp.). States that hidden Markov models gave lower recognition results, but were more interesting (?). Analyzes results using confusion matrices. [HannRL97a] Hann, R.L. "A Conversation with Stuart Card", Cseriac Gateway, Vol VIII No 1, 1997, pp 11..12 Invention of the mouse, and introduction circa 1983: long-lived and successful, because the limitations were in eye-hand coordination, not in the mouse hardware or electronics. Compare with performance characteristics of digitizer tablets.



[HansenDJ96a] Hansen, Daniel J. Jr.; Gough, Michael L.; and Hollowell, J. Rhoads "Method and apparatus for using stylus-tablet input in a computer system", United States Patent 5,534,893, July 9, 1996 Detecting strokes on a touchscreen display. Refers to "ink pool memory" to keep track of (uninterpreted) strokes. Cites to "Using Penpoint". Compare with PenPoint implementation of strokes, Pen Windows implementation of ink objects? Barrel button (toggle on the stylus barrel) to change input modes e.g. forcing stroke to be X or Y axis only. [HansenGA97a] Hansen, George A.; Robnett, Robert C.; Lozares, Lawrence; and Mernyk, Paul A. "User interface with individually configurable panel interface for use in a computer system", United States Patent 5,659,693, August 19, 1997 User interface with main panel module (window) and additional panel modules (windows). Tool window can be rotated for landscape or portrait orientation. [HansonKJ96a] Hanson, Karrie J. and Kochanski, Gregory P. "Stylus-based keyboard key arrangement", United States Patent 5,483,235, January 9, 1996 On-screen keyboard layout: circular, arranged dynamically based on dictionary likelihood of next character. May show multiple instances of same character key so that the most probable next characters are adjacent to the most recent character. Shown with touchscreen keyboard: compare with layouts developed for IBM Simon? Prototype looks like early cellphone/smartphone with alphabetic keyboard. [HargroveRR97a] Hargrove, Richard R. "Quickselect icon button on a computer display which redisplays the last view style activated by the icon button", United States Patent 5,602,981, February 11, 1997 Dual-selection drag gesture: mouse button down selects object, mouse button object over icon selects action to be performed on object. Compare with drag-and-drop gesture onto an icon? [HarrisonBL96a] Harrison, Beverly L. and Vincente, Kim J. "An Experimental Evaluation of Transparent Menu Usage", Proc. CHI 96, April 13-18, 1996, pp. 391-398 Legibility and usability in transparent menus (including tear-off / pop-up menus): font size/style, size, outlining. Cites to early work by Knowlton on relabled keys. [HarrisonBL97a] Harrison, Beverly L. "The Design and Evaluation of Transparent User Interfaces: From Theory to Practice", Cognitive engineering Laboratory report CEL 96-10, U. of Toronto, Canada, 1996. Copyright 1997 as PHD thesis Alpha-blending transparency (p. 186), also use of additional hardware overlay planes for display. Notes that X11 visual model does not support layering. Refers to transparent overlays of windows as "depth multiplexing", on contract to spacial multiplexing (side-by-side windows) and time mutiplexing (overlapping opaque windows, only one visible at a time on top). Transparency via stippling (screen-door) or dithering. Physical overlay (pop-up) planes in hardware for showing overlaid graphics (compare with Freestyle "Guide"). [HarrisonBL98a] Harrison, Beverly L.; Fishkin, Kenneth P.; Gujar, Anuj; Mocon, Carlos; Want, Roy "Squeeze Me, Hold Me, Tilt Me! An Exploration of Manipulative User Interfaces", Proc. CHI 98, April 13-23, 1998, pp. 17-24 Direct manipulation interface / tactile/haptic user interface: used Casio Cassiopeia hand-held computer and Palm Pilot PDA. Page-turning gesture, grasp/squeezing gesture, compare with flick gesture. [HearnD96a] Hearn, Donald and Baker, M. Pauline "Computer Graphics, C Version 2nd edition", Prentice-Hall, May 1996 Computer graphics reference work, 1996. Describes BitBlt/pixBlt/bit-block transfer as one-bit case of general rasterOp raster operation, with mask. BitBlt boolean operations: exclusive or: executing same operation twice returns target to original. BitBlt can do 90 degree rotations: other angles require blending/interpolation of pixels. Semitransparency via blending of background and foreground pixels (without alpha factor for alpha blending) using opacity/transparency parameter/coefficient. [HeindelLE97a] Heindel, Lee Edward; Kasten, Vincent Alan; and Schlieber, Karl Johannes "Methods and apparatus for managing information on activities of an enterprise", United States Patent 5,655,118, August 5, 1997 Project and resource management system [HendersonA97a] Henderson, Austin "Tailoring Mechanisms in Three Research Technologies", Proc. of Group '97, 1997 Trillium user interface for copier interfaces (1983), Buttons as first class objects 1991, Usertailorable/definable buttons 1991. Compare with Gesture Buttons? [HendryI97a] Hendry, Ian and Puckett, Michael "System for dynamically accommodating changes in display configuration by notifying changes to currently running application programs to conform to changed configuration", United States Patent 5,682,529, October 28, 1997 PDA/smartphone (Apple Newton): automatically rotate user interface on display when display changes configuration e.g. rotation between landscape vs. portrait orientation without restarting system. Applications re-layout UI, if they do not then the UI is displayed appropriately in new configuration. [HerbertBK98a] Herbert, Brian K. "Touch Sensitive Flat Panel Display", United States Patent 5,777,596, July 7, 1998



Capacitive touchscreen using inherent change in capacitance of normal LCD pixels, by measuring charge time of pixels. Self-capacitance? Shows image of fingertip touch. Requires that LCD display pixels be changed to detect touch: blinking cursor, or blinking row. [HertzfeldAJ97] Hertzfeld, Andrew J.; Atkinson, William D.; and Kare, Susan D. "Graphic User interface for Modifying Object Characteristics using Coupon Objects", United States Patent 5,611,031, March 11, 1997 Drag and Drop of attributes (tags) from one object to another object [HertzmannA98a] Hertzmann, Aaron "Painterly Rendering with Curved Brush Strokes of Multiple Sizes", Proceedings of SIGGRAPH 98 Student of Perlin: simulation (no actual stylus input) of stroke direction to simulate oil paintings [HightowerRR98a] Hightower, Ron R; Ring, Laura T.; Helfman, Jonathan I.; Bederson, Benjamin B; and Holland, James D. "Graphical Multiscale Web Histories: A Study of PadPrints", Proc. HyperText 98, Pittsburgh PA, pp. 58-65 Pad++: zoomable/zooming graphical U/I for web browser to show small tree view hierarchy of browsing history. Different sizes of thumbnails depending on depth in tree. [HillenmayerS98a] Hillenmayer, Stefan "Communication device for mobile operation having a telephone and notebook with display", United States Patent 5,719,936, February 17, 1998 Multimedia smartphone / communication device: touchscreen, uncoverable keyboard, video display and camera [HinckleyK97a] Hinckley, Ken; Pausch, Randy; and Proffitt, Dennis "Attention and Visual Feedback: The Bimanual Frame of Reference", Proc. 1997 ACM ymposium on Interactive 3D Graphics, Providence, RI Experiment to show that Two-handed/bimanual input not just natural, but more productive, in part because depends less on visual feedback (prioreceptive sense?). bimanual/multi-touch? [HinckleyK98a] Hinckley, Ken; Czerwinski, Mary; and Sinclair, Mike "Interaction and Modeling Techniques for Desktop Two-handed Input", Proc. UIST'98, pp. 45-ff. Two-handed (puck + stylus) multi-touch stretch (pinch) gesture [HinckleyK98b] Hinckley, Ken; Pausch, Randy; Proffitt, Dennis; and Kassell, Neal F. "Two-handed Virtual Manipulation", ACM Trans. on Computer-Human Interaction, Vol 5 No 3, September, 1998, pp. 260-302 Two-handed user interface for neurosurgeons. User interface uses virtual user interface props, e.g. an instrumented dolls head in one hand to manipulate a 3-D image of a brain, and a pointing/slicing pointer in other hand to manipulate virtual slices, plus a touch screen for command and other interactions. References to two-handed input and direct manipulation. [HoSB96a] Ho, Seng Beng "Computer user interface system and method having book image features", United States Patent 6,064,384, May 16, 2000 Desktop book metaphor for UI user interface: show information in heirarchy as a set of books, a book a set of pages that can be flipped/scrolled, objects on the pages. Compare with PenPoint? Includes touchpad (force and position sensor: Interlink thin-film linear potentiometer) for scrolling. [HolmerT98a] Holmer, Torsten; Lacour, Laurent; and Streitz, Norbert "i-LAND: An Interactive Landscape for Creativity and Innovation", CSCW '98, April 1998, Los Angeles: Video presentation Video demonstration of i-LAND -- may be same video as CHI '99. InteracTable, DynaWall, CommChair: Passage tagged objects recognizable by touchscreen. Video listed in archive at http://www.mprove.de/uni/asi/video.html (2013). [HourmandB98a] Hourmand, Byron "Capacitive responsive electronic switching circuit", United States Patent 5,796,183, August 18, 1998 "zero-force" touch switch: Capacitive touch/proximity sensor, finger affects oscillating frequence of a tuned circuit. Arranged in array/matrix/grid of touchpads for touch keyboard (or touchscreen panel). [HuJ97a] Hu, Jianying; Rosenthal, Amy S.; and Brown, Michael K. "Combining High-Level Features with Sequential Local Features for On-Line Handwriting Recognition", ICIAP (2) 1997: 647-654 Extract features from on-line cursive (sometimes broken) handwriting recognition input (cusps, crossings, loops), spread location across adjacent points, stays better classification results. Filter out very small loops, collapse to a cusp. Results of improvement are modest: stated in terms of reduction in error rate, does not state Type I or Type II errors, or overall error rate. [HuJ98a] Hu, Jianying; Lim, Sok Gek; and Brown, Michael K. "HMM Based Writer Independent On-line Handwritten Character and Word Recognition", Proceeding of 6th International Workshop For Handwriting Recognition, Taejon City, Korea, August 1998 Handwriting recognition: combine language grammar model (character pairs?) as context constraint for improved results. [HuerstW98a] Huerst, Wolfgang; Yang, Jie; and Waibel, Alex "Error Repair in Human Handwriting - An Intelligent User Interface for Automatic On-Line Handwriting Recognition", IEEE International Joint Symposia on Intelligence and Systems, May 21..23, 1998, pp. 389-395 Addition of a UI for correcting handwriting recognition errors to the NPen++ handwriting recognition system: user can write over or cross out erroneous script recognition. User can write an additional



character in the middle of a script-writing handwritten word. Delayed strokes (dots over lower-case i or j; cross-marks on a t) are thus also considered to be "repair" user interface. [HughesRG96a] Hughes, R.G. and Forrest, A.R. "Perceptualization using a Tactile Mouse", IEEE Visualization 96, pp. 181-187 Mouse designs with haptic tactile feedback, equivalence of sonification (audio feedback) and touch. Use of touch feedback to convey information not associated with touch: e.g. map vegetation and map elevation. Definitions of haptic/kinesthetic/tactile feedbacks. [Husky97] Husky Computers Limited "FC-PX5 User's Guide", Husky Computers Limited, Coventry England (abstract only) Ruggedized pen computer using Pen Windows. Text emphasizes keyboard over gestures. "Microsoft Windows Guide to Pen Computing" [IBM96a] IBM "IBM 4612 Sure Point Mobile Computer "Fast Path - DOS/Windows" Software Installation", ftp.software.ibm.com/software/retail/poseng/posthw/fastdos.txt Installation instructions for IBM Windows for Pen Computing pen computer: installation steps include selecting landscape/portrait from screen rotation icon / "Rotate" icon to rotate screen landscape/portrait. [IBM97a] IBM "Secure Electronic Transactions: Credit Card Payment on the Web in Theory and Practice", IBM International Technical Support Organization, Raleigh Center, June 1997 IBM SET standard for protocol for electronic payments (e.g. credit card) [ITL97a] NIST National Institute of Standards and Technology "Public Key Infrastructure Technology", ITL Bulletin, July 1997 Introduction to X.509 certificate/CA/PKI standard, interoperability standards of NIST. Points out technical challenges of getting updated CRLs certificate revocation lists: does not mention OCSP on-line certificate status protocol. [Igarashi96] Igarashi, Takeo; Kawachiya, Sachiki; Matsuoka, Satoshi; and Tanaka, Hidehiko "A Pen-Based Constraint Drawing System Combining Dexterity and Precision", Proc. GCAD 96 (In Japanese, English-language abstract) pen-based stylus-based sketch program with pie menus and sliders, beautification/sketch recognition(?) implemented on "IBM pen PC" and Xerox Liveboard [InterLink98] InterLink Electronics, Inc. "New Versapad Touchpad Sensor", InterLink Electronics, Camarillo, California Thin touchpad digitizer: quotes Jeff Dao of CIC / Communications Intelligence Corporation. PenOP E-sign company. CyberSign (www.cadix.com) signature verification, Sony as a customer for the PCG 505 Touchpad digitizer claims 1000 lines per axis / 200 counts per inch: implies resistive sheet digitizer [Interfaces97a] Istance, Howell "Eye-based Interaction for Users with Special Needs: Visual Keyboard", Interfaces: Human-Computer Interaction, Number 35, Summer 1997, published by British HCI Group, pages 9-12 Accessibility visual/on-screen keyboard with head-pointing device. Distinguishes in user interface between precision and accuracy. Virtual keyboard has sections left of command keys, right of keypad/keyboard selector, center is actual keyboard/keypad image. Zoom key for mouse emulation. Modified keyboard layouts. [Intermec98a] Intermec Technologies Corporation "110 Hand-Held Computer Technical Reference", Intermec Technologies Corporation, Cedar Rapids Indiana, Copyright 1998 Pen computer using Windows 95 for Pen Computing / PenWindows 2.0 or DOS. Half-size (240x320) pixels with display panning for 640x480. Display modes landscape / portrait (four modes) rotated by system software, resistive analog touchscreen, alphabetic keypad: full-size keyboard can be attached. [Intermec98b] Intermec Technologies Corporation "6632 Pen Computer Technical Reference", Intermec Technologies Corporation, Cedar Rapids Indiana, 1998 Pen computer using Windows 95 for Pen Computing / PenWindows 2.0, mentions screen rotation landscape/portrait. [IshiiH97a] Ishii, Hiroshi and Ullmer, Brygg "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms", Proc. CHI '97, March 22-27, 1997, 6 pages Tangible user interface using objects instrumented as touch/location input devices. Two-handed (multitouch) rotation of maps on tabletop display. Compare with Wellner? See also Ullmer 97 video demonstration [ItoK98a] Ito, Kenji "Integrated construction project information management system", United States Patent 5,761,674, June 2, 1998 Project planning and management system combining a heirarchy of tasks and a heirarchy of physical components. Multiple views: project, organization, construction planning, construction management, etc. [ItoM97a] Ito, Masanobu; Motohashi, Shunji; and Higuchi, Yoshinari "Video apparatus with image forming means responsive to touch sensitive display", United States Patent 5,671,014, September 23, 1997 Resistive touchscreen (with stylus) user interface for video camera.



[JacobRJK96a] Jacob, Robert J.K. "Human-Computer Interaction: Input Devices", ACM Computing Surveys, Vol 28 No 1, March 1996 Taxonomy of user input as discrete (keyboard), continuous (some kind of relative or absolute pointing device: mouse, tablet, rotary dial, sliders, etc.), Voice, etc. [JainAK97a] Jain, Anil K. and Zongker, Douglas "Representation and Recognition of Handwritten Digits Using Deformable Templates", IEEE PAMI, Vol 19 No 12, December 1997, pp 1386..1391 Deform outlines of bit-map templates (?) prior to computing degree of matching: compare with elastic matching by Tappert Optical recognition of handwritten digits from a 2,000 sample subset of NIST Special Database 1 of handwritten characters from ZIP codes: also mentions an IBM data collection. Electronic file (C2) has CV for author at Read-ink technologies lower-case handwriting recognition. See also Zongker publication "Chicken Chicken" [JambhekarSN98a] Jambhekar, Shrirang N.; Bleimann, Gregor E.; and Pisutha-Arnond, S. Num "Method for implementing icons in a radio communication device", United States Patent 5,848,356, December 8, 1998 Smartphone/communicator: show phone/email contacts in groups, represented by icons. Also shows onscreen/virtual keyboard screen in both landscape and portrait orientation/rotation. [JanikCM98a] Janik, Craig M. "Wearable computing device with module protrusion passing into flexible circuitry", United States Patent 5,798,907, August 25, 1998 Wearable computer with electronics in a belt or utility harness, head-mounted display (semi-transparent single occular) [JohnsonNF98a] Johnson, Neil F. and Jajodia, Sushil "Steganalysis: The Investigation of Hidden Information", Proc. IEEE Inf. Techn. Conf., Syracuse, New York, September 1..3, 1998, pp 113..116 Overview of steganography for text (e.g. extra spaces/line breaks in HTML), digital images, steganography (hidden) file system, etc. Steganalysis/detection by analysis of multiple instances to detect patterns of modification. Disabling measures to destroy steganographic content by adding noise (of similar characterists to steganographic data). (Check references) [JohnsonNF98b] Johnson, Neil F. and Jajodia, Sushil "Steganalysis of Images Created Using Current Steganography Software", Springer Verlag. Information Hiding: Lecture Notes in Computer Science Vol. 1525, 30 Nov 1998, pp 273..289 Overview of defences against steganographic messages a.k.a. invisible watermarks: detection, and destruction. Counterfeit watermarks can be detected / destroyed by same techniques. Detection by comparing with original image, atypical "noise" is steganographic image. [JosephER96a] Joseph, Eugene R. and Tractman, Michael "Computer Modelling System and Method for specifying the Behavior of Graphical Operator Interfaces", United States Patent 5,405,600, January 16, 1996 Animation of user interface by example [KaiT97a] Kai, Tsutomu and Matsunami, Masahito "Coordinate Input Device", United States Patent 5,606,346, February 25, 1997 capacitive/electrostatic touchscreen digitizer with X and Y electrodes integrated between pixels of an LCD panel. Shows "wavy line" distortion of grid, even after double measurement to reduce. Mentions multiple measurements to get better resolution [KajiwaraS98a] Kajiwara, Seiichi "Image processing system and its method and electronic system having an image processing system", United States Patent 5,848,201, December 8, 1998 Algorithm for rapid calculation of rotated/scaled view of windows/objects in X/Y/Z orientations in real time, also transformations such as sphere formula (see App Lens?). [KambaT96a] Kamba, Tomonari; Elson, Shawn A.; Harpold, Terry; Stamper, Tim; and Sukaviriya, Piyawadee "Noi" "Using small screen space more efficiently", Proc. CHI 96, April 13-18, 1996, Vancouver, Canada pp. 383-390 Small touchscreens on portable computers (pen computers) mean on-screen widgets use up much of the screen, especially virtual keyboard. Solution is transparent (semi-transparent) widgets and virtual keyboard. Work better if partially-obscured objects respond only after a slight delay (press-and-hold?) Makes comparison with marking menus. [KayA98] Kay, Alan "History of the Character Recognizer; Was: Interaction between Keyboard and Mouse", Posting to SQUEAK Mailing List November 7 1998, archived at http://minnow.cc.gatech.edu/squeak/605 Refers to early character recognition / pen-input on Dynabook, and why it was dropped from the prototype: Refers to GRAIL recognizer as "near perfect". Describes early GRAIL system as using Unistrokes, because of fluidity of user interaction, user would learn the recognizer. Describes one step beyond unistrokes, recognition before pen is lifted (compare to Jourjine unpublished demonstration at Wang 1991), said incremental recognition on the fly also done in GRAIL (see refs?) [KellerNM98a] Keller, Neal Martin and Pickover, Clifford Alan "Visual Access to Restricted Functions Represented on a Graphical User Interface", United States Patent 5,821,933, October 13, 1998 User can make up graphical/iconic password by selecting icons to touch, in an order, and positioned at particular locations. Slide to unlock?



[KeysonDV96a] Keyson, Davic Victor "Virtual Workspace with User-programmable Tactile Feedback", International Patent Application WO 96/07965, march 14, 1996 Software to provide tactile feedback to match regions of a workspace or GUI, based on pre-defined characteristics, so that the tactile haptic effects match the workspace [KiernanCL97a] Kiernan, Case L. and Jancke, Gavin "Method for separating a hierarchical tree control into one or more hierarchical child tree controls in a graphical user interface", United States Patent 5,701,137, December 23, 1997 Project and resource management system [KolnickFC96a] Kolnick, Frank C. "Object-oriented software architecture supporting input-output device independence", United States Patent 5,502,839, March 26, 1996 Graphical user interface combined with virtual devices: e.g. a virtual PC (complete with BIOS boot screen) running on a Xerox workstation. Also virtual printers, other devices. Virtual pop-up keyboard. Describes drawing with a transparent color in certain graphical layers. [Kurtenbach97a] Kurtenbach, Gordon "Methods and System of Controlling Menus with Radial and Linear Portions", United States Patent 5,689,667, November 18, 1997 Pop-up marking menus with stylus or other pointing device: linear menu displayed below hotspot, marking/pie menus displayed around rest of hotspot. Permits more menus items to be selected by a single gesture, permits multi-step marking menus. Dwell/press-and-hold with "holding down the (pen or a) button on a mouse and waits for a short interval of time" [Kurtenbach97b] Kurtenbach, Gordon; Fitzmaurice, George; Baudel, Thomas; and Buxton, Bill "The Design and Evaluation of a GUI Paradigm based on Tablets, Two-hands, and Transparency", ACM CHI '97, March 22-27, Atlanta Georgia, 1997 Two-handed simultaneous use of stylus and puck with a Wacom tablet: compare with multi-touch. Digitizer puck also reported rotation on surface. Prototype system called "T3" (Tablets, Two-hands, Transparency), using ToolGlass (menu moved by one hand) and drawing/touching/etc. by other hand. [KwatinetzA96a] Kwatinetz, Andrew "Scrolling Contents of a Window", United States Patent 5,495,566, February 27, 1996 Scroll/jump a window faster depending on how far to scroll, also depending on input: relation to flick gestures, dynamic scrolling [LaPedusM96a] LaPedus, Mark "Apple and Motorola will launch new products with Chinese handwriting recognition", BYTE Magazine, September 1996: also available at http://byte.com/art/9609/sec17/art7.htm Lexicus division of Motorola with Chinese handwriting recognition product: WisdomPen Other Chinese pencomputing products: Taipei GoTop; Han Wang Science and Technology in Beijing; Pen Power Technology of Taiwan; Taiwan Palmax in partnership with Casio, InfoRay PD-96 organizer recognizes more than 13,000 Chinese characters Quotes Derek Ling of Motorola/Lexicus that Chinese handwriting recognizers often have higher recognition rates than Roman alphabet recognizers, because Chinese is always written in the same stroke order and style by users (sic: this contradicts study done at Pencept by Salzer) [LandayJA96a] Landay, James A. "Interactive Sketching for the Early Stages of User Interface Design", Ph.D. Thesis, Carnegie Mellon University, CMU-HCII-96-106 SILK User interface these: Annotated references to sketch recognition, character recognition, gesture Reference PerSketch (PenSketch?) system for editing sketches by user interface which allows multiple interpretations of input: four lines as four lines, and as a rectangle. Cited in MyersB96 for free-hand gestures and storyboards for sketching (and application generator / automatic generation) of user-interface designs. [LazzouniM97a] Lassouni, Mohamed; Yousaf, Mohamed; Qureshi, Rizwan A., and Nazir, Naveed A. "Pen and Paper Information Recording System using an Imaging Pen", United States Patent 5,661,506, August 26, 1997 Optical digitizer using printed pattern on paper, using squares and clusters of squares, compare with Anoto. See also US 5,626,412. [LeopoldJL97a] Leopold, J.L. and Ambler, A.L. "Keyboardless visual programming using voice, handwriting, and gesture", Proceedings of IEEE Symposium on Visual Languages, Sep 23-26, 1977, Pages 28-35 (abstract only) Voice, handwriting and gesture pen-computing user interface, using a visual programming environment: states useful for accessibility for manually or visually impaired. [LevineSR97a] Levine, Stephen R; Harui, Alex J; Hsiao, Chia-Chuan; Donaghue, Karen; and Schirpke, Michael W. "Document Annotation and Manipulation in a Data Processing System", United States Patent 5,625,833, April 29, 1997 Wang Freestyle patent: annotation (electronic ink and voice/audio) of documents in Freestyle. Mentions state-machine implementation of system. [LewisJR96a] Lewis, James R. "Reaping the benefits of modern usability evaluation: The Simon story", In Salvendy, G. and Ozok, A., (eds), Advances in Applied Ergonomics: Proceedings of the 1st International Conference on Applied Ergonomics - ICAE '96, pp. 752-757



Review of usability studies for Simon hand-held smartphone/PDA with touchscreen, circa 1993. "CGA" resolution of touchscreen, includes appointment calendar, address book, to-do list, sketch pad, sending and receiving electronic mail and fax (pre-Internet), system settings GUI, and security. [LewisJR97a] Lewis, James R.; Potosnak, Kathleen M.; and Magyar, Regis L. "Keys and Keyboards", Handbook of Human-Computer Interaction, Second completely revised edition, 1997, Chapter 54, pp. 1285..1315 Review of keyboard layouts, physical construction: alphabetic, QWERTY, Dvorak, auditory visual tactile feedback including touchscreen/stylus virtual keyboards. [LiebermanH96a] Lieberman, Henry and Maulsby, David "Instructable agents: Software that just keeps getting better", IBM Systems Journal, Vol 35, Nos. 3&4, 1996 task automation in which software agent infers completion of some steps, but overall planning and direction is by user. When agent encounters user doing new step/operation, may prompt user for information/parameters for how to complete the step. Cites to Mondrian, Eager, and Cima systems. [LinesA96a] Lines, Andrew; Bamberger, Matthew C.; and Yamasaki, Brian H. "Method and system for rotating a three-dimensional model about two orthogonal axes", United States Patent 5,682,649, October 28, 1997 User interface for rotating a 3D image (3D bar graph) around X and Y axes with fluid motion, by moving pointer on image of a reference sphere. Also can return rotation back to original orientation. [LinfordRA98a] Linford, Ray A. and Blanchard, Perin "Method and apparatus for selectively illustrating image modifications in an aesthetic imaging system", United States Patent 5,854,850, December 29, 1998 Generally about editing images of people's faces "aesthetically" for plastic surgery. Also mentions "menu bar is preferably translucent to allow the user to view the patient's image through the menu bar" [LinnettBJ97a] Linnett, Barry J.; Fries, Karen E.; and Powelson, Lawrence H. "Software platform having a real world interface and animated characters", United States Patent 5,682,649, October 28, 1997 Desktop user interface simulating multiple rooms in a house, e.g. Attic, Family room, drawers, workshop, Locked safe, etc. Animated characters act as expert guides for novice user: different animated characters for different apps or aspect. . Compare Wizard dialogs? Includes animation editor for animated sprites. Microsoft "Bob"? Compare with General Magic. [LongA98] Long, A; Landay, J.; and Rowe, L. "PDA and Gesture Uses in Practice: Insights for Designers of Penbased User interfaces", Report #CSD-97-976, CS Division, EECS Department, UC Berkeley, Berkeley, CA. January 1998 Study comparing pen-input gesture use on Apple Newton and early Palm Pilot: Gestures used more often on Newton than on Palm Pilot. Also studies used of Graffiti unistroke characters: noted that for Grafitti, users gave different performance ratings when asked about Graffiti than when asked about handwriting recognition although it was the same thing. Primary use of pen computers in study was note-taking during meetings. [LoprestiDP98a] Lopresti, Daniel P. and Tomkins, Andrew "Method for Locating a Penstroke Sequence in a Computer", United States Patent 5,809,498, September 15, 1998 Index retrieval of electronic ink images for a pen-based computer, by comparing a user input to each of the stored images, rather than comparing to pre-defined characters and doing handwriting recognition of text. Refers to user viewing of images as the final step in indexing and matching: compare to other methods for doing picture image search for similarity of images [LoprestiDP98b] Lopresti, Daniel P. "Ink as Multimedia Data", Proceedings of the Fourth International Conference on Information, Systems, Analysis and Synthesis, July 1998, Orlando, FL, pp. 122-128 Find electronic ink in a database by comparing number of points, point sequences, feature vectors, stroke types, characters, words. References to pretty-fying of electronic ink images. Compare to Slate "deferred translation" and DayTimer indexing methods [Lotus98a] Lotus Development Corporation "Lotus Organizer -- the Leading Time Manager: Exploring Organizer for OS/2 Warp 4", Lotus Development Corporation, 1991, 1998 PIM personal information manager: uses notebook/dayplanner/binder concept for user interface, calendar pages shown in simulated ring binder with sections tabs. Linking of pages (calendar, notes, to-do list, etc.) -- compare with Slate DayTimer PenScheduler for PenPoint/Pen Windows. [LucenteSAM96a] Lucente, Samuel A.m.; Balasubramanian, Peruvemba S.; Sapper, Richard F.; and Lee, Nathan J. "Rotatable pen-based computer with automatically reorienting display", United States Patent 5,566,098, October 15, 1996 Pen computer/PDA changes landscape/portrait orientation automatically, based on mercury switch to determine direction of gravity. User option to prevent automatic orientation rotation. [Luckie98a] Luckie, Douglas "Newton MessagePad Models: original (OMP) and 100 (MP100)", Biographical notes on early Apple Newton developers and personalities, http://www.msu.edu/~luckie/gallery/mp100.htm, fetched 2005 Newton 100: built-in applications included notetaking with mixed ink handwriting graphics text [LynchKM97a] Lynch, Kevin M.; Hertzfeld, Andrew J.; and Atkinson, William D. "Graphical user interface for



navigating between street hallway room metaphors", United States Patent 5,896,133, April 20, 1999 General Magic: Desktop user interface based on metaphor of heirarchy of objects in a room, rooms, buildings, town. e.g. library containing books for data storage, office directory on wall. etc. Compare with Microsoft "Bob"? Larry Yaeger, Brandyn Webb and Michael Kaplan - Fathers of Newton's "Print Recognizer": neural network not enough for handwriting recognition, need character segmentation, word segmentation, multiple geometric context models, language context models, search engine, in addition to character classifier [MacIntyreB96a] MacIntyre, Blair and Feiner, Steven "Future Multimedia User Interfaces", Multimedia Systems, Springer Verlag, 1996, Vol 4: pp. 250-269 multimedia and interactive user interfaces: one new requirement is the need for the use of multiple pens (multi-touch/multi-user) on a single large display. In ubiquitous computing, touchscreens will be more common than mice, because of integration with real-world objects. Existing windowing systems not appropriate (location-dependent GUI needed) [MacNeillD96a] MacNeill, David "What's on your Newton?", Pen Computing Magazine, Issue #11 July/August 1996 Short article on actual user use of Newton pen PDA computer: mentions Zaurus and Pilot text-based PDAs compared with Apple Newton MessagePad: Pocket Quicken, Graffiti [MalacariaCF98a] Malacaria, Charles F. "A Thing, Flexible Matrix-Based Pressure Sensor", Sensors Magazine, September 1998 Matrix pressure sensitive material with grid of conductors, covered with pressure-sensitive resistive ink on the same conductors. Rows and columns can be read independently and sequentially: 250,000 sensels per second to 50,000,000 sensels per second. FlexiForce sensors from Tekscan. [MalamudMA97a] Malamud, Mark A.; Elsbree, John E.; Butler, Laura J; and Barnes, David A. "Context Sensitive Menu System / Menu Behavior", United States Patent 5,665,133, September 2, 1997 Pop-up context-sensitive menus displayed near selected object, combined with cascading menus. Uses short time-out (delay / period of time) for cascading. Refers to selection and context button of mouse or any other selection signal, not left or right button. Compare to press-and-hold. [MamiyaM97a] Mamiya, Masaru; Hasegawa, Hiromasa; Sekino, Yoshio; Kawakami, Masahiko; Nishiki, Tomonori; Tsuji, Yshitaka; Okamoto, Akito; and Fujita, Toshihiro "A New Way to Overcome the Uneasy Operation of TouchSensitive Displays by Incorporating "Click" Mechanism CC Switch", Proc. HCI Int'l. 97, 7th Int'l. Conf. on HumanComputer Interaction, Vol 1, pp. 619-622 Touchscreen (digitizer and display) by itself uncomfortable to use because of lack of haptic feedback: add a mechanical click to touchscreen operation. Best operation is if touch sensor responds to pressure (force), so that user can trace with finger before applying sufficient force/pressure to activate switch. [MankeS96a] Manke, Stefan; Finke, Michael; and Waibel, Alex "A Fast Search Technique for Large Vocabulary On-Line Handwriting Recognition", International Workshop on Frontiers in Handwriting Recognition, University of Essex, England, Sep. 2..5, 1996 Use of a dictionary of words to aid in handwriting recognition: tree search of dictionary is faster than flat (linear) search. Word dictionary taken from "ARPA Wall Street Journal" project (for speech recognition evaluation), but no citation to the source. [MankoffJ98a] Mankoff, Jennifer and Abowd, Gregory D. "Cirrin: A word-level unistroke keyboard for pen input", Proc. of UIST 98, 1998, pp. 213-214 Large pie menu, user moves and dwells at certain characters in one compound stroke: can pop up at any location on screen [MarshDM96a] Marsh, Donal M. and Zias, Jeff "Method and apparatus for compositing display items which minimizes locked drawing areas", United States Patent 5,487,145, January 23, 1996 Transparent or semi-transparent objects which may be overlaid on the same display space. Transparent windows. [MarshallCC97a] Marshall, Catherine C. "Annotation: from paper books to the digital library", Proc. ACM Digital Libraries DL 97, Philadelphia PA 2007, pp. 131-140 Study of personal annotations, rather than group-editing annotations. Study based on college textbooks (that students have annotated personally: asterisks, arrows, underlining, highlighting, note, etc.) and bookstore books that customers have annotated. Notes that electronic annotation can be more flexible than regular book annotation, because the book itself still stays clean. Annotation (marking up) a fundamental part of reading or studying a text. [MatsushitaN97a] Matsushita, Nbuyuki and Rekimoto, Jun "HoloWall: Designing a Finger, Hand, Body and Object Sensitive Wall", Proc. UIST 97, Banff Canada, pp. 209-210 Capacitive-sensing wall/whiteboard with rear projection to create an interactive environment: compare with VideoPlace, Wellner. Can detect multiple hands/fingers, other objects. Two-handed manipulation/gesture for size, stretch, zoom. multi-touch. [McCloudSR98a] McCloud, Seth R. "Apparatus and method of communicating using three digits of a hand", United



States Patent 5,808,567, September 15, 1998 Multi-touch input for three fingers using 3x3 matrix of touch buttons for index finger and two larger side buttons for thumb and middle finger. Intended as very small keyboard/keypad on wrist and hand-held devices such as PDAs. [McCraryV98a] McCrary, Victor, and Smith, Alyssa "Electronic Book '98 Workshop, "Turning a New Page in Knowledge Management"", NIST Conference Proceedings, NIST-IR 6372, published 2001 Mostly about E-books and document publishing, Ralph Sklarew also present [McCrickardDS97a] McCrickard, D. Scott and Catrambone, Richard "Beyond the Scrollbar: An Evolution and valuation of Alternative List Navigation Techniques", Georgia Institute of Technology Technical Report GIT-GVU97-19, 1997 list UI objects similar to scrollbars, show additional information: mural bar and pile bar: "trough" (center part) of slidebar (not the slider) shows small graphical indicators of whether item has been viewed, or which are selected, type of item (to see similar items), etc. [McGrawG96a] McGraw, Gary and Hofstadter, Douglas R. "Emergent Letter Perception: Implementing the Role Hypothesis", Proceedings of the Eighteenth Annual Conference of the Cognitive Science Society, pages 621-626, July 1996 Cites early work by Blesser, Shillman, Cox et al on perceptual/cognitive basis for human letter character recognition: describes "Letter Spirit grid", similar to line-segment display, as a model for human recognition. Refers to letter vs. spirit, apparently spirit refers to an experimental model for writing/shape (not handwriting) style variability [MeadCA96a] Mead, Carver A.; Wolf, Ralph; and Allen, Timothy P. "Paintbrush Stylus for Capacitive Touch Sensor Pad", United States Patent 5,488,204, January 30, 1996 Capacitive/electrostatic stylus with wire brush on the end, determine "paintbrush" stroke by capacitive coupling to the brush, including area. Determines centroid positions from multiple grid/array points, each node read separately: multi-touch capable? [MenendezN96a] Mendendez, Norberto; Potrebic, Peter J.; and Sharpe, Benjamin W. "Method of Creating Packets for a pointer-based Computer System", United States Patent 5,555,369, September 10, 1996 UI Development by creating icons displayed in a rectilinear grid [MenezesAJ97a] Menezes, Alfred J.; van Oorschot, Paul C. ; and Vanstone, Scott A. "Handbook of Applied Cryptography", CRC Press, 1997 (hardcopy book) Thick reference/tutorial on PKI, cryptography practice. [MertzC96a] Mertz, Christophe P. and Lecoanet, Patrick "GRIGRI: Gesture Recognition on Interactive Graphical Radar Image", Proc. of Gesture Workshop '96, March 19th, 1996York, U.K. Published in Progress in Gestural Interaction, Springer Verlag (abstract only) GRIGRI: French vernacular for "scribble". Gestures on radar images for air traffic control. Single-stroke numerics (alphabetic) characters as gestures, compare with PenCept recognition macros [MerzEA96a] Merz, Eric A. "Multidimensional User Interface Input Device", United States Patent 5,565,657, October 15, 1996 pressure (force)-sensitive touch table using FSR force sensing resistors. Uses compressible semiconductive (resistive) material between two grid layers. Detects area of fingerprint touch, and uses center of touch area: multi-touch? [MicroTouch98a] MicroTouch Systems Inc. "TouchWare for OS/2 User's Guide", MicroTouch Systems, Inc. Methuen Mass First copyright is 1991: mentions double-click area setting required for using a touch-screen or touch-pad with a mouse driver, settings for the touch-screen affect the mouse behavior also, because it is the same driver. Desktop mode / Drawing mode / Button mode: controller uses quick touch (tap) to position the cursor, long touch to simulate mouse button behavior. See also UnMouse. Mentions Pen and Touch modes. Bimanual/multi-touch? [MicroTouch98b] MicroTouch Systems Inc. "MT3000 Capacitive Controller", MicroTouch Systems, Inc. Methuen Mass Controller (with firmware) for Microtouch touchscreens, UnMouse, touch-pads: Calibration is not four-point or orthogonal calibration, but two-point to get size of visible display [MicroTouch98c] MicroTouch Systems Inc. "TouchWare for Macintosh (USB controllers) User's Guide", MicroTouch Systems, Inc. Methuen Mass First copyright is 1991: mentions double-click area (in TouchWare control panel) setting required for using a touch-screen or touch-pad: evidently no OS support for double-click area [MicroTouch98d] MicroTouch Systems Inc. "MicroTouch ThruGlass Touchscreen Hardware Installation Guide", MicroTouch Systems, Inc. Methuen Mass ThruGlass capacitive/electrostatic touchscreen digitizers sensitive to air gap between touchscreen and first layer of non-conductive material in mounting, and to nearby metallic objects



[MicroTouch98e] MicroTouch Systems Inc. "MicroTouch Touch Controllers Reference Guide", MicroTouch Systems, Inc. Methuen Mass Command and protocols (serial and parallel interface) for Microtouch touchscreen digitizer controllers: Pen/Finger, Finger Only, Pen Only modes. Separate calibration (actually: defining scaling of active area with two points) for pen and for finger input (optical parallax?) [MicroTouch98f] MicroTouch Systems Inc. "MicroTouch Touch Kits Installation Guide", MicroTouch Systems, Inc. Methuen Mass Installation guide for resistive touchscreen sensors: bezel mounting: trimming bezel so no contact with front of CRT, spacing between touchscreen and CRT glass, Serial/SMT/PC-Bus/ADB Apple Desktop Bus interfaces (MacIntosh display) [Microsoft96a] Microsoft Corporation "Problems Encountered by Some Windows 95 Applications on Windows NT", technet.microsoft.com, 749964, October 1996 Incompatibilities with Windows 95 on Windows NT, including right-click menus / popup context menus on Windows 95. File contains additional references for right-click menus on Windows95. [Microsoft96b] Microsoft Corporation "Microsoft to Acquire aha! Software", Press Release, Microsoft Corporation, April 8, 2006 Microsoft acquires Aha! software, product is InkWriter electronic-ink note-taking and editing, appears similar to van Raamsdonk. Later became Windows Journal / NoteTaker application for Tablet PC. See also Dan Bricklin "Tablets Old and New". [Microsoft96c] Microsoft Corporation "Microsoft Announces Broad Availability of Handheld PCs With Windows CE", Microsoft Corporation press release, Nov. 18 1996 Microsoft Windows CE clamshell Pocket PC / PDA with stylus and touch screen (resistive?) / touch stylus input, Instant On [Microsoft97a] Microsoft Corporation "Microsoft Visual Studio 97, Enterprise Edition", Microsoft 0197 Part Number X03-08924 (hardcopy book)(Data CD) Visual Studio / C++ for Windows95 / WindowsNT. Does not include developer SDKs. Does not appear to include Pen SDK/PenWindows PenAPI.hlp [Microsoft98a] Microsoft "Accessibility Products for Microsoft Windows, Windows NT, and Windows 95", Microsoft.com, March 1998 Third-party accessibility extensions for Microsoft Windows: Screen enlarger/magnifier, screen review / screen reader, on-screen keyboard for pointing device: point-and-dwell / press-and-hold, autoclick / dwell selection. [Microsoft98b] Microsoft "Microsoft Introduces Palm PC, PC Companion Powered by Windows CE 2.0", Microsoft.com, January 8,1998 Press release for Microsoft Palm PC (Windows CE 2.0). New version of Velo from Philips. Hardware upgradeable by ROM replacement. No mention of stylus or touchscreen. [MillerP97a] Miller, Peter "Recursive Make Considered Harmful", Available from the author: millerp-atcanb.aaug.org.au Implementation problems of recursive make. Refers to recursive make documentation in 1991 and 1993 [MillerRJ96a] Miller, Robert J.; Bisset, Stephen; Allen, Timothy P.; and Steinbach, Guenter "Object Position and Proximity Sensor", United States Patent 5,495,077, February 27, 1996 Capacitive touchscreen digitizer: conductive object increases capacitive coupling between two wires of X/Y grid, simultaneous measurement. Conductive foam on top of digitizer allows it to detect any object, not just conductive. One application is musical instrument played by touchings of the hands. Also multi-touch: detect and report if one or more points are being touched, also sense pressure of the touch (in a "finger pointer" embodiment) [MillerRJ97a] Miller, Robert J. and Bisset, Stephen J. "Object Position Detector", United States Patent 5,648,642, July 15, 1997 Matrix capacitive touchscreen sensor, proximity sensing: analog circuitry to find peak. Can detect multiple fingers for gestures, infer pressure from height of peak. multi-touch capable?. [MillerRJ98a] Miller, Robert J. and Bisset, Stephen J. "Object Position Detector", United States Patent 5,841,078, November 24, 1998 Matrix capacitive touchscreen sensor, proximity sensing: analog circuitry to find peak. Can detect multiple fingers for gestures, infer pressure from height of peak. States that can detect multiple touches (multitouch), pressure (from ara), user need not actually touch (proximity). Second "position matrix" embodiment reports data at every node for multiple finger gestures. [MillerT98a] Miller, Timoth and Zeleznik, Robert "An insidious Haptic invasion: Adding force feedback to the X desktop", Proc. UIST 98, San Francisco, pp. 59-64 Add mechanical force-feedback (haptic) to X-Windows [Minatuchi98] Minatuchi, Yu; Okuyama, Satoshi; Fakue, Akiko; and Kamata, Hajime "Apparatus for Manipulating



an Object Displayed on a Display Device by Using a Touch Screen", United States Patent 5,844,547, December 1, 1998 Manipulating an image in response to touch input on a touchscreen. Mentions flick, pick, push, flip, distortion of image in response to pressure touch gesture, scrolling. [MintzerF97a] Mintzeer, Fred; Lotspiech, Jeffrey; and Morimoto, Morishige "Safeguarding Digital Library Contents and Users: Digital Watermarking ", D-Lib Magazine, Decmeber 1997 Overview of digital watermarks, primarily for images: visible, invisible, data-hiding. Invisible (imperceptible) watermarks) may be protected with a password/key. Using same invisible watermark in many images increase database for hacking. Reversible visible watermarks. [MobilePoint97a] Weimar, Clayton "Document archive regarding MobilePoint", AllAboutCocoa.com file Mp.zip: date approximate, fetched 2011 (ZIP file) Document archive concerning Wanderbook/MobilePoint, (see PenPoint) from Clay Weimer [MobilePoint98a] Weimar, Clayton "The Electronic Briefcase Interactive Sales System: architecture Overview", MobilePoint 03/07/1998 Fleet sales application using Windows 95 ostensibly without Pen Windows / Pen Services -- dedicated "ecase" application, user interface metaphor looks like that of PenPoint. Refers to pen computer hardware as a Tablet PC (not same as later Microsoft use of the name), Fujitsu Stylistic 1000 with electromagnetic digitizer, small display 640x480 (portrait mode). PAGE (Paper and Gesture) metaphor: Notebook Organization Model and Gesture Command model: visual appearance much like that of PenPoint with outlines, tabs on the right. Basic gestures are caret (insert), circle (edit), tap (turn to / go to), scratch-out (delete), checkmark (options/attributes), flick (scroll) and question mark (help). Also tap Pres, Drag to select copy move, press to select (hold), press drag (hold and drag) gestures. Swoop gesture -- single letter to move through index. Double tap gesture on electronic ink to select then circle (edit) to translate (recognize). As of 2008, MobilePoint.com platforms are Microsoft Tablet PC, as of 2011 mentions iPad. [MobilePoint98b] Wanderbookman (YouTube user) "iPad precursor: Penpoint's last chance: Face-to-Face SFA", http://www.youtube.com/watch?v=yINfscbeXRsa, fetched 2011 1998 video of MobilePoint electronic briefcase AKA PenPoint: appears to be porting of original (?) PenPoint code to be a pen-centric application running on Windows. [MoranTP97a] Moran, Thomas P.; Chiu, Patrick; van Melle, William "Pen-Based Interaction Techniques For Organizing Material on an Electronic Whiteboard", Proc. UIST 97 Symposium on User Interface Software and Techology Recognize long lines, especially vertical and horizontal, as border gestures, editing gestures to separate or align objects, text sketches on a whiteboard system. tap, balloon/lasso, pig-tail/pigtail, left-wedge, rightwedge, bump gestures for editing. [MoranTP98a] Moran, Thomas P. and Chiu, Patrick "Apparatus and Method for Executing Multiple-concatenated Command Gestures in a Gesture Based Input System", United States Patent 5,809.267, September 15, 1998 Multiple gestures in a single stroke, separate by break features: sharp points of high curvature. For combining two gestures, break is at end of first recognized gesture component. Based on Liveboard whiteboard system: most gestures are loops (lasso) to select objects, with combined with command gesture into one stroke. [MorganMW97a] Morgan, Michael W. "Method and apparatus for recognizing and performing handwritten calculations", United States Patent 5,655,136, August 5, 1997 Mathematical expression parsing and editing in a touchscreen calculator application. Uses gestures e.g. rub-out, pigtail/delete, arrow to move to hotpoint of arrow, loop to select. Page splitting edit, adaptive recognition/training. [MoskowitzS98a] Moskowitz, Scott "So this is Convergence? Technical, Economic, Legal, Cryptographic, and Philosophical Considerations for Secure Implementations of Digital Watermarking", White paper, BlueSpike.com, March 1998 Review of legal/technical questions on digital watermarking (as steganography) for digital assets. Audio digital watermarking by suppressing frequencies as specific points in the recording, the points determined by a randomly-generated watermarking key to prevent suppression of the watermark. [MulderJD98a] Mulder, Jurriaan D.; Groen, Frans C.A.; and van Wijk, Jarke J. "Pixel masks for screen-door transparency", Proc. 1998 IEEE Conf. on Visualiation VIS '98, pp. 351-358 Screen-door transparency (stippling): Polygons (fill areas?) drawn to be transparent/translucent using a mask: mask is small 4x4, 32x32 etc. for fill. Mask patterns chosen to have less visual artifacts (Moire patterns?) Sub-pixel screen door: image rendered at higher resolution than display, then averaged for larger visual pixels. [MunichME96a] Munich, Mario E. and Perona, Pietro "Visual input for pen-based computers", Proc. Int. Conf on Pattern Recognition, Vol 3, pp. 33-37, Vienna, Austra 1996 Optical tracking of pen/stylus on paper touchpad. Uses overhead camera to image the pen. Pen up/down



determined by segmenting into strokes based on motion (inflection points?) and compared with image of ink. Testing system compared with Wacom tablet digitizer. Cites to Wellner, finger tracking, 1995 technical report. [MunyanDE98a] Munyan, Daniel E. "Personal Electronic Book System", United States Patent 5,761,485, June 2, 1998 Two-sided (two-page) electronic book with dual touchscreens -- compare to Refalo, with in-app purchasing and electronic commerce, handwriting input, TV or radio video receiving [MyersB96a] Myers, Brad A. "User Interface Software Technology", ACM Computing Surveys, Vol 28 No 1, March 1996, pp. 189-191 Short review of user-interface systems, User interface management systems UIMS (application generators): HP/Apollo Open-Dialog, VAPS by Virtual Prototypes; Automatic generation tools for user interface such as ITS from Baecker et al IMB Research. Refers to direct-manipulation interfaces as synonymous to GUI graphical user interface. Cites SILK, free-hand gestures and storyboards for sketching initial user-interface designs. [MyersB98a] Myers, Brad A.; Stiel, Herb; and Gargiulo, Robert "Collaboration using multiple PDAs connected to a PC", Proc. CSCW 98 ACM Conference on Computer Supported Cooperative Work Remote control of a PC (mouse/keyboard) from a resistive touchscreen PDA (Palm Pilot): problems of emulating mouse buttons on touchscreen. Experimental system to have multiple users access a single collaborative/group application. [MyersB98b] Myers, Brad A. "A Brief History of Human-Computer Interaction Technology", Interactions, March+April 1998, pp. 44-54 History of windows, gesture, mouse, direct text manipulation back to approximately 1960. Traces history of WIMP/GUI interface from Windows 95 back to Macintosh, back to Xerox Park, back to SRI / Stanford Research Laboratory and M.I.T. Direct manipulation user-interface from Sketchpad. No citations before 1960. Gesture recognition in AMBIT/G at MIT Lincoln Labs in 1968. Cites light pen to 1954, touch screen. Digitizing tablet used to create movie "Hunger" in 1971. Mistakenly states that Rand tablet was the first pen-based input device. Teitelman 1964 first trainable gesture recognition, gesture input on GRAIL system in 1964. UIMS systems in 1996, interface builders (application generator) in 1979. File contains further revisions [MynattED97a] Mynatt, Elizabeth D. "Transforming Graphical Interfaces into Auditory Interfaces for Blind Users", Human-Computer Interaction, 1997, Vol 12. pp. 7-45, Lauwrence Erlbaum Associates Review of screen-reader problems for graphical user interfaces: spatial layout structure of user interface optimized for small display, not informative value, making navigation difficult to translate into spatial feedback for the blind. Numerous citations, none (?) to commercial products e.g. JAWS, outSpoken. [NFB97a] Mazrui, Jamal (National Federation of the Blind) "Screen Review Program: Comprehensive Reviews of Speech Access Programs for the Blind", National Federation of the Blind, 1997 Reviews/evaluations (circa 1995-1997) of screen readers and speech output accessibility products and systems: definition of screen reader. DOS and Graphical (Windows/MacIntosh) platforms. Arctic Technologies. Microtalk. Flipper. JAWS. IBM Screen Reader for DOS. Mentions shortcut keys (system level) and Windows95 Clipboard (Unix earlier) as accessibility features. [NarayanaswamyS96a] Narayanaswamy, Shankar "Pen and Speech Recognition in the User Interface for Mobile Multimedia Terminals", Ph.D. Thesis in EE/CS, University of California, Berkeley, 1996 "InfoPad" project at Berkeley. Citation to Go/PenPoint handwriting recognition with dictionary. Describes use of context areas to disambiguate gestures from handwriting. Cites problem of double-tapping (versus double-click) with stylus, because position is not the same. Cites problem of tip-switch sticking on digitizer at ends of strokes, added tails to character prototypes rather than correct them. Suggests combining digitizer/tablet for pointing/navigation with voice recognition for disabled accessibility. Mentions Pen PDAs as of 1996: Casio Z-7000, Sharp Zaurus, Amstrad PIC 700, Sony PIC-1000. PDA mentions Scriptel digitizer, term "raised strokes" for proximity detection (shake gesture of Tablet PC?) [NationalInstruments98a] National Instruments "LabVIEW User Manual", National Instruments, January 1998, Part Number 320999B-01 Visual programming language for laboratory instruments: earlier product versions exist, touchscreen and mouse input. Page 3-19shows lock switch (and unlock switch): Compare to Apple slide-to-unlock?. [NetManage97a] NetManage.com "NetManage ECCO Plus Complete PC Connectivity Solution - Getting Started Guide Version 4.01.1", NetManage.com, July 1997 PIM Personal Information Manager / Contact Manager. Interface to Timex Data Watch. [NewtonTechnologyJournal96a] Giles, Rick "Newton 2.0: Backward Compatible = New User Features; Gotcha! Common Problems Converting From 1.x to 2.0", Newton Technology Journal, February 1996, Vol. II No. 1, page 7, pp, 1, 22-24 Developer articles on Newton 2.0 Operating System for Newton MessagePad pen computer: rotation of



screen orientation (especially for spreadsheets, ReorientToScreen method override to for application to perform an action (e.g. re-layout the UI) on screen orientation change. On orientation/rotation of display, all child windows are re-oriented also, may require UI to be rebuilt. [NgE98a] Ng, Elizabeth; Bell, Tim; and Cockburn, Andy "Improvements to a Pen-Based Musical Input System", OzCHI '98: The Australian Conference on Computer-Human Interaction, 29 November to 4 December, 1998, pp. 230..252, IEEE Press Presto: Handwriting input for musical scores. (See earlier reference.) Additional gestures: right angles, appear similar to those of ritePen / EverNote. Pop-up menus via gestures, one motion for selection. Cites "Schoolchildren" pen gesture application, Air-traffic control, others. Includes use of velocity in recognizing "flick" gestures, distinct from normal writing speed. Examples of delete/rubout gestures from various systems: dot, squiggle, air traffic control, Penpoint, Graffiti, Windows fro Pe Computing. [NielsenJ98a] Nielsen, Jakob and Mankoski, Andrea "Method and apparatus for allowing a user to select from a set of mutually exclusive options", United States Patent 5,845,122, December 1, 1988 Describes select on mouse button down/activate on release the mouse button, so that user can move off the button if it was selected in error. Explicitly mentions touchscreen. While selected objects are shown in excited state -- compare with marquee in PenPoint? [Nokia97a] Nokia "Nokia 9000 Communicator Owner's Manual", Nokia, 1997 Smartphone/communicator: phone opens up sideways-clamshell style to show additional landscape-mode display and full alphanumeric keyboard/keypad [Norand97a] Norand Corporation "PenKey 6622 Pen Computer User's Guide", Norand Corporation, Cedar Rapids, Iowa Marcy 1997 Pen/slate touchscreen computer with MS-DOS, CIC/Handwriter, PenRight application software. On-screen virtual keyboard. Windows for Pen Computing (Windows 3.1) or Windows 95 Pen Services (pen extensions to Windows 95) [NovakJL98a] Novak, James L. and Petterson, Ben "Object Locating System", United States Patent 5,764,066, June 9, 1998 Electrostatic/Mutual capacitance proximity sensor for objects: single pair of electrodes to sense distance, multiple pairs of electrodes to determine shape and location of object. [OSROnline98] OSROnline "Purpose of the Pen Driver", www.osronline.com/ddkx/w98ddk/pen_0bc1.htm Several pages on Windows 95 DDK Pen-Windows drivers, reference to Programmer's Guide to Pen Services for Microsoft Windows95. "Required Pen Driver Messages", "Other Pen Driver Messages" (load/unload, configuration, etc.) [ObbinkK98a] Obbink, Kim; Ballinger, Bryan; and Sproule, William David "Method and system for displaying buttons that transition from an active state to an inactive state", United States Patent 5,767,835, June 16, 1998 If release mouse button after moving off selected objects, all pending highlights/indicators on object since mouse down are undone in order. Includes audio feedback. Explicitly mentions touch screen. [OhnsorgeH96a] Ohnsorge, Horst "Hand-held radiotelephone with video transmission and display", United States Patent 5,485,504, January 16, 1996 Smartphone/radiotelephone with video camera and display [OmniVision97a] OmniVision Technologies "OV5017 Single Chip Video Sensor", OmniVision Technologies, October 20, 1997 Single chip analog video camera with 8-bit A/D to convert at 50 FPS. No lens. [OmniVision98a] OmniVision Technologies "OmniVision Technologies Announces Camera-to-USB Dridge IC that delivers Real-time Video Image Transfer", Press Release, OmniVision Technologies, November 16, 1998 OV511 ACUB USB interface chip to go with OmniVision camera-on-a-chip video sensors [OmniVision98b] OmniVision Technologies "OmniVision Technologies Announces Single-Chip CMOS Image Sensors ("Camera on a Chip") with VGA Resolution", Press Release, OmniVision Technologies, October 19, 1998 OmniVision OV7600 OV7100 digital imaging device video cameras for use in picture phones [OppenheimAV97a] Oppenheim, Alan V. and Willsky, Alan S. "Signals and Systems 2ed", Prentice Hall, 1997 Engineering textbook on signals and systems, in both analog and digital domains [Oracle97a] Oracle Corp. "Oracle Projects User's Guide, Release 10SC (Version 4.1)", Oracle, 1997 Web-based project management software [Oracle98b] Oracle Corp. "Oracle Projects User's Guide, Release 11", Oracle, 1998 Web-based project management software [OuelletteD96a] Ouelette, Danile and D'Auteuil, Sylvain "Method and Apparatus for Displaying Simulated Keyboards on Touch-Sensitive Displays", United States Patent 5,581,243, December 3, 1996 Pop-up virtual keyboard on touchscreen display, covers bottom part of display, may be virtually transparent [PCSC97a] PC/SC Workgroup "PC/SC Workgroup Releases First Specification for Integration of Smart Cards with Personal Computers", Press Release CardTech/SecurTech West trade show, December 10, 1997



www.smartcardsys.com Device-independent APIs for smart card readers / cryptographic tokens. Compare with PKCS#1 standard from RSA. [PalermoV98a] Palermo, Vicent; Cobler, Patrick J.; and Butler, Neal R. "Short-range magnetic communication system", United States Patent 5,771,438 Smartphone or device with magnetic coupling to a base station or charger. Includes video conferencing on a cell phone. [Palm96a] Palm Pilot / 3Com Corporation "Graffiti reference card", 1996 Palm Pilot personal organizer product information Unistroke character set, except for cap lock gesture [Palm96b] Kairer, Ryan "Palm Pilot 1000 Retrospective", PalmInfocenter.com, fetched 2011 PalmOS 1.0, two weeks on two batteries, Graffiti alphabet on back of unit. [Palm96c] U.S. Robotics "Palm Pilot PDA with GoType! external keyboard / docking station", Palm Pilot, U.S. Robotics, 1996 (physical device) Operational Palm Pilot PDA, with external keyboard. Multiple Palm Pilots in collection. [Palm96d] U.S. Robotics "Palm Pilot PDA with GoType! external keyboard / docking station", pocketcomputing.com Technial information on Palm Pilot touchscreen PDA, created for creating market for hand-held devices with handwriting recognition (graffiti). [Palm97a] Palm Pilot / 3Com Corporation "PalmPilot Handbook", 3com Corporation, 1997, Part Number 423-021001B Resistive-film PDA: gesture full-screen pen stroke activates on-screen virtual keyboard. Graffiti handwriting recognition (unistroke) with separate areas for alphabetic and for numeric handwriting. Shortcut / pigtail gesture, other shortcuts pre-defined are two-letter combinations. Command gestures of single diagonal stroke (command stroke) following by mnemonic character e.g. "C" for cut. Other gesture strokes for cut, copy, next field, etc. [Palm97b] Palm Pilot / 3Com Corporation "US Robotics announces two new models of Palm Pilot", 3com Corporation, 1997 Resistive-film PDA: TCP/IP connection to desktop computer, portable snap-on (phone) modem for remote connection to desktop computer, Internet Sidekick from Starfish Software [Palm97c] Palm Pilot / 3Com Corporation "Palm Pilot Professional PDA", Palm Pilot, 3com Corporation, 1997 (physical device) Palm Pilot Professional, functional physical unit, original packaging. [Palm98a] Palm Pilot / 3Com Corporation "Getting Started with the Palm VII Organizer", 3com Corporation, 1998 iMessenger phone organizer on Palm VII [Palm98b] Palm Pilot / 3Com Corporation "Handbook for the Palm VII Organizer", 3com Corporation, 1998 iMessenger phone organizer on Palm VII: Web browsing, wireless email. [Palm98c] Palm Pilot / 3Com Corporation "Palm Exec gives more details on new Palm VII Device", 3com Corporation, 1998(fetched 2005) Palm VII as first PDA with wireless internet connectivity without external or add/on wireless modem [Palm98d] Palm Pilot / 3Com Corporation "Handbook for the Palm III Organizer", 3com Corporation, 1998 Palm III PDA: Grafitti handwriting recognition, separate modes for alphabetic and numeric input. [Palm98e] 3Com Corporation "3Com Palm III PDA Organizer", Palm III, 3com Corporation, 1998 (physical device) Palm III PDA, functional. With leatherette case and docking/charger/holder. [Palm98f] Handspring Corporation "Handspring Palm Visitor PDA Organizer", Handspring Palm, Handspring Corporation, 1998 (physical device) Palm Visor PDA, functional. [PanditMS97a] Pandit, Milind S. and Kalbag, Sameer "The Selection Recognition Agent: Instant Access to Relevant Information and Operations", Proc. IUI 97, Orlando Florida, pp 47-52 Recognition (by patterns) of phone numbers, dates, proper names in text, which are then turned into clickable links. Cites similar features in Netscape Navigator (email text recognized), others. Highlighting of text by patterns, then clickable? [Papyrus96] Papyrus Associates: Bill Kania "Recognition by Papyrus: single-stroke character recognition", Papyrus Associates, [email protected], www.papassoc.com Unistroke character recognition [ParaGraph98a] ParaGraph "ParaGraph Announces Advanced Handwriting Recognition Software for Palm PC And H/PC Platforms", Press Release, April 6 1998, vbce.com CalliGrapher handwriting recognition software version 5.1 for Windows CE. Punctuation entered by pop-up menus, color electronic ink. [ParadisoJA96a] Paradiso, Joseph A. and Gershenfeld, Neil "Musical applications of electric field sensing", Computer Music Journal Vol 21 No2, pp. 69-89, 1997



Review of capacitive sensing in musical instruments: Theremin, Boie radio drum, touchpad sensors on guitar, hypercello instrumentation of bow position, fingerboard, bow pressure/force, wrist angle. Refers to transmit mode (active signal), and shunt mode (self capacitance or change in mutual capacitance) [Paragon98a] Paragon Software "Paragon Software Corporate History", Paragon Software (web site) Handwriting recognition for PDAs, 1994 through 2013: WindowsCE, Nokia, etc.. Confused with Paragraph (similar?) [ParkinsonK97a] Parkinson, Kelly "Dwell Selection Mouse Button Emulation", Senior Thesis, Ryerson College, Canada, 1997 (abstract only) Mouse button emulation: press-and-hold, using dwell selection and gestures to simulate mouse button actions. Left button and right button emulated by zag-left and zag-right (left-right, right-left) gestures. [PenComputing96a] Pen Computing Magazine "Handwriting Recognition", 1996(?), http://whiterabbit.com/amug/...PenReport/Handwriting.html Review of Unistroke recognition, PDA products and technology (HWX), mentions UNIPEN / JOT electronicink format, stylator Mentions graphics handwriting versus gestures, context example of circle as a drawing, as digit zero, character O, or Edit gesture (in PenPoint). [PenComputing96b] Pen Computing Magazine "Pen Services for Windows 95 Product Review", Pen Computing Magazine, August 1996, Issue No. 11 PenWindows/Windows for Pen Computing essentially off the market, available only in 2.0 version for OEMs . Mentions Circle-S/S-circle gesture (for "space"), changes in other gestures [PenComputing96c] Pen Computing Magazine "Graffiti 2.0 Product Review", Pen Computing Magazine, January/February 1996, Issue No. 8 On Newton: Double-tap gesture in any text entry field to bring up Graffiti input. Mentions split of two areas for alphabetic and for numeric/digit input. Mentions that OmniGo allows input anywhere on the screen. [PenComputing96d] Pen Computing Magazine "Freestyle 1.0 Product Review", Pen Computing Magazine, January/February 1996, Issue No. 8 Not the Wang Freestyle product: CalliGrapher handwriting recognition from ParaGraph: later used by Microsoft. Comment on difficulties of using editing gestures. [PenComputing96e] Pen Computing Magazine "CIC Handwriter for Windows 95 Product Review", Pen Computing Magazine, January/February 1996, Issue No. 8 "Scribe" character recognizer for Windows for Pen Computing 2.0: YPad electronic note-taking application. Signature Sentinel uses biometric dynamic signature verification as password. [PenComputing96f] Pen Computing Magazine "Digitizer Whiteboards", Pen Computing Magazine, May/June 1996, Issue No. 10 Whiteboard digitizing tablets: Microfield Graphics SoftBoard with map-projection and medical applications; Numonics IPM Interactive Presentation Manager with cordless stylus/pen; Smart Technology SMARTboard (using resistive technology) with rear-projection [PenComputing98a] Pen Computing Magazine "Why did Apple kill the Newton?", Pen Computing Magazine, June 1998 Review of Newton touchscreen/PDA political history. Sculley/Jobs rivalry. Intel cancelling StrongARM processor vs. Intel. Smaller size/form-factor not released (Palm successful). Emphasis on handwriting recognition versus portable digital communications (real use). Handwriting handicapped by small dictionary (10000 word), not single character. Notes optional keyboard. [PenWindows96a] PenWindows: Microsoft Corporation "Programmer's Guide to Pen Services for Microsoft Windows 95 - PENAPI.HLP", Microsoft system documentation HLP file with documentation on Pen Windows 2.0 -- longer version of material in PenWindows95b. Also provided as part of Windows programming documentation in third-party development tools such as Delphi7. Note: Note: Pdf page 62, landscape and portrait mode: Tablet Orientation. Transparent window overlay for annotation ink on a document. [PerkinsR97a] Perkins, Rod; Ludolph, Frank; and Keller, Dan Smith "Inventing the Lisa User Interface", Interaction, Vol 1. No 1, 1997, pp. 40-53 Desktop manager UI for Lisa Computer (predecessor of MacIntosh), mentions explicit inspiration from SmallTalk. (ACM reprint) [PerlinK98a] Perlin, Ken "Quikwriting: Continuous Stylus-based Text Entry", Proceedings of ACM UIST '98 conference Compared with Grafitti: single-stroke / unistroke character input, recognizing motions around a circular menu without lifting the pen between strokes (rather than recognizing normal characters): cites T-cube by Venolia [PerlinK98b] Perlin, Ken "NYU Medial Research Lab Projects: Quikwriting", mrl.nuy.edu, 1998 single-stroke / unistroke character input, recognizing motions around a circular menu without lifting the



pen between strokes (rather than recognizing normal characters). Windows CD touchpad computer. [PetzoldC96a] Petzold, Charles "Programming Windows 95", Microsoft Press, 1996 (hardcopy book) Windows 95 UI and programming guide, 1996. Nothing on context menus in Windows 95? Does have example of mouse emulation/simulation via keyboard. [PharesR98a] Phares, Robert "Resistive Touchscreen having Multiple Selectable Regions for Pressure Discrimination", United States Patent 5,815,141, September 29, 1998 Resistive touchscreen digitizer with separate regions that can be turned off, e.g. for palm rejection [PhilippH98a] Philipp, Harald "Time domain capacitive field detector", United States Patent 5,730,165, March 24, 1998 Capactive sensing of user proximity to a water faucet or other object, using very short (100 nanosecond) sensins pulse that are faster than the conduction time through water. Can detect user's hand in the water. [Philips97a] Philips Electronics "VELO1 User's Guide -- Windows CE PDA -- 1997 Edition", Philips Electronics / Microsoft, 1997 Windows CE PDA with touchscreen, adjustable properties. User gestures are tap and drag: reference to text editing using the stylus as one would a mouse. No reference to handwriting recognition. [Philips98a] Philips Electronics "Philips Velo 1 with Microsoft Windows CE 2.0 User's Guide -- Windows CE PDA -1998 Edition", Philips Electronics / Microsoft, 1998 Windows CE PDA with touchscreen, adjustable properties. Windows CE applications installed from desktop PC with Windows CE Services 2.0. Touchscreen recommends stylus for finer pointing/ writing, built-in telephone modem. Refers to PDA as H/PC Handheld PC. [PortraitDisplays97a] Portrait Displays "Pivot 1700 Rotatable CRT display", Portrait Displays, Inc. www.portrait.com, 1997 Rotatable display (originally for MacIntosh) between landscape/horizontal, software in OS to re-lay-out and redisplay windows/application automatically when display is rotated. [Poyner96] Poyner, Rick "Wintab Interface Specification 1.1: 16- and 32-bit API Reference", LCS/Telegraphics, Cambridge Massachusetts, revised May 9, 1996 Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages. [PriceMN98a] Price, Morgan N.; Schilit, Bill N.; and Golovchinsky, Gene "XLibris: The Active Reading Machine", Proc. CHI '98 Conf. on Human Factors in Computing Systems, 1998, pp. 22-23 XLibris: pen-computer tablet/display that allows users to make electronic ink annotations on scanned images of pages: making annotations improves understanding. Annotated/circled/highlighted sections can be shown as clipping/snippets in a notebook view. [PrinceKR96] Prince, Kevin R. "Finger Mounted Computer Input Device", United States Patent 5,581,484, December 3, 1996 Finger-mounted digitizer acceleration "glove" using accelerometers and pressure-sensitive sensors in the fingertips, to detect simulated keyboard and mouse input. Describes simulating a computer text keyboard, and a musical instrument keyboard. Compare to Kaplow. [Psion97a] Psion, Inc. "Psion Series 5 Handheld Computer User Guide", Psion, Inc., 1997 (partial copy) Psion clamshell Palm-top (a bit large) with half-sized touchscreen and full-size abbreviated keyboard. Sketch drawing program, images can be inserted into Word documents. Voice note audio recording. [Qualcomm98a] Qualcomm Incorporated "Qualcomm Unveils pdQ CDMA Digital Smartphone", Qualcomm Incorporated, September 21, 1998 Announcement of Palm OS pdQ Smartphone / pen-based organizer with wireless internet access via CDMA [Qualcomm98b] Qualcomm Incorporated "Qualcomm Announces Software Developer's Program for the pdQ Smartphone", Qualcomm Incorporated, November 18, 1998 Announcement of Palm OS pdQ Smartphone / pen-based organizer with wireless internet access via CDMA [Qualcomm98c] Qualcomm Incorporated "Qualcomm to trial pdQ phone", Qualcomm Incorporated, September 28, 1998 Announcement of Palm OS pdQ Smartphone / flip-down keypad and touchscreen, pen-based organizer with wireless internet access via CDMA [QuekFHK96a] Quek, Francis K.H. "Unencumbered Gestural Interaction", IEEE Multimedia, Vol. 4, No. 3, Winter 1996, pp. 36-47. Also VISLab Report: VISLab-96-0 (abstract only) Further research on FingerMouse optical detecting of finger pointing, and video recognition of 3-D hand gestures. [RAND98a] RAND Corporation "50 Years of Looking Forward", RAND Review, Fall 1998, vol 22 no 1. Available at http://www.rand.org/publications/randreview/issues/rr.fall.98/50.html



In 1961, researchers create the RAND Tablet, the first two-dimensional writing surface that allows humans to communicate instantly with computer through characters printed on a tablet. [REX98a] www.starfish.com "What is REX?", www.mcmullon.com/rexreviews/whatisrex.htm, www.starfish.com, 1998 Handheld PDA product by Phillipe Kahn, Franklin Electronic Publishers, Starfish Software, and Citizen's Watch. No touchscreen interface, on-screen alphabetic keyboard accessed by physical scrolling buttons. Electronic file contains additional materials. [RSA97a] RSA Laboratories "PKCS #11: Cryptographic Token Interface Standard, Version 2.0", RSA Laboratories Technical Note, Version 2.0, April 17, 1995 PKCS11 API standard Version 2.0 for smartcards and other cryptographic tokens: single-user, mostly portable devices. Object-based in "C". [RabaeyJM96a] Rabaey, Jan M. "Digital Integrated Circuits, 1st Edition", Prentice Hall, 1996 Undergraduate textbook on digital circuit design. Description of half-adders and adders from logic/boolean elements. [RauterbergM98a] Rauterberg, Matthias; Fjeld, Morten; Krueger, Helmut; Bichsel, Martin; Leonhardt, Uwe; and Meier, Markus "BUILD-IT: A Planning Tool for Construction and Design", Video presentation: Proc. CHI '98, April 18-23, 1998, pp. 177-178 Tabletop touchscreen test application for collaborative building design: hand gestures to select, deselect, finger touch to move and position objects. Plant equipment represented by physical "bricks" on tabletop -see "Bricks". Video camera (similar to Wellner) for determining hand/finger/object positions. Included 3D views. Cites to Wellner and Tognazzini "StarFire". [Redhat98a] RedHat-list "Re: four button mice?", [email protected]/msg01799.html, 28 June 1998 Posting concerning mapping of UI events for mouse buttons to other UI events, such as keyboard input: four-button multibutton mouse. [RedmayneDV97a] Redmayne, Derek V. "Capacitive Touch Sensor", United States Patent 5,650,597, July 22 1997 Capacitive grid (sort-of) sensor using single layer of horizontal resistive conductors. Vertical position determined by which bar or conductor has strongest signal (to passive finger/stylus) with interpolation, horizontal by relative strength of signals measured at two ends/sides of the bar or conductor. [RegisterDS97a] Register, David S. "Hand held computer with dual display screen orientation capability controlled by toggle switches having first and second non-momentary positions", United States Patent 5,661,632, August 26, 1997 Hand-held PDA or touchscreen computer with button to rotate display between landscape and portrait orientation/rotation. [RekimotoJ96a] Rekimoto, Jun "Tilting Operations for Small Screen Interfaces", Proc. UIST '96, Seattle, Washington, pp. 167-168 Small-screen devices (PDAs) use pens or touchscreens, which require two hands: one to hold the device, one for the pen/stylus/finger on the screen. Very small devices (pagers) too small for a pen. Use tilting menus instead: pie/cylindrical menus work well with visual feedback for small tilt angles. Cites to Chameleon system. [RekimotoJ97a] Rekimoto, Jun "Pick-and-Drop: A Direct Manipulation Technique for Multiple Computer Environments", Proc. UIST '97, Banff, Alberta, Canada, pp. 31-39 User interface for moving data between multiple devices: use same pointing device on multiple computers: Pick/Select (perhaps with loop gesture) first on one computer, then drop on Window on another computer. Can be combined with drag-and-drop (on same computer) using press-and-hold to initiate cross-computer "pick". Or by using out-of-proximity for cross-computer. Mentions Wacom tablet can handle three styli (multi-touch) by using the barrel button to distinguish the third stylus/pen. Combine with virtual desk a la Wellner as additional input surface to pick-and-drop from paper to electronic desktop. ClearBoard: whiteboard with video interaction. Cites to Wellner and PARCTAB. [RekimotoJ98a] Rekimoto, Jun "A multiple-device approach for supporting Whiteboard-based Interactions", Proc. CHI '98, April 18-23, 1998, Los Angeles, pp. 344-351 Whiteboard system where user input is from a portable touchscreen computer and shown on an electronic whiteboard. User can pick-and-drop objects and data from portable computer display to whiteboard display. [ReynoldsAR96a] Reynolds, Aaron R.; Adler, Dennis R.; Lipe, Ralph A.; Pedrizetti, Ray D.; Parsons, Jeffrey T.; and Arun, Rasipuram V. "Common name space for long and short filenames", United States Patent 5,579,517, November 26, 1996 Long and short (8.3) filenames on extended version of Windows FAT file system, for backwardscompatibility with MS-DOS programs. [RichardsonFB96a] Richardson, Frederic B. III "System for Software Registration", United States Patent 5,490,216, February 6, 1996



Copy-protection system generating a license code from information of the system on which the software will execute: changes in system information (configuration) mean license not valid. See also Uniloc and Bluecava for additional materials [Rolodex97a] Franklin Electronic Publishers "Rolodex Electronics 64K Organizer RF-2064 User's Guide", Franklin Electronic Publishers, 1997 Small PDA device, with address book / phone numbers [RoofL98a] Roof, Larry "Professional Visual Basic Windows CE Programming", Wrox Press, 1998 (hardcopy book) Windows CD / Visual Basic programming reference [RosenbergLB97a] Rosenberg, Louis B. and Brave, Scott B. "Method and Apparatus for providing Force Feedback for a Graphical User Interface", International Published Patent Application WO 97/21160, June 12, 1997 Add force feedback to a mouse or other input device to indicate sliders, buttons, scrollbars, windows, and other components of a GUI [RosenthalAS97a] Rosenthal, Amy S., Hu, Jianying, and Brown, Michale K. "Size and Orientation Normalization of On-Line Handwriting Using Hough Transform", IEEE International Conference on Acoustics, Speech, and Signal Processing, April 21-24, 1997, Munich, GermanyMulti-Dimensional Signal Processing Poster Session MDSP6P.13 Vol. 4, pp. 3077 Correct for rotation, base-line drift in on-line recognition: claims 25% improvement in recognition accuracy, but does not describe recognizer or how the improvement was measured Boundary-line extraction, backtrack strokes: appears to be segmenting cursive/connected text at local Y extrema [RuedisueliLW98] Ruedisueli, Laurence, W. and Wilfong, Gordon Thomas "System and Method for Processing and Managing Electronic Copies of Handwritten Notes", United States Patent 5,838,819, November 17, 1998 Filing and archiving system for electronic-ink handwritten notes. Identifiers for each page/sheet of writing, so that pages can be recalled. Identifiers left open to be numeric, graphical icons of the page, sequential, etc. [RyzhovVA98a] Ryzhov, Vladimir Alexandrovich and Trofimov, Sergei Viktorovich "Method of entry of symbolic information into computer system", United States Patent 5,786,809, July 28, 1998 Virtual keyboard on touchscreen: may be semi-transparent. [SakaiM97a] Sakai, Makoto; Dewa, Koichi; Tsukada, Hiroyuki; Uehara, Keiichi; Mamata, Tohru; Nishino, Yashuhiro; and Oda, Hiroyuki "Portable computer having dedicated register group and peripheral controller bus between system bus and peripheral controller", United States Patent 5,613,135, March 18, 1997 Keyboard interface bus/controller, including hotkeys: contrast adjustment, brightness adjustment keys on keyboard as hotkeys. [SakamotoH98a] Sakamoto, Hirotaka "Document Image Processing System", United States Patent 5,774,233, June 30, 1998 Display can be rotated landscape/portrait orientation. Menu display window is rotated and positioned to compensate. Portions of application window that end up off-screen in changed orientation can be scrolled. [SalcudeanSE98a] Salcudean, Septimiu Edmund, and Kelly, Allen J. "Controller", United States Patent 5,790,108, August 4, 1998 Force-feedback haptic mouse. Also see 1993 "MagicMouse: Tactile and Kinesthetic Feedback in the Human-Computer Interface using and Electromagneticlaly Actuated Input Device with Two or Three Degrees-of-Freedom" and "Electromagnetic Force-Feedback Mouse for the SUN SPARCstation: Prototype Design Document" (1992) [SamarV98a] Samar, Vipin "System and method to transparently integrate private key operations from a smart card with host-based encryption services", United States Patent 5,778,072, July 7, 1998 Some users have smartcards with secure private key storage, others log in an the private key is kept on a decryption/encription server (key store manager). Encrypt/decrypt operations with the private key (e.g. for a document) are only performed on the smartcard, or by the keyserver: private key is never transmitted. [SandersonSM98a] Sanderson, Stephen M. and Davidson, Thomas J. "Method and system for creating computerprogram-based applications with developer specified look and feel", United States Patent 5,818,445, October 6, 1998 GUI user interface, specification language for separately defining user interface graphical appearance, interactions of modules/operators, and functional modules/engines. Cites to X-Windows. Compare with color/UI management in Java widget toolkits, AWT. [SanemitsuY98a] Sanemitsu, Yoshikado "IC Card Having Camera, Microphone and Modem for use in Information Processors", United States Patent 5,708,853, January 13, 1998 IC card (PCMCIA?) with video and microphone for wireless teleconferencing: shows use in portable laptop. Camera and microphone can be integral to housing, or external. Cites to 1995 proposed standards for PCMCIA video cards. [SarkarS98a] Sarkar, Sudeep "Case of Vermont Microsystems Inc. vs. Autodesk", Available at http://courses.cs.vt.edu/~cs3604



Case study in course on ethics: Prepared during NSF Workshop on Teaching Ethics and Computing. Otto Berkes, later director of Origami / UMPC project at Microsoft, trade-secret lawsuit between VMI and Autodesk on misappropriation of proprietary information [SchifferS98a] Schiffer, Stefan "Visuelle Programmierung: Grundlagen und Einsatzmöglichkeiten", Addison-Wesley, Munich, 1998 Publication of Ph.D. dissertation on visual programming: Shows virtual slide switch from LabView from 1994 [SchilitBN98a] Schilit, Bill N.; Golovchinsky, Gene; and Price, Morgan N. "Beyond Paper: Supporting Active Reading with Free Form Digital Ink Annotations", Proc. CHI 98, Los Angeles, CA, pp. 249-256 XLibris: Electronic ink mark-up of documents shown in commercial pen-tablet display as a "Reader's Notebook", permitting search on annotation text -- compare with later developments on Microsoft Tablet PC, earlier work by Slate on DayTimer. Distinguishes between annotations related to book text, and notes which are not related. Gestures more convenient than barrel button. Uses Muto pen/tablet display tethered to Windows 95 PC -- no mention of W4PC/PenWindows? Press-and-hold: Dwell gesture instead of barrel button, barrel button is mouse 2nd button? [SchmidtA98a] Schmidt, Albrecht; Beigl, Michael; and Gellersen, Hans-Werner "There is more to context than location", Proc. Int'l. Workshop on Interactive Application of Mobile Computing IMC98, November 1998, Rostock, Germany Smartphone, PDA other devices adapt UI user interface to context of environment or user action: sensor for orientation change orientation of display (orientation-aware), ambient light for contrast, etc. Shows screen orientation of Apple Newton. Electronic file contains slides from talk, paper of same title. Includes pictures of landscape/portrait rotation/orientation on Newton. [SchneidermanR97a] Schneiderman, Ron "Future Talk: The Changing Wireless Game", IEEE Press, 1997 (hardcopy book) Review of Wireless communication standards 1997, slightly pre-smartphone (although post-IBM-Simon, etc.) Includes glossary/definitions of telecommunications terms [SchoedlA97] Schoedl, Arno "The NewtonScript Programming Language", www.cc.gatech.edu, source notes for CS 6751, Fall 1997 Description of NewtonScript programming language, includes discussion of slots and dual inheritance through both prototypes and parent class heirarchy. Section 2.1 has code fragment with "slot functions" on objects, visually similar to case-statement-from-hell [SchomakerL96a] Schomaker, L. "The NICI stroke-based recognizer of on-line handwriting", Available at http://hwr.nici.kun.nl/recog/nici-stroke-based-recognizer.html Velocity-based stroke: uses changes in stroke velocity to recognize the segmentation of characters: does not work with children's handwriting or handwriting with tremor. Makes use of UNIPEN software, refers to "Papyrus" project from 1990. Refers to other paper on handwriting recognition system based on human motor system. Points out some of the factors making it hard to judge recognition accuracy: count accuracy by letters or words, selection of test data, diversity in background of writer subjects, confusion of training and test sets, etc. [Schumer98] Schumer, Alfred L. "Digitizer Interface", United States Patent 5,768,492, June 16, 1998 Subdivide digitizer tablet into virtual tablet areas in different parts of the tablet surface: patent is to software, but disclosure is to hardware implementation only. See files on Schumer and LCS/Telegraphics. [SchwartzRL97a] Schwartz, Randal L.; Olson Erik; and Christiansen, Tom "Learning Perl on Win32 Systems, First Edition", O'Reilly, 1997 (hardcopy book) Perl interpretive language on Windows platforms. [Scriptel97a] Scriptel Holding "Scriptel Holding annual report (10-K)", Scriptel Holding, available at sec.edgaronline.com, Marcy 27 1997 Scriptel electrostatic/capacitive digitizer can sense touch and proximity and is durable, unlike direct contact touch screen technology; is less subject to noise than electromagnetic digitizer in a layer, and more robust and higher resolution than membrane pressure-sensitive digitizer touchscreen. [Sekendur98] Sekendur, Oral F. "Absolute Optical Position Determination", United States Patent 5,852,434 Digitizer tablet "smart paper" that puts a printed code on paper, position is read by optical sensor in the stylus tip. Compare with technology by Anoto et al. [SelkerT97a] Selker, Ted "A Conversation with Ted Selker", Interactions Magazine, September/October 1997, pp. 34-47 Interview (Kate Ehrlich) with Ted Selker, inventor of trackpoint small joystick pointing interface embedded in laptop keyboard near location of index finger, as alternative to touchpad. Advantage over mouse is user need not move eyes away from screen to move hand to mouse. Describes two-handed trackpoint input with drawing in right hand, tools manipulated via trackpoint for left hand index finger -- compare with Buxton/Kurtenbach?



[SellenA97a] Sellen, Abigail and Harper, Richard "Paper as an Analytic Resource for the Design of New Technologies", Proc. CHI '97, Atlanta, Georgia, March 1997 Comparison of paper for collaborative work with electronic systems for collaborative work. Continuing user of paper is probably a work-around by users for shortcomings of electronic systems: e.g. users can tell whether others are paying attention to a paper document, items can be recalled by approximate position (flipping through pages). Touchscreen or Stylus-based systems with electronic ink markings would probably help. Cites to Wellner for combining paper and electronic documents. [Sharp96a] Sharp/Zaurus "Sharp Model OZ-5600 Electronic Organizer Operation Manual", Sharp, 1996 Keyboard PDA with financial accounting software, automatic recording of transactions similar to Money or Quiken. Printer and PC cables for Organizer. [Sharp97a] Sharp/Zaurus "Sharp ExpertPad PI-7000 Newton Pictures", Sharp, 1997 (source: ebay.com) Pictures of Sharp PI-7000 ExpertPad -- Newton MessagePad. [ShiehJMH98a] Shieh, Johnny Meng-Han "Virtual Pointing Device for Touchscreens", United States Patent 5,808,605, September 15, 1998 Multi-touch touchscreen, which fingers (gestures) are touched to screen have different commands. Fingers identified from contact area of palm. Compare with Fingerworks? One use is different fingers for different "virtual" pointing devices -- fine and coarse motion, commands, etc. Show virtual mouse on touchscreen: if user removes hand, virtual mouse is "hibernated" and shown as translucent. [SinclairM98a] Sinclair, Michael; Presti, Peter; Robertson, Scott "The Haptic Lens", SIGGRAPH, August 2-7, 1998, California Summary announcement of US Patent 5,459,329: 3D (X/Y plus force) force-sensitive tablet digitizer, using clear elastomer flexible rubber sheet over a semi-opaque grey fluid filler. File includes slides from presentation. [SmallD97a] Small, David and Ishii, Hiroshi "Design of spatially aware graspable displays", Proc. CHI '97 Extended Abstracts, March 1997 Early study on using tilt, gestures, spatial orientation in a portable (tablet, not PDA) to browser a large display/information space. Example: browsing an electronic newspaper. Cites to Chujigiga, story-telling scrolls. [SmartQuill97] Williams, Lyndsay (nee Robinson) "SmartQuill", Personal webpage for Lyndsay Robinson, formerly British Telecom, currently at Microsoft Research SmartQuill: accelerometer based digitizer pen with handwriting recognition, VGA small display, additional buttons. Functions somewhat as a PDA. Project renamed to WxPen at Microsoft? Paper file contains additional references. See also patent for Lindsay Williams (SmartQuill) [SmithAR96a] Smith, Alvy Ray; Catmull, Ed.; Porter, Thomas; and Duff, Tom "The Alpha Channel: A Simple Concept with Profound Implications", Award citation: Academy of Motion Picture Arts and Sciences, Scientific and Engineering Award, March 2, 1996. Available at www.alvyray.com/awards/AwardsAcademy96.htm History of Alpha Channel / alpha blending, invention in 1977 along with alpha and RGBA teminology, to support partial transparency. [SmithDC98a] Smith, David Canfield and Irby, Charles H. "Xerox Star Live Demonstration", CHI 98, p. 17, 18-23 April 1998 (Demonstrations) Xerox Star (1970s) had very few discrete named command or menus items: presentation on consistency and context. gestures? File contains poster paper, not the video [SmithJ98a] Smith, Joshua; White, Tom; Dodge, Christopher; Paradiso, Joseph; Gershenfeld, Neil; and Allport, David "Electric Field Sensing For Graphical Interfaces", IEEE CGA, May/June, 1998, pp. 54-60 Tutorial on electric fields (capacitive sensing) for touchscreen and computer input. Three modes of operation/sensing: Transmit mode (signal sent through body), shunt mode (mutual capacitance affected by shunt to ground), loading mode (self-capacitance affected by shunt to ground). Low frequencies 30 to 100 kHz. Prefer shunt mode to load mode (mutual to self capacitance). Matrix used for imaging. 3D information from poximity measurement: two-handed user interfaces with gestures. Very high accuracy (millimeter) with wide electrode spacing (15 cm or more). Example of under-table electronics: Compare to Rekimoto multi-touch Smartskin. [SmithJR96a] Smith, J.R. "Field mice: Extracting hand geometry from electric field measurements", IBM Systems Journal, Vol 35 Nos 3 and 4, 1996, pp. 587-608 Review of capacitive sensing (electric fields) for touchscreen and computer input. Cites to Theremin for two-handed input, hand gesture interfaces, Gesture Wall. Describes three-dimensional 3D mouse using capacitive sensing of a hand and multiple fingers. [SmithiesCPK98] Smithies, ChristopherPaul Kenneth; and Newman, Jeremy Mark "Document And Signature Verification System and Method", United States Patent 5,818,955, October 6, 1998 signature verification system, primarily about the use of a data envelope that includes identity information, and a record of the "gravity prompt" message displayed to user to inform user about the seriousness of



signing. Also claims use of signed checksum on document. [SpindlerM97a] Spindler, Martin; Stellmach, Sophie; and Dachselt, Raimund "PaperLens: Advanced Magic Lens Interaction Above the Desktop", Proc. ITS '09, ACM Int'l. Conf. on Interactive Tabletops and Surfaces, Nov. 2325, 2009, Banff, Canada, pp. 69-76 Projected tabletop touchscreen surface, overhead projector: project onto a sheet of paper that can be lifted, Z position sensed optically from infra-red markers at the corners; projected image onto paper changes (e.g. different layers of object underneath). Cites to Bier MagicLens. Refers to paper display as projected touchtable surface: cites to DigitalDesk. File also contains New Scientist "Paper screens could provide depth to computer display" [SrinivasanSR96a] Srinivasan, Seshan R. "Automated, electronic network based, project management server system, for managing multiple work-groups", United States Patent 5,548,506 August 20, 1996 Project and resource management system [StantonN96a] Stanton, Neville ed. "Human Factors in Nuclear Safety", Taylor&Francis, 1996 Review of engineering design in nuclear power systems for safety, especially in the design of the control room. Extensive discussion of touchscreens, uses and applications. [StarnerT97a] Starer, Thad; Weaver, Joshua; and Pentland, Alex "A Wearable Computer-Based American Sign Language Recognizer", Proc. ISWC '97 Video camera mounted on baseball-cap visor looking down: recognition of ASL American sign language hand gestures -- very limited vocabulary of 40 simple signs. Suggests possibility of recognizing finger spelling gestures. [StefikM96a] Stefik, Mark "Letting Loose the Light: Igniting Commerce in Electronic Publication", Published in "Internet Dreams: Archetypes, Myths, and Metaphors", Mark Stefik ed., MIT Press, September, 1996 Essay on historical role of copyright in intellectual property, need for electronic/internet support for copyright and royalty payments: more important than electronic book hardware: trusted systems, welldefined machine-readable security language [StefikM97a] Stefik, Mark and Lavendel, Giuliana "Libraries and Digital Property Rights", Proc. First Euro. Conf. on Digital Libraries ECDL '97, Pisa Italy, September 1..3, 1997 Trusted Systems and digital rights for electronic libraries, cites Xerox DPRL digital property rights language. Digital rights: Transport (including loan) from one trusted/secure system to another, Render rights, Derivative rights, Library Loans, fair use. [StefikM97c] Stefik, Mark J. "Trusted Systems: Devices that enforce machine-readable rights to use the work of a musician or author may create secure ways to publish over the Internet", Scientific American, March 1997, pp. 78-81 Essay on DRM lending-library rights enforcement on the Internet. Features Xerox digital rights language. Part of special section on "The Internet: Bringing Order from Chaos / Fulfilling the Promise" (e.g. support of business interests). [StefikMJ97b] Stefik, Mark J. and Casey, Michalene Ma. "System for Controlling the Distribution and Use of Digital Works", United States Patent 5,629,980, May 13, 1997 DRM usage rights management, usage rights control on permissions for what rendering/use is granted to a document or object. On-demand publishing, uses PKI for authentication and security [SteinDK98a] Stein, Deborah Kent "The Optacon: Past, Present, Future", The Braille Monitor, Vol 41 No 5, May, 1998 Personal report on using Optacon, tactile visual scanner. Particularly notes ability to read small daily items: old ID cards, receipts, etc. Request for tactile braille display (1998). [SteinM97a] Stein, Michael; Kaufman, Toby Ray; Richarz, Alesander Yves; Tarlow, Allen Kenneth; and Nesbitt, Craig Bryce "User Interface for Home Automation System", International Patent Application Publication WO 97/11448, 27 March, 1997 Touch-sensitive virtual buttons on touchscreen with haptic tactile feedback, other sensory feedback: visual feedback (LEDs), and audio feedback (buzzer) [StifelmanLJ96a] Stifelman, Lisa J. "Augmenting Real-World Objects: A Paper-Based Audio Notebook", Proc. CHI '96, April 1996, Vancouver, pp. 199-200 Electronic recording of writing/notes on paper (portable tablet) with synchronized recording of audio lecture or discussion. Information can be located physically on the particular page (user remembers spatially), jump to matching audio. Compare with Anoto pen, Wang Freestyle. [StifelmanLJ97a] Stifelman, Lisa Joy "The Audio Notebook: Paper and pen interaction with structured speech", Ph. D. Thesis, M.I.T. Program in Media Arts and Sciences, August 8, 1997 Electronic recording of writing/notes on paper (portable tablet) with synchronized recording of audio lecture or discussion. Information can be located physically on the particular page (user remembers spatially), jump to matching audio. Compare with Anoto pen, Wang Freestyle. [StoneMC97a] Stone, Maureen C. and DeRose, Anthony "User interface technique for producing a second image in



the spatial context of a first image using a model-based operation", United States Patent 5,652,851, July 29, 1997 Patent for MagicLens: move a Viewing Operation Region (VOR / Magic Lens) over a graphical object to see a different view of the object. [StoneMC98a] Stone, Maureen C.; Bier, Eric A.; and DeRose, Anthony "User-directed method for operating on an object-based model data structure through a second contextual image", United States Patent 5,729,704, March 17, 1998 Patent for MagicLens: transparent/translucent tool window that operates when over an information object. Defines apparently transparency/substantially transparent as image substantially similar to first image except for relatively small perceived modification. [StoneMC98b] Stone, Maureen C.; Fishkin, Kenneth P.; and Bier, Eric A. "Method and apparatus for operating on the model data structure of an image to produce human perceptible output using a viewing operation region having explicit multiple regions", United States Patent 5,818,455, October 6, 1998 Patent for MagicLens: move a Viewing Operation Region (VOR / Magic Lens) over a graphical object to see a different view of the object. VOR (a tool window) has distinct input regions (e.g. buttons, etc.) and output regions (where it displays something -- a semi-transparent portion). [StreitzNA98a] Streitz, Norbert A.; Geissler, Joerg; and Holmer, Torsten "Roomware for Cooperative Buildings: Integrated Design of Architectural Spaces and Information Spaces", Proc. First Int'l. Workshop on Cooperative Buildings CoBuild'98, Darmstadt, Germany, February 25-26, 1998, pp. 4-21 Roomware: instrumented DynaWall and InteracTable touchscreen environment using i-LAND. Rearprojection electronic whiteboards : whiteboard accessible from laptop with digitizer screen. Objects/windows rotatable on InteracTable via gestures using pen/stylus or fingers. See also InfoWall, InfoTable. [StreitzNA98b] Streitz, Norbert A. "Roomware and Cooperative Buildings: Extending the Scope of Interaction and Cooperation beyond Desktops", Presentation at the International Computer Science Institute, Berkeley Univ., Berkeley, CA, April 30, 1998 Roomware: instrumented DynaWall; InteracTable tabletop/touchscreen environment using i-LAND; CommChairs. Includes same video presentation given in 1999 of tabletop system with multiple users and windows? [StrickonJ98a] Strickon, Joshua and Paradiso, Joseph "Tracking Hands Above Large Interactive Surfaces with a Low-Cost Scanning Laser Rangefinder", Proc. ACM CHI'98 Conf., Paris 21-23 1998, Los Angeles Multipoint touchscreen interface using video shadow projection: shadowing of one hand by another resolved by adding inertia / extrapolating motion of hidden hand. [StroustrupB97a] Stroustrup, Bjarne "The C++ Programming Language, Third Edition", Addison-Wesley, 1997, ISBN 0201889544 Standard for C++ object-based programming language. Includes Standard Library, Standard Template Library (STL), Abstraction mechanisms, Design concepts (roles of classes e.g. interface classes, application frameworks) [StuckaJE97a] Stucka, Joan E. and Wiss, Richard "Method and system for dynamically sharing user interface displays among a plurality of application program(s)", United States Patent 5,596,702, January 21, 1997 User interface is separate from application program: user can use any windowing system dynamically, change window manager on-the-fly, or for a single application. Application code thus need not be recompiled (or re-written) for different window managers (X Windows and Microsoft Windows) Example is based on X Windows. [SugiuraA98a] Sugiura, Atsushi and Koseki, Yoshiyuki "A User Interface Using Fingerprint Recognition - Holding Commands and Data Objects on Fingers", Proc. UIST '98, San Francisco, pp. 71-79 Prototype system for touchscreen (actually touch-button) user interface using fingerprint scanner, rather than 3 different buttons for operations, can recognize which finger touches a single button for different operations, or different operations depending on which user touches. Cites to different COTS fingerprint scanners/sensors. [SunMicrosystems96a] Gosling, James and McGilton, Henry "The Java Language Environment: A White Paper", Sun Microsystems / JavaSoft, May 1996 Design rationale and description for the Java language. Mentions HotJava web browser written in Java, AWT Abstract Window Toolkit [SunMicrosystems96b] Sun Microsystems "Sun Labs: Highlights of the First Five Years: Sun Lab's Technological Contributions to Sun", Sun Microsystems, www.sun.com, February 1996 Green Project (Star7 touchscreen PDA-like device) spun off to FirstPerson, Inc. in 1993, later renamed to JavaSoft. [SunMicrosystems98a] Gosling, James and McGilton, Henry "PersonalJava Technology White Paper", Sun Microsystems, 1998 Description of Java development environments/SDK. Describes Star7 portable home controller / PDA.



Touchscreen input (no mouse), high-speed graphics, spread-spectrum wireless networking. Implementation difficulties on Star7 with C++ led to rationale for Java (Oak). Inclusion of Java in NetScape web browser. [SunakawaS97a] Sunakawa, Shinich; Shimada, Kazutoshi; Tatsumi, Eisaku; Suzuki, Noriyui; and Hagasaki, Katsuhiko "Information Processing Apparatus and Control Method Thereof having User Character Recognition", United States Patent 5,644,653, July 1, 1997 Pen computer with handwriting recognition, detect whether user is left or right handed (based on handwriting position), and use different recognition dictionary. Pen computer that user can rotate for any of 2x2 landscape or portrait orientation/rotation, display screen adapts, reflows-text. [Symantec98a] Symantec "ACT! 4.0 for Windows User's Guide act40", Symantec, 1988 PIM personal information manager and contact manager for Windows 95/NT. [SymbolTechnologies96] Symbol Technologiess "PPT 4600 Software Development Kit and Getting Started for DOS and Windows", Symbol Technologies Inc., One Symbol Plaza, Holtsville, New York Cross-development tools for low-end embedded pen computers: Support for DOS and Windows. Appears to be a third-party package (Windows for Pen computing / Pen-Windows, and MSDOS). Hardware is a ruggedized single-hand-held (large, but with strap) unit with built in scanner for barcodes (twodimensional bar codes PDF417). File include product technical information PPT 4600 Series Portable Pen Terminals with Integrated Scanners. [Synaptics97a] Synaptics Inc. "Synaptics TouchPad C++ API, Revision 1.5", Synaptics.com, 1997 API for Synaptics capacitive touchpad/digitizer, sensing both position and contact area (force/pressure). Includes virtual scroll bar in separate area of TouchPad surface, scrolling by edge pressure at edge of surface. Includes press gesture (press-and-hold?): drag, draglock, tap gestures (started in separate area of tablet?) Includes proximity sensing. Supports connecting multiple tablets. [TaguchiT98a] Taguchi, Toshio; Yamamoto, Shin-ichiro; and Kanamura, Hiroki "Writing Device for Storing Handwriting", United States Patent 5,774,602, June 30, 1998 Optical digitizer in pen images and compresses writing marks / ink in compressed form while user is writing, small images of paper near writing point of pen. Compare with Anoto, Sekendur. [TakayaK98a] Takay, Kei and Kakegawa-shi, Shizuoka "Contrast adjustment method and information processing apparatus having touch-sensitive device", European Patent Application 98105730.0 EP 0 869 424 A2, July 10, 1998 Contrast adjustment gesture on a touch-screen: press-and-hold followed by move ment in one direction or another to adjust screen contrast: also adjust brightness / volume? [TamTH98a] Tam, Tommy H. and Gough, Michael L. "Method and Apparatus for Blending Images", United States Patent 5,574,186, May 19, 1998 Image blending (e.g. two windows, two images) using a lookup table indexed by the pixel value of one image and the pixel value of the other image: the look-up table has appropriate pixel values for combing the colors (or shades of gray) of the two pixels. Intended for virtual on-screen keyboard on hand-held touchscreen/pen computer, so that virtual keyboard does not obscure/block view. [TaylorAG97a] Taylor, Ashley George "WIMP Interfaces: CS6751 Topic Report: Winter '97", www.cc.gatech.edu Review of WIMP user interfaces: cites to Smalltalk visual language in early 1970s. and Sketchpad from 1963. Mentions metaphors of buttons and controls resembling those on a video editing deck, physical slide switches: virtual devices: slide-to-unlock. [TaylorS96] Taylor, Scott "Pen for OS/2 Update", Pen Computing Magazine, Issue #8 January/February 1996 Review of WARP, Pen extensions to OS/2. Mentions desktop Pen PC using external tablet. Cursor change to indicate in user interface when over a handwriting free-form input field. Pen Computing development environment: Pen Assist from IBM for OS/2: C subroutines from IBM and VisualAge C++ for OS/2, VXREXX from Watcom, AM for OS/2 from Intelligent Environments [TaylorSA98a] Taylor, Stuart A. "CCD and CMOS Imaging Array Technologies: Technology Review", Technical Report EPC-1998-106, Xerox Research Centre Europe Tutorial on CCD and CMOS digital camera imaging technology circa 1998. [Tecskor96a] TecSkor Software Inc. "TecSkor CPR Corporate Process Renewal Software User's Manual", TecSkor Software Inc., 1996 Project Management system [Telxon97a] Telxon Corporation "Telxon PTC-1184DX-11 Pen Computer User's Guide", Telxon Corporation, 1997; Symbol Technologies, Inc. 2000 Pen-DOS/PenRight! pen computer with built-in wireless radio. Separate handwriting recognition on top of Windows 95 / Windows 3.1: separate mention of Windows for Pen Computing. Electromagnetic (Wacom?) or resistive touch digitizer. Triple-tap to bring up software keyboard on screen (from DOS only). Includes "Learning Pen Basics" tutorial for pen-based computing. Tablet of PenDOS gestures: copy, paste, tap, undo, insert space, insert text, "X" for delete block, delete character gestures.



[TheAppleCollection97a] The Apple Collection "Prototype and Design: MoCCA (Mobile Computing and Communication Appliance)", www.theapplecollection.com, fetched 2013 "Conceptual Prototype" if MoCCA (Mobile Computing and Communication Appliance): PDA with voice and email messages, videocamera, visual and voice input only (no touchscreen or keyboard) [ThomasBH97a] Thomas, Bruce Hunter "Animating Direct Manipulation in Human Computer Interfaces", Ph.D. Thesis, Dept. of Comp. Sci., Flinders University of South Australia, Adelaide, August 22, 1997 Animating UI interactions: e.g. rotating palette menus, buttons that resize and rotate when pushed, animated drag and resize. [ThompsonRohrlich97a] Thompson-Rohrlich, John "Recognition Keypad", United States Patent 5,677,710, October 14, 1997 Recognition pad (writing pad?) in user interface, has a button and gesture/character recognition modes (command mode==gesture). Cites to Agulnick/Penpoint. Compare also with Doster / Wang "Guide"? [TisueS98a] Tisue, Seth "Indie Version History: New Stuff, 26 June 1998", Internet posting June 26, 1998, available at www.cs.northwestern.edu/~riesbeck/indie/versions.html Minimum movement threshold of five pixels to initiate a drag to prevent problem of inadvertently moving/dragging an object (widget) when user's intent is to select or edit the object. [Tou96a] Tou, Frederich N. and Capps, Stephen "Method and apparatus for inserting text on a pen-based computer system", United States Patent 5,528,743, assigned to Apple Computer, Inc., June 18, 1996 Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? Automatic recognition of handwritten text in the context of paragraphs: text is either placed into the appropriate place in an existing paragraph, or starts a new paragraph Text-editing, determining whether input is text, and location/targetting in relation to an existing paragraph. (Compare with Slate?) Shows use of writing baseline [TruchetP97a] Truchet, Philippe "Display and Input Control Device", United States Patent 5,606,345, February 25, 1997 Overlay direct touch input windows/buttons on touchscreen separate from application program. on-screen virtual keyboard? [TrumanTE98a] Truman, Thomas E.; Pering, Trevor; Doering, Roger; and Brodersen, Robert W. "The InfoPad Multimedia Terminal: A Portable Device for Wireless Information Access", IEEE Trans. on Computers, Vol 47, No 10, October 1998, pp. 1073..1087 InfoPad: prototype portable wireless device for video, PDA, text. Presumes pen/touch user interface, and speech recognition via a server, instead of keyboard. Focus is on low power hardware circuitry and components. [TsubaiR98a] Tsubai, Ryozo "Data Entry System", United States Patent 5,793,312, August 11, 1998 13-key single-hand keyboard/keypad layout. Uses two shift keys for thumb and other finger. Assigned to FrogPad: Frogpad.com video states that this is one of several multi-touch patents (?). [TurpinWM97a] Turpin, William M.; Brown, Kevin L.; and Bogrett, Steven W. "Development system with methods for maintaining data integrity of Information stored as a data record in a database with a copy of the information displayed on a screen device", United States Patent 5,608,898, March 4, 1997 User interface: in a list control, show previously-entered or database values to pick from, and do not write to database if already in data base. Compare with pick list / edit list history in Slate PenApps? [UllmerB97a] Ullber, Brygg and Ishii, Hiroshi "The metaDESK: Models and Prototypes for Tangible User Interfaces", Proc. UIST 97, Banff, Alberta, Canada, pp. 223-232 Combine physical objects ("phicons" -- physical icdons) and graphical objects ("icons") in one user interface. Two-hand (multi-touch) zoom/rotate using physical bar with two sensor pucks. Most objects tracked optically. Prototype uses VisionMaker rear-projection tabletop display from Input Technologies. Phicons can be rotated. [UllmerB97b] Ullber, Brygg and Ishii, Hiroshi "Tangible Media Group: The metaDesk Video", Tangible Media Group, MIT, 1997 demonstration of metaDesk, parallel to Ullmer paper same year. [UlrichRR96a] Ulrich, Robert R. "Method and Apparatus for a Recognition Editor and Routine Interface for a Computer System", United States Patent 5,566,248, October 15, 1996 Pen input area with text recognition (on Newton?) [UmezawaK96a] Umezawa, Koichi; Kuriyama, Hiroyuki; Nishiyama, Takanori; Ishida, Kiyoshi; and Ishinabe, Iwao "Video Telephone Equipment", United States Patent 5,491,507, February 13, 1996 Handy-type cellphone/smartphone with video camera and display for real-time video conferencing. [UnitedStates96a] National Communications System, Technology and Standards Division "FED-STD-1037C Federal Standard 1037C Telecommunications: Glossary of Telecommunications Terms (dictionary)", General Services Administration / Information Technology Service, August 7, 1996 Dictionary/glossary of computer and telecommunications terms for Federal departments and agencies.



Terms used in telecommunications and directly related fields by international and U.S. Government telecommunications specialists. [ValloppillilV98a] Valloppillil, Vinod "Halloween I: Open Source Software: A (New?) Development Methodology", virtualschool.edu/mon/ElectronicProperty/Halloween1.html, first posted by Eric S. Raymond 1998 Posting by Eric S. Raymond, author of the "The Cathedral and the Bazaar", of originally internal memorandum in Microsoft concerning quality of Open Source software (OSS): privately concerned about high quality, publicly dismissive. Electronic file also contains Microsoft memo Halloween II: Linux OS Competitive Analysis: The Next Java VM? concerning Linux distribution as multi-platform competitor to Windows. [VanCruyningenI98a] van Cruyningen, Izak "Popup Menus with Directional Gestures", United States Patent 5,805,167, September 8, 1998 Mentions marking menus / pop up context menus. When a menu pops up in response to a gesture, user can move stylus to selected item from the menu without lifting the pen beforehand. Popup menus include Japanese katakana, icons, numeric keypad/keyboard. Dwell/press-and-hold? [VanDerGoesFML96a] van der Goes, Frank M.L. and Meijer, Gerard C.M. "A Novel Low-Cost Capacitive-Sensor Interface", IEEE Trans. on Instrumentation and Measurement, vol 45 no 2, April 1996, pp. 536..540 Highly noise-resistant capacitive measurement sensor with continuous autocalibration to calibration capacitor: applicable to capacitive touchscreens. Compare to syncronous demodulation in Blesser, Rekimoto? [VasudevanV97a] Vasudevan, Venu; Gossain, Dhiraj Kamar; Rigg, Dana Mark; Gibson, Don Michael; and Womack, Trou Kyea "Method for managing a hierarchical design transaction", United States Patent 5,678,040 October 14, 1997 Project and resource management system [VerplaetseC96a] Verplaetse, Chrisopher "Inertial proprioceptive devices: Self-motion-sensing toys and tools", IBM Systems Journal, Vol 35 No 3/4,, 1996, pp. 639-650 Accelerometer/motion-sensing input toys to make them interactive: mentions acceleromiter pen for handwriting recongition with optical registration for calibrating absolute position (compare Hugh Crane, Anoto). Review of accelerometer components circa 199, piezoresistive, capacitive, differential capacitive. Compare with WII? [ViirreE98a] Viirre, Erik; Pryor, Homer; Nagata, Satoru; and Furness, Thomas A. III "The Virtual Retinal Display: A New Technology for Virtual Reality and Augmented Vision in Medicine", Proc. of Medicine Meets Virtual Reality, San Diego, California, USA, (pp. 252-257) Virtual retinal display: laser projection of display image directly on eye retina. Single-eye display intended for subjects with visual impairments. [ViredazMA98a] Viredaz, Marc A. "The Itsy Pocket Computer Version 1.5 User's Manual", Compaq Western Research Lab WRL Technical Note TN-54, July 1998 Itsy Pocket Computer prototype very small hand-held computer running Linux. Resistive touch-screen user interface, simple gestures. [VirtualInk97] Virtual Ink Corporation "Virtual Ink Corporation: The Company & The Product", entrepeneurship.mit.edu/15975/VirtualInk_CompanyProduct.pdf Business Plan and profile for the epen / e*pen product, an electromagnetic digitizer to retrofit to a large existing whiteboard, bundled with remote whiteboarding application software [VirziRA96a] Virzi, Robert A.; Sokolov, Jeffrey L.; and Karis, Demetrios "Usability Problem Identification Using Both Low- and High-Fidelity Prototypes", Proc. CHI '96, Vancouver BC Canada, April 10-13, 1996, pp. 236-243 Paper prototypes: a low-fidelity prototype finds most/much the same usability problems as a high-fidelity prototype or actual implementation. Compare with Tognazzini comments on (movie) scripting? [W3C98a] W3C www.w3.org "Extensible Markup Language (XML) 1.0", REC-xml-19980210, www.w3.org, W3C Recommendation 10-February-1998 XML Extensible Markup Language draft standard version 1.0: based on SGML, interoperability with SGML and HTML [WacholderN97a] Wacholder, Nina; Ravin, Yael; Choi, Misook "Disambiguation of Proper Names in Text", Proc. ANLC '97 fifth conference on Applied natural language processing, pp. 202-208 Heuristics for identifying proper names in text, based on context, case use, etc. [Wacom97a] WACOM Company Ltd. "WACOM PenPartner User's Manual for Windows", Wacom Company, May 21, 1997 PenPartner digitizing tablet with UltraPen for Window 3.1, Windows95 and Windows NT 4.0. Force-sensing (pressure-sensitive) stylus with side button, and eraser tip on back end of stylus (similar to Wang Freestyle). Tablet/Mouse driver. Shows using stylus to erase text (with eraser cursor/icon) in WordPad. Both absolute positioning and relative (mouse) positioning motion, user selectable. Side button programmable to simulate keyboard modifier keys for size, drag, etc.



[WantR98a] Want, Roy and Collins, Jennifer P. "Apparatus and configuration method for a small hand-held computing device", United States Patent 5,825,675, October 20, 1998 Hand-held palm-sized computer with touchscreen: display contents can be flipped or rotated 180 degrees for orientation for left or right-handed users. [WardJR96] Ward, Jean Renard, Barret, David M., Martin, Patricia A., and Mokoski, Christopher D. "User Interface having Simulated Devices", United States Patent 5,491,495, assigned to Wang Laboratories, Lowell, Massachusetts, February 13, 1996 Simulated virtual soft keyboard, handwriting recognition, simulated mouse input, in a GUI implemented in a separate processor from the main processor. [WeimerC98a] Weimer, Clay "The Story of MobilePoint", www.allaboutcocoa.com/golucent/storyofmo.doc, fetched July 6, 2011 History of MobilePoint through approximately 1998. MobilePoint spin-off/spin-out from GO/PenPoint, field sales force application, user interface and features reminiscent of PenPoint's notebook metaphor. MobilePoint acquired other GO/PenPoint third-party application provides in pen computing. Mentions history of PenPoint/Windows for Pen Computing/Microsoft PenWindows. State Farm used 5000 pen computers (PenPoint) with field representatives. IBM took over maintenance of PenPoint code for ThinkPad 730T pen computer. Claims first use of Smart PHone as product name, PenPoint documentation went to unknown third party. Source code for Numero and MagicScript acquired from PenMagic/Pivotal, Gnosis from Deskin. [WellnerPD96a] Wellner, Pierre D. "Interactive Copying System", United States Patent 5,511,148, April 23, 1996 User interface and touch sensor using optical projection on to a desktop, user can manipulate projected documents, make annotations, etc. Two cameras make use of parallax and edge detection by comparing images to determine outline edges of finger. Tap gesture to select. Refers to multi-user systems for concurrent editing from two different desks. Multi-touch gestures: touch two fingers to start defining a selection and move fingers to drag boundaries -- pointing with a plurality of fingers at corners of a shape. [WhitakerS97a] Whitaker, Steve; Swanson, Jerry; Kucan, Jakov; and Sidner, Candy "TeleNotes: Managing Lightweight Interactions in the Desktop", ACM Trans. on Computer-Human Interaction, Vol 4 No 2, June 1997, pp. 137-168 Early shared whiteboard / message board note-taking application with electronic sticky notes / post-it notes. Keyboard/mouse based, cited elsewhere for electronic ink note taking. [WhiteT98a] White, Tom "Introducing Liquid Haptics in High Bandwidth Human Computer Interfaces", Master's Thesis, M.I.T, June 1998 LiquidPad: Combined touch sensor (optical) and haptic output sensor. Uses a thin bag of dark liquid over grid/matrix of optical sensors, where user presses (with any object) the liquid is thinner, optical sensors detect light. Using a magneto-rheological fluid and additional matrix of electromagnetics underneath, can also provide haptic feedback -- in presence of strong magnetic field, fluid becomes rigid/solid. Compare Fingerpaint demo with later demonstrations of multi-point/multi-touch input by Jeff Han, Rekimoto. Cites to FishFace electric field touch sensors. Response time of MR fluid is in milliseconds. MR elastomer even faster. [WhitmerC97a] Whitmer, Charles "Method and system for dynamically generating computer instructions for performing a logical operation on bitmaps", United States Patent 5,604,850, February 18, 1997 Sort of a reversely-implemented BitBlt: given a set of bitmaps and their composition (image composing), determine what computer instructions are needed for a BitBlt to perform that composition, so that composition can be done in one BitBlt instead of multiple BitBlt operations. Applied to example of transparent arrows or other GUI objects. [Wikipedia97a] Wikipedia.org "PalmPilot", wikipedia.org, fetched 2011 History of Palm Pilot touchscreen PDA with Graffiti/unistroke handwriting character recognition, Memo Pad, To Do List, two scroll buttons. [WilcoxLD97a] Wilcox, Lynn D.; Schilit, Bill N.; and Sawhney, Nitin "Nick" "Dynomite: A Dynamically Organized Ink and Audio Notebook", Proc CHI 97, March 1997, pp. 186-193 Ink and Audio annotation system similar to Freestyle, but with a number of gestures: drag-rectangle to select, tap to select, horizontal line for new page, etc. [WilleyM97a] Willey, Mark "Design and Implementation of a Stroke Interface Library", Technical report, Department of Electrical and Computer Engineering, Purdue University, March 24, 1997 Simple zone-based (here called "bins") single/uni-stroke recognition for command input: compare specifically to PenCept "Recognition Macros" of 1985 [WindowsSystemJournal98a] Windows System Journal "Microsoft Windows CE 2.0: It's Not Just for Handheld PCs Any More", Windows System Journal, May 1998: available at http://www.microsoft.com/msj/0598/wince.aspx Review of Windows CE on Handheld PC (with physical keyboard), Palm-size PC (with handwriting recognition or pop-up virtual keyboard), Auto PC (for automobile touchscreen), Pocket Explorer. Palm-size



PC (P/PC) include Casio Cassiopeia E-10, www.casiohpc.com with MIPS processor (not Intel) [WirthN96a] Wirth, Niklaus "Compiler Construction", Niklaus Wirth, 1996 Niklaus Wirth lecture/course notes for upper-level compiler construction course, in book form. Originally distributed in 1996: electronic version with minor changes 2005. Top-down parsing examples use BNF Backus-Nauer-Form, recurstive definitiones. [WoodKR96a] Wood, Kenneth R.; Richardson, Tristan; Bennett, Frazer; Harter, Andy; and Hopper, Andy "Global Teleporting with Java: Towards Ubiquitous Personalised Computing", Nomadics 96, San Jose, U.S.A., March, 1996. Also appeared in IEEE Computer, 1997, vol 30, pp. 53-59 Original publication on VNC Virtual Network Computer remote access. Remote "X"-windows sessions hosted on browser web page as a Java applet, remote frame buffer (RFB). Mentions porting to Windows, mentions briefly security, and renderings on very different devices (e.g. Windows CE). Cites to Dardailler. Compare with later PCAnywhere, GotoMyPC, etc. [WymanRM98a] Wyman, Robert M. "Method and system for managing execution of licensed programs", United States Patent 5,745,879, April 28, 1998 Software licensing (DRM) by calling central license server. Delegation to sub-servers, licenses for different policies, e.g. units of use, which units, location, etc. Calling Card: authorization for one program to call services of a licensed program on another system. [XLibris98a] Xerox PARC, xLibris project Golovchinsky et al "xLibris: The Active Reading Machine", Xerox PARC, 1998 (video demonstration at CHI '98) Electronic tablet/display presenting pages of book/article, user can mark-up/annotate/circle/lasso/underline with electronic ink as aid to "active reading", similar to paper. Automatic indexing of electronic ink, visual/graphical index by showing sections that were underlined/circled/selected. Page turning with physical buttons, not flick gesture. [YaegerLS96a] Yaeger, Larry S. "Apple-Newton Handwriting Recognition", Slide presentation, "Machines that Learn" segment, Snowbird '96 conference. Presentation of Apple-Newton handwriting recognition, later included in Macintosh OS X 10.2 for tablet input. Trainable Neural Net, speaks both of sensitivity to training set, and to on-the-fly training by user: approximately 20 minutes training for full enrollment. Segmentation also by neural net increases word accuracy, based on overlaps, crossings, aspect ratio. Human performance is 90% on same test database (compare: Neisser 1960). Context (n-grams, word list, regex expressions): hard to get meaningful alternates out of neural net. "Stroke warping" artificial data to simulate writing styles (Compare Ward/Kuklinski). Viterbi beam search of word/segmentation hypotheses. [YaegerLS98a] Yaeger, Larry S., Webb, Brandyn, J. and Lyon, Richard F. "Combining Neural Networks and Context-Driven Search for On-Line, Printed Handwriting Recognition in the Newton", Preprint from AAAI "AI" Magazine, Spring 1998. Available at www.beanblossom.in.us/larry/ANHR.html Newton "hand-printing" (versus the "cursive") recognition: character classifiers, integrated multiple representations, normalized output error, negative training, stroke warping (elastic matching?), frequency balancing, error emphasis, quantized weights Mentions extension to cursive as a "challenge" -- which implies that it didn't work? [YaegerLS98b] Yaeger, Larry S. "Neural networks provide robust character recognition for Newton PDAs", IEEE Expert, 1998(?) Copy on file does not give date: also available at http://www.beanblossom.in.us/larry/ANHR.IEEEExpert.html Mentions use of dictionaries, special constructs for dates, time, phone numbers, and ability to write words that are outside the dictionaries: general use of context information Separate references look like patents were pending on several of these features Frequency balancing/weighting in ANN / Adaptive Neural Networks: under-represented writing styles are a problem, such as a three-stroke "5" variant [YanoH96a] Yano, Haruto; Tokunaga, Toshimich; Yoshida, Chisato; Yokouchi, Kauzhiro; Goto, Hirofumi; and Matsumoto, Katsuaki "Touch Device Panel", United States Patent 5,539,429, July 23, 1996 Automobile touchscreen panel: overlay image with keypad or other UI, remove automatically after a timeout. [YarinP98a] Yarin, Paul; Smith, John; and Ishii, Hiroshi "FishFace: An Interactive Device with Integrated Sensing and Display", Submitted to UIST '98. MIT Media Lab: 1998 (fetched 2013 alumni.media.mit.edu/~yarin) Original paper removed from site: now has link to video of FishFace, a combined LED display matrix and touchpad combining input and output in same device. Electric field sensor (capacitive), apparently multitouch. Cited by Tom White. [YasutakeT96a] Yasutake, Taizo "Graphical Input Controller and Method with Rear Screen Image Detection", United States Patent 5,483,261, January 9, 1996 Multitouch optical touchscreen sensor: semi-transparent optically diffusing surface, video camera behind panel sees shadows (dark spots) where user is touching surface with fingers on the front side. Only needs



to detect high contrast dark shadows, so no problems with ambient shadows. Fig. 13 shows two-finger twist/rotate gesture. Mentions using multiple panels for multi-dimensional/three-dimensional/3D input (bimanual?) [ZDNet98a] ZDNet.com "Palm VII ist auf Draht", ZDNet / News, 3. December, 1998 Announcements of Palm VII by 3Com, including wireless network, browser (Netscape), eMail. Palm V with similar features previously announced. [Zeitungstechnik98a] Zeitungtechnik "Die Redaktionsräume der Zukunft stehen ganz im Zeichen von Kommunikation und Kooperation", Zeitungstechnik, April 1998, pp. 28-29, www.ifra.com Press article (in German) on i-LAND project DynaWall, CommChairs, and InteracTable (Streitz) and the future of paperless newspaper editing [ZeleznikRC96a] Zeleznik, Robert C., Herndon, Kenneth P., and Hughes, John F. "SKETCH: An interface for sketching 3D scenes", In Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, page 163170, August 1996 Sketch system for 3D drawing, uses gestures to indicate type of object: two diagonal stroke coming together at the top indicates a cone, three lines are three edges indicating a cube, etc. 3D rendering is approximate. Lasso gesture. User can sketch shadows to indicate 3D perspective by "shadow" gesture plus hand-stroke shading marks. Uses stylus active at both ends ("eraser" end used for camera/point-of-view manipulation), pressure-sensitive stylus on tablet. [ZeleznikRC97a] Zeleznik, Robert C., Forsberg, Andrew S; Strauss, Paul S. "Two Pointer Input for 3D Interaction", 1997 Symposium on Interactive 3D Graphics, Providence RI, pp. 115-120 Two-handed user interface using two independent cursors: prototype using Wacom tablet with simultaneous touch and stylus [ZhaiS96a] Zhai, Shumin; Buxton, William; and Milgram, Paul "The Partial-Occlusion Effect: Utilizing Semitransparency in 3D Human-Computer Interaction", ACM Trans. on Computer-Human Interaction, Vol 3 no 3, Sept. 1996, pp. 254-284 Semi-transparency as depth cue (oclusion) to simulate 3D position helps user target graphical objects (smulated swimming fish, etc.) [ZhaiS97a] Zhai, Shumin; Smith, Barton A.; and Selker, Ted "Dual Stream Input for Pointing and Scrolling", Proc. CHI 97, March 22-27, 1997, pp. 305-306 Study of dual-stream input from two independent devices at once: Mouse with tracking wheel (IntelliMouse), Mouse with joystick button (Trackpoint), and mouse plus keyboard keys, two-handed input. General dual-stream input improve user performance with scrolling and other functions, except for wheelmouse. [ZhuM96a] Zhu, Min "Method to Provide for Virtual Screen Overlay", United States Patent 6,677,188, November 19, 1996 Transparent overlay inking window for adding annotations to a document: compare with transparent inking windows in Pen Windows? User switches between inking overlay window and input into application program. [ZimmermanTG96a] Zimmerman, T.G. "Personal Area Networks: Near-field intrabody communication", IBM Systems Journal, Vol 35, Nos 3 and 4, 1996, pp. 609-617 Near-field (capacitive/electrostatic) communication using human body as part of the antenna. Near-field does not need large antenna matched to wavelength of far-field/electromagnetic/radio signals, can be at very low frequency, and does not involve FCC or similar regulations. Drops off as cube of distance. Can also be very low powers. Concept came from considering combining with capacitive touch/position sensors for users, electric field imaging -- compare with Dietz DiamondTouch touchscreen tablet? Cites to other references on capacitive/electrostatic/near-field imaging. biblio95 (Prev page)

Main Page







Main Page


[AbbateJ99a] Abbate, Janet "Inventing the Internet", M.I.T. Press, 1999 Comprehensive and detailed history of the development of ARPANET/Internet, 1950s to 1999, with dates of technical development related to Internet, remote and network computing, browser/world-wideweb/hyperlinks, applications. [AllenT99a] Allen, Timothy; Day, Shawn P.; and Ferrucci, Aaron T. "Method_and_apparatus_for_scroll_bar_control", United States Patent 5,943,052, August 24, 1999 Touchscreen slider: only use motion along direction of slider, stop slider action of user moves finger perpendicular to slider [AllportDE00a] Allport, David E. "System for Dual-Display Interaction with Integrated Television and Internet Content", United States Patent 6,097,441, August 1, 2000 Touchscreen TV remote control also accessing internet, data encoded in television signal. Videoconferencing, connection to telephone lines. [AlphaPad00] American Tel-A-Systems Inc. "About AlphaPad -- AlphaPad technical information", www.alphapad.com Full-alphabet recognition using 12-key keypad, alternative to Mosaic zone-based handwriting recognition for PDAs by the same group [ArnstGJ00a] Arnst, G. Jeffrey "Schumer v. Laboratory for Computer Systems", July 6, 2000 Report Shumer patent on virtual tablets concerning WinTab specification [ArvoJ00a] Arvo, James and Novins, Kevin "Fluid Sketches: Continuous Recognition and Morphing of Simple Hand-Drawn Shapes", Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology, San Diego, California, November 2000 Pretty-fying of electronic ink and sketches by least-squares fit to a clean shape or line, before the pen/stylus is lifted Does not cite it, but appears similar to spline fitting of electronic ink done by Slate / GO Corporation in the PenPoint operating system: The Slate/Go process was invoked only after the pen was lifted Relies on some simple recognition of which strokes are characters, and which are sketching: also similar to Slate/GO Says that providing initial recognition results while input is still being drawn (with



sketches, not characters) allows lower-quality recognition to be used, because the user can stop as soon as the correct result is displayed [ArvoJ00b] Arvo, James and Novins, Kevin "SmartText: A Synthesis of Recognition and Morphing", AAAI Spring Symposium on Smart Graphics, Stanford California, March 2000, pages 140-147 Shows a UI for handwriting where handwritten characters are gradually replaced by font recognition results, and pretty-fied / normalized in position [AscensionTechnology99a] ascension-tech.com "Ascension Technology: MotionStar - Position and Orientation Measurement System, Installation and Operation Guide for Flock of Birds MotionStar Model with Extended Range Transmitter, Rev.A", Ascension Technology Corporation, October 1, 1999 (First revision 1995) 3D/6DOF "tablet" position sensors, extended range over 170 inches: at greater distances, greater filter lag. [AscensionTechnology00a] ascension-tech.com "Ascension Technology: pcBIRD-Position and Orientation Measurement System: Installation and Operation Guide", Ascension Technology Corporation, October 1, 1999 (First revision 1995) 3D/6DOF "tablet" position sensors, range 4 feet, extended range over 10 feet. Pulsed DC magnetic field. One sensor per interface card, one transmitter for all. [AshdownMSD00a] Ashdown, Mark S.D. "Large-Format Computer Interfaces", Ph.D. First Year Report and Research Proposal, Univ. of Cambridge Computer Lab., UK, July 2000 Bimanual two-handed input user interface. Polhemus multi-stylus (multi-touch) hardware, Wacom multistylus Into tablet, Mimio by Virtual Ink (note: not the display company). [BalakrishnanR99a] Balakrishnan, Ravin and Hinckley, Ken "The Role of Kinesthetic Reference Frames in Two Handed Input Performance", Proc UIST 99, Asheville, NC, pp. 171-178 Study of asymmetric two-handed / two-pointer user interface using Wacom two-puck digitizing tablet: multi-touch. [BalakrishnanR99b] Balakrishnan, Ravin; Fitzmaurice, George; Kurtenbach, Gordon; and Buxton, William "Digital Tape Drawing", Proc UIST 99, Asheville, NC, pp. 161-169 Simulate laying flexible tape in two-handed graphical drawing user interface, to model behavior of tape drawing in automobile body design to get smooth curves. Prototype did not use multi-touch touchscreen, used Ascension Flock-of-Birds electromagnetic trackers. [BarnettS99a] Barnett, Shawn "Jeff Hawkins: The man who almost single-handedly revived the handheld computer industry", Pen Computing Magazine, 1999. Archive at www.pencomputing.com/palm/Pen323/hawkins1.html Interview with Jeff Hawkins: GridPad, Handspring, Palm, Zoomer, Newton, Graffiti [BartlettJF00a] Bartlett, Joel F. "Rock 'n Scroll is Here to Stay", IEEE CGA, May/June 2000WRL Report 2000/3, Compaq Western Research Laboratory, Palo Alto Embodied user interface: Hand-held device with accelerometer in a display: user input is by gestures moving device: pitch, roll, slosh, snapping, tapping, fanning. Device automatically changed from portrait to landscape mode based on how it was held: compare with iPhone accelerometer. Cites to earlier work on accelerometers, graspable user interfaces. [BartlettJF00b] Bartlett, Joel F.; Brakmo, Lawrence S.; Farkas, Keith I; Hamburgen, William R.; Mann, Timothy; Viredaz, Marc A.; Waldspurger, Carl A.; Wallach, Deborah A. "The Itsy Pocket Computer", WRL Research Report 2000/6, Compaq Western Research Laboratory, October 2000 Hand-held/palm computer running Linux and X-Windows, touchscreen. Fanning gestures to review photo albums, tilt-and-scroll using built-in accelerometer. Pictures and be shown in either landscape or portrait mode. Long section on power management, clock rate. [BartlettJF00c] Bartlett, Joel F. "Wearable Computing Device Mounted on Superior Dorsal Aspect of a Hand", United States Patent 6,151,208, November 21, 2000 Proprioceptive (skin/stretchable) sensor on back of hand, includes a display. Slide-to-unlock gesture? [BatesCL99a] Bates, Cary L. and Day, Paul R. "Graphical user interface method and system that provides an inertial slider within a scroll bar", United States Patent 5,872,566, February 16, 1999 Scollbar scrolls faster velocity when scrolling larger objects (larger objects have greater inertia), or user moves mouse or touchscreen input faster: no coasting, object stops immediately when mouse is no longer moved. [BeatonBF00a] Beaton, Brian Finlay; Smith, Colin Donald; and Stalkie, Bruce Dale "Navigation Tool for Graphical User Interface", United States Patent 6,037,937, March 14, 2000 Transparent graphical user interface tools on touchscreen of smartphone. Press-and-hold of one to two seconds. Analog (resistive sheet?) touchscreen, but can determine pointer size (stylus, finger) from contact area / touch point distribution, or by speed of pressure change. One feature is showing translucent/transparent navigation arrows on small screen all the time, can be enabled when needed (reduces screen real estate needed). [BenderGT99a] Bender, Gregory T. "Touch Screen Performance as a Function of the Duration of Auditory



Feedback and Target Size", Ph. D. Thesis, Wichita State University, Spring 1999 touchscreen targetting accuracy improved -- especially for small targets -- with an auditory tone between 50 and 400 ms, not longer or shorter. Shape of target/button has less effect. Mentions that resistive overlay touchscreens may transmit only as little as 55% of light, problems of visual parallax. Touch/tap and long-hold gestures for single and double-click emulation. No references to haptic feedback. Stylus more accurate than finger (but no discussion of specific stylus problems or uses) [BensonG00a] Benson, Glenn "Computer system for protecting software and a method for protecting software", United States Patent 6,047,242, April 4, 2000 Siemens e-license: challenge/response with asymmetric keys (compare with HTTPS/PKI) for DRM rights managements for software [BeriS00a] Beri, Sanjay and Slivka, Benjamin W. "Method and system for displaying hypertext documents with visual effects", United States Patent 6,141,018, October 31, 2000 Animated rolling/moving marquee of thumbnails of HTML documents or images, either circular cycle at end, or bounce at end of marquee [BierEA99a] Bier, Eric A. "Customizable User Interfaces for Programmed Computer Systems", United States Patent 5,862,395, January 19, 1999 Customizable buttons Bbackground describes Apple's Hypercard system; describes creating pop-up buttons, multi-state buttons, and radio buttons; providing feedback to show selected menu entry. Mentions GUIDE hypertext system. says "the actions performed by the buttons can refer to the contents or appearance of the button itself, and hence to any properties that the document editor supports. Thus buttons can be used to modify their own appearance or to transfer their properties to other objects in the same document editor." [BissetSJ99a] Bisset, Stephen J.; Kasser, Bernard; Joss, Berni; Tareyev, Aleksey "Touch Sensing Method and Apparatus", United States Patent 5,920,309, July 6, 1999 Capacitive matrix touch-screen sensor for simultaneous proximity of one or more fingers: multi-touch. Also refers to transcapacitance / mutual capacitance in a grid being well known in the field. [Blickenstorfer00] Blickenstorfer, Conrad "PenLab: A much faster, extensively reworked pen tablet", Pen computing magazine, June 2000. Available at www.pencomputing.com Fujitsu 3400 PenLab portable/docking pen computer, Windows 95 with Microsoft pen extensions. Mentioned design change to avoid rear-mounted electromagnetic digitizer, in favor of front-mounted resistive film digitizer. Optical plate to reduce visual parallax. [Bloomberg99] Bloomberg News "Microsoft, others sued over pen patent", CNETNews.com, April 29, 1999 Mitchell Forcier user-interface patent lawsuit against Microsoft, claims that technology misappropriated by Aha! Software and licensed to Microsoft. File contains additional information: "MS sued over Win CE patent clash": Microsoft acquired Aha! (see file) in 1996 for electronic ink recognition: "Smartink" for editing handwriting like text (compare with van Raamsdonk) for InkWriter on Windows CE (sketch editing). Aha! patent reverted to Forcier. [BlouinF99a] Blouin, Francois "Touch Pad Panel with Tactile Feedback", United States Patent 5,977,867, November 2, 1999 Adding a haptic vibrator for tactile feedback to any kind of touch input: tablet, keypad, keyboard, etc. Tactile feedback is in acoustic range: 100 to 5000 Hz. [BohanM99a] Boha, Michael; Phipps, Chad A.; Chaparro, Alex; and Halcomb, Charles G. "A Psychophysical Comparison of Two Stylus-Driven Soft Keyboards", Proceedings of Graphics Interface GI'99 Comparison of QWERTY and "T9" (TegicCommunication) simulated keyboard layouts: QWERTY wins, because users can find the keys more easily. References to papers comparing performance and usability of handwriting recognizers [BricklinD99a] Bricklin, Daniel; Johnson, Eric E.; Friend, John L.; Kelly, Winslow B.; and Levin, Peter H. "Graphic Indexing System", United States Patent 5,867,150, February 2, 1999 PenScheduler patent, divisional of 5,539,427. Recognition technique, user interface GUI techniques for closed loops / lasso gesture. [BrodersenRW99a] Brodersen, Robert W. "InfoPad - past, present and future", ACM SIGMOBILE Mobile Computing and Communications, Vol 3 No 1, January 1999, pp. 1..7 Prototype low-power portable multimedia terminal (slate format) with pen input, speech recognition, video [BrownCM99a] Brown, C. Marlin "Human-Computer Interface Design Guidelines", intellect, Exeter England 1999 Compendium of user-interface guidelines and principles from many reference. Sections 8.4 refers to accidental touch activation with a touchscreen: 1985 reference on switch? [BrownH99a] Brown, Heather and Robinson, Peter "Integrating Paper and Digital Documents", in section 'Intelligent Paper' of "Electronic Documents, Artistic Imaging, and Digital Typography", Roger D. Hersch, Editor. Available at http://www.cl.cam.ac.uk/~pr/publications/dmf99/dmf99.pdf Review of various work on intelligent paper -- Cites to Wellner and DataGlyphs: compare with Anoto (not



cited?). Mentions TEI Text Encoding Initiative for encoding data in printed documents, dataglyphs, electronic ink on paper. Xlibris Active Reading Machine and Dynomite appear similar to Slate DayTimer Notebook product of approximately 1992 (not cited?). Dataglyphs make paper function as a touchscreen. [BrowneH00a] Browne, Hilary; Bederson, Ben; Druin, Allison; Sherman, Lisa; and Westerman, Wayne "Designing a Collaborative Finger Painting Application for Children", Univ. of Maryland Technical report HCIL-2000-17, CS-TR4184, UMAICS-TR-2000-66, September 2000 Multi-touch finger-painting educational prototype using keyboard-sized Multi-Touch Surface (MTS). For children, gestures are too complicated, suggest template of menus / control areas along with drawing: Compare with Doster and Oed? Mentions Fingerworks multi-touch hardware in 2000, Lee/Buxton papers from 1980s, Nakatani 1991. [Bull00a] Bull Electronics Angers, France "AIX Command Reference Vol. 6, X to zcat", Bull Electronics Angers, France, Order Reference 86 A2 43JX 02, April 2000 AIX Windowing system command reference: drag-and-drop, X11 color database for color management in windows. [BurgessD00a] Burgess, David; Felsenstein, Lee; Saunders, Steven E. "Personal Object Detector", Personal Object Detector Stereo object detector / navigation aid for the visually impaired: mounted in eyeglass frames. Two sonar devices, using continuous swept-frequency, measures frequency difference (caused by phase shift from time delay) between reflected signal and internal signal. Output is generated to provide audible psychoacoustic location in user's hearing. Can be one sonar transmitter and two separated receivers, or two transmitter/receiver pairs. [BuxtonW99a] Buxton, William "Case Study 1: Touch Tablets; Chapter 5 of "Haptic Input", Draft Document", www.billbuxton.com Short overview of touch tablets: refers to "dual pressure" sensing and infrared sensing proximity (but not touch) to simulated proximity sensing. States that touch tablets do not actually send signal that stylus is not pressing on tablet, but instead this must be inferred from timing of touch tablet data (tablet stops sending) (Was he sure on this?). [BuxtonW00c] Buxton, William; Fitzmaurice, George; Balakrishnan, Ravin; and Kurtenbach, Gordon "Large Displays in Automotive Design", IEEE Computer Graphics and Application, July/August 2000, pp. 68-75 Review of uses of Large displays (with touchscreen or tablet or pointing input) like Active Desk/Whiteboard to design styling of automobiles: also ImmersaDesk Virtual Reality VR with two datagloves for hand input and stereo headtracking imager. Laser pointer instead of touchscreen, multiple display walls create a "cave". [BuxtonWAS00a] Buxton, William Arthur Stewart and Bell, Jeffery Allen "System for maintaining orientation of a user interface as a display changes orientation", United States Patent 6,115,025, September 5, 2000 Keep user interface elements in display at constant orientation for the user, as the display itself is rotated in a multi-user system As flat display rotates, re-draw image so that image does not appear to rotate to the user: useful in spinning-display 3D display? [BuxtonWAS00b] Buxton, William A.S. and Kurtenbach, Gordon P. "Graphical keyboard", United States Patent 6,094,197, July 25, 2000 Simulated keyboard on a display: however, typing action is gesture-like: tap on key for lower case, drag upwards from key for upper case, drag right for control-c, drag right-up for control-C. Diagonal stroke used for whitespace/backspace/return/delete. Describes pendown/pendrag/penstill/penup events (press-andhold), not just mouse button up/down. See also http://www.billbuxton.com [BuxtonWAS00d] Buxton; William A.S.; Harrison; Beverly L., Vicente; Kim J "Graphical user interface with optimal transparency thresholds for maximizing user performance and system efficiency", United States Patent 6,118,427, September 12, 2000 Transparent user interface tools on touchscreen user interface: e.g. keypad menu. Describes dithering/stippling/screen-door transparency, including use of a mask. Based on user experiments, less transparent overlays used over text/wire-frame background for visibility, vs. solid/video image underneath. Surface (blank) area of overlay window automatically transparent, transparency of objects in overlay ("nonsurface") varied for visibility depending on type of forground image (text/line-drawing vs. solid icons). [CERT00] CERT Cordination Center "Results of the Security In ActiveX Workshop", Security in ActiveX Workshop, Pittsburgh, Pennsylvania USA, August 22-23, 2000. Published at the CERT Coordination Center, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA 15213, December 21, 2000 Review of ActiveX security problems and measures [CHI00a] CHI 2000 "Touchscreen Toggle Design - Slider", CHI posting, dated 2000/2001 Digitization of 1992 CHI demonstration Video of "Touchscreen Toggle Design"? [CIDCO99a] CIDCO "CIDCO State-of-the-Art Screen phone -- iPhone desktop smart telephone", www.cidco.com, 1999



same names as Apple iPhone, but different product. Desktop stylus touchscreen/display telephone with web surfing (via phone modem) and email. [CappsS00x49] Capps, Stephen et al "System and method for organizing recognized and unrecognized objects on a computer display", United States Patent 6,021,218 Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990. Compare to van Raamsdonk, Pencept, and CIC? [CarrollDW00a] Carroll, David W.; Carroll, James L.; and Case, Steven V. "Touch Screen Systems and Methods", United States Patent 6,121,960, September 19, 2000 Display semi-transparent virtual/software keyboard over application image in a touchscreen display ("variable-pixel control"), for example in a wearable or hand-held computer. Overlaying may be by blending images, see-through transparency, or dotted-lines for key/keyboard image. Switches between keyboard mode and cursor (drawing) mode based on continuous touching for a period of time (dwell?). Zoom gesture requires multiple fingers. multi-tap (multi-touch) gestures. Dwell/press-and-hold. [Casio99a] Casio Inc. "Casio PV-200 PV-100 Pocket Viewer User Manual", Casio Inc. Casio Touchscreen PDA with stylus: tap/double-tap/drag, on-screen keyboards for specific applications, electronic ink sketching and drawing, on-screen keyboards (alphanumeric, symbols/punctuation, date keyboard (date picker, with drag for ranges), calculator keyboard, time/dial with rotary/circular virtual key arrangement. Electronic file has additional materials, including PV Merlin handwriting/gesture recognition (backspace, enter gestures, etc.) Unistroke? [Casio00a] Casio Inc. "Mobile Datenerfassung: Hand-PC mit Touch-Panel und Belegdrucker IT-2000", CASIO Computer Deutschland, D-22848 Norderstedt, Germany, Dec 12 2000 Product information on IT-2000, hand-held touchscreen/pen computer for data collection. [Casio00b] Casio Inc. "IT-2000 Data Collector User's Guide, 2000", CASIO Computer Deutschland, D-22848 Norderstedt, Germany, Dec 12 2000 User guide, Product information on IT-2000, hand-held touchscreen/pen computer for data collection. Refers to stroke keys (physical buttons) and touch keys (virtual buttons): virtual buttons can show multiple user-selectable keyboard layouts. [ChambersP00a] Chambers, Peter and Wehunt, Omer Lem Jr. "Cache based scan matrix keyboard controller", United States Patent 6,040,788, March 21, 2000 Scan matrix keyboard that allows multiple keys to be pressed simultaneously an recognized (chord input). Rows are pulse signalled sequentially, column conductors sense when signals are detected from that particular column, thus scanning entire grid matrix. Key switch technology can be mechanical, electrical, or capacitive. Compare to Kaplow multi-touch programmable keyboard/input device. [ChanCF99a] Chan, Chow Fong; Maisy, Mun Lan Ng; Ong, Eng Yue; Geng, Xia; Lim, Swee Hock Alvin "Touchpad Pen-Input and Mouse Controller", United States Patent 5,995,084, November 30, 1999 Tap, drag, tap-and-drag gestures on touchpad digitizer, with check for moving out of working area. Note: press-and-hold: tap-and-drag does *not* have a hold. Switches between pen-input and mouse-input mode. Refers to light pen. [ChanCF99b] Chan, Chow Fong; Maisy, Mun Lan Ng; Ong, Eng Yue; Geng, Xia; Lim, Swee Hock Alvin "Touchpad Pen-Input Controller", United States Patent 5,883,622, March 16, 1999 Force/pressure sensing touchpad digitizer tablet using resistive film: compare with Elographics [ChanCF99c] Chan, Chow Fong; Ong, Eng Yue; Geng, Xia; Lim, Swee Hock Alvin "Automatic Cursor Motion Control for a Touchpad Mouse", United States Patent 5,880,717, March 9, 1999 simulated velocity/momentum when scrolling when object moves outside/into work area boundary: compare with artificial reality project? [ChanCF00a] Chan, Chow Fong; Maisy, Mun Lan Ng; Ong, Eng Yue; Geng, Xia; Lim, Swee Hock Alvin "Touchpad Mouse Controller", United States Patent 6,057,830, May 2, 2000 Tap, drag, tap-and-drag gestures on touchpad digitizer, with check for moving out of working area. Note: press-and-hold: tap-and-drag does *not* have a hold. [ChanCF00b] Chan, Chow Fong; Maisy, Mun Lan Ng; Ong, Eng Yue; Geng, Xia; Lim, Swee Hock Alvin "Command Set for Touchpad Pen-Input Mouse", United States Patent 6,061,051, May 9, 2000 tap/click, move, press and drag gestures on touchpad/touchscreen digitizer. Uses two distance thresholds, time threshold also for establishing contact in press-and-hold gesture. [ChenJF00a] Chen, James F.; Wang, Jieh-Shan; Brook, Christopher T; and Garvey, Francis "Multi-access Virtual Private Network", United States Patent 6,158,011, December 5, 2000 Authenticate both ends of a VPN virtual private network connection via software layer (shim) hooked into socket connection code: multiple connections on VPN. Authentication handled by central server, communications are peer-to-peer. [ChengEC00a] Cheng, Edward C. "Dynamic organization model and management computing system and method



therefor", United States Patent 6,067,548, May 23, 2000 Project management and organization system [ChewCH99a] Chew, Chee Heng; Konzen, Neil; Guzak, Christopher J.; Laney, Stuart T.; Pitt, George H. III; Ellison-Taylor, Ian M. and Gery, Ron O. "System Provided Child Window Controls", United States Patent 5,889,522, March 30, 1999 UI Window controls: image lists, hot keys, hot key configuration. Specification includes verbatim excerpts from Microsoft documentation for Header Controls [ChiuP99a] Chiu, Patrick; Kapuskar, Ashotuosh; Reitmeier, Sarah; and Wilcox, Lynn "NoteLook: Taking Notes in Meetings with Digital Video and Ink", Proc. ACM Multimedia '99, October 1999, Orlando Florida, pp. 149-158 Electronic ink note-taking application: computer (pen computer, Windows 95) shows video and slides from live classroom presentation, user can make ink and/or voice annotations at the same time. Nikon Coolpx 300: digital camera plus stylus capture for electronic ink notation for screen displays. Compare to Wang Freestyle (not cited). [Cirque99a] Cirque Technology "GlidePoint: Advanced PC-Compatible Features", www.cirque.com, 3 October 1999 Touchpad gestures: can move past edge by moving to edge, touchpad firmware holds position for three seconds to allow user to scroll/drag farther by stroking pad again. Compare with "edge motion" references. [ClineTL00a] Cline, Tory Lee and Poston, Ricky Lee "Method and apparatus for improved scrolling functionality in a graphical user interface utilizing a transparent scroll bar icon", United States Patent 6,069,626, May 30, 2000 Pop-up transparent scroll bar icon: in response to a gesture? press-and-hold? [ComputerWorld99a] Kay, Russell "Qbe Tablet is Stylish and Function-Rich", ComputerWorld, November 29, 1999, p. 56 Qbe Tablet-form pen computer by Access Technologies, Windows 98 Windows NT with Calligrapher handwriting recognition, virtual keyboard, built-in camera. Mentions Fujitsu and other tablet computers with touchscreen. [ComputingUnplugged99a] Computing Unplugged "A first look at the Palm VII", Computing Unplugged, June 1, 1999 Palm VII PDA/cell/smartphone with touchscreen user interface: includes wireless web browsing [CortopassiM99a] Cortopassi, Michael; Gladwin, Christopher; and Voegeli, Derick W. "Resume on pen contact", United States Patent 5,974,558, October 26, 1999 Low-power mode on portable pen computer, passive stylus: monitors for stylus contact on touchscreen to wake device up from hibernation. Larger system has movable virtual on-screen keyboard, other GUI. Compare with PenPoint, Wang Freestyle? [DAmicoVE99a] D'Amico, Vincent E.; Gehrig, Jay J.; Surprenant, Joseph E.; and Robsky, Steven R. "Touchscreen Controller with Pen and/or Finger Inputs", United States Patent 5,956,020, September 21, 1999 Dual mode digitizer for finger input or stylus/pen input. Says "finger and pen", but only reports one, with pen/stylus getting priority. Includes built-in calibration for active area by touching two corner locations via controller commands. Capacitive film digitizer technology (refers to 4,198,539 for sensor details). Looks like MicroTouch TouchPen. [DammCH00a] Damm, Christan Heide; Hansen, Klaus Marius; and Thomsen, Michael "Tool Support for Cooperative Object-Oriented Design: Gesture Based Modeling on an Electronic Whiteboard", Proc. CHI 2000, April 2000, pp. 518-525 Knight tool: Gesture-based multi-user whiteboard user interface. Says unfortunately no multi-touch (single users must take turns) on prototype hardware (Smart Board), unlike Xerox Liveboard (Tivoli). Gestures include sketch recognition, e.g. UML rectangle objects. Context-dependent pie menus (marking menus) for large surface. "Radar window" shows small image of entire workspace drawing. Notes that Xerox Liveboard could accept multiple pen touches at the same time. [DavisJ00a] Davis, Jame and Chen, Xing "LumiPoint: Multi-User Laser-Based Interaction on large tiled displays", Stanford University Technical Report CS-TR-2000-04, June 2000 Multi-user (multi-touch) large screen display for collaborative work using laser pointers. Rear cameras detect light spot from laster pointer: multiple cameras to cover large (tiled) display. Corrections needed for parallax/other offests from different cameras to correct position. Strokes (by different users) determined by proximity at 60 Hz frame rate. Gestures using Goldberg unistroke recognition (see 1993 paper). [DavisTR99] Davis, Tom R. "Bibliography on Handwriting, natural, taught styles, copy-books", University of Birmingham, U.K. Shorthand, taught handwriting styles by education, single-stroke writing, Pittman [DeFreitasSC99a] DeFreitas, Scott C. "Filling of graphical regions", United States Patent 5,870,100, February 9, 1999 Rapid algorithm for flood fill of a region of same color (or other same attribute) of a pixel in the region. [DeIpinaDL00a] de Ipina, Diego Lopez "Building Components for a Distributed Sentient Framework with Python and CORBA", Unknown conference proceedings, date approximate



The title is odd, but the paper makes substantial references to 2D circular barcode (ringcode) tags, and inferring the approximate location, orientation, and identifier of the tags using optical scanning / visual processing. Compare with Sekendur. [DeJudicibusD99a] de Judicibus, Dario and Albano, Gherardo "System and Method for Arranging Windows Displayed by a Graphical User Interface", United States Patent 5,874,962, February 23, 1999 Automatic placement of windows in graphical user interface, most important window largest in center with other windows radially around. Orientation / rotation? [DeindlM00a] Deindl, Michael and Witzel, Martin "Method and system for the secure transmission and storage of protectable information", United States Patent 6,031,910, February 29, 2000 Patient has smart card to decrypt/encrypt data from medical records for doctor (or other electronic documents) [DitzikRJ99a] Ditzik, Richard J. "Modular notebook and PDA computer systems for personal computing and wireless communications", United States Patent 5,983,073, November 9, 1999 Electronic notebook/PDA/smartphone, stylus/touchscreen, clamshell/convertible/detachable display, wireless networking, video [DourishP99a] Dourish, Paul; Edwards, W. Keith; LaMarca, Anthony; and Salibury, Michael "Using Properties for Uniform Interaction in the Presto Document System", Proc. UIST '99, Asheville, NC, pp. 55-64 Alternative to folders: objects (files) stored in a database with multiple attributes for each object, objects can be viewed, grouped, or displayed by any combination of attributes (file can be in two different groups at the same time): placeless documents. [DrobneS99] Drobne, Samo "Prenosno in Terenski Racunalnistvo", Master's Thesis in spatial planning, University of Ljubljana, Faculty of Civil and Geodetic Engineering, 1999 Contains examples of gestures: spiral, dash/flick, square. Example of circle/lasso gestures from Handwriter for Windows by CIC Communication Intelligence Corporation, Pen Windows / Pen Extensions, Windows CE, Fujitsu 2300 pen computer [DruinJ00a] Druin et al "KidPad: A collaborative storytelling environment for children (Video demonstration)", Univ. of Maryland Human-Computer Interaction Lab, 2000 Video demonstration of KidPad: See other references. [DuSS99a] Du, Sterling S. and Lee, Yung-Yu Joe "Touch-pad Digital Computer Pointing Device", United States Patent 5,856,822, January 4, 1999 Touchscreen/touchpad device using multiple finger contacts: second finger contact is in a separate sensing area. When user moves finger rapidly above a speed threshold, touchpad sends different data (e.g. it scrolls faster, or enters a power-scroll mode) to compensate for some of the limitations of a small touchscreen on a portable device. When nothing is touching, device automatically adjusts to ambient environment. [DulbergM99a] Dulberg, Martin S.; St. Amant, Robert; and Zettlemoyer, Luke S. "An Imprecise Mouse Gesture for the Fast Activation of Controls", Proc. INTERACT '99, eight pages Mouse click and drag-and-drop gestures with mouse flick gestures: minimum number of pixels movement to distinguish mouse clicks from flicks, velocity threshold for flick and drag: mentions that with stylus selection should be on touch-down, contrasted to mouse on mouse-up event. Note: press-and-hold: threshold, equivalent to tap threshold, velocity to other threshold. [Dynapro99a] Dynapro Systems Inc. "Dynapro SC4 Touch Screen Controller User's Guide", Dynapro Systems Inc, Copyright 1999. www.dynapro.com Electrostatic/capacitive touchscreen digitizer with eight-wire resistive touch screen, silver ink as conductor at sides, to overcome resistance changes and drift due to aging and wear, ratio-metric conversion for A/D converter output. [Dynapro00a] Dynapro Systems Inc. "Dynapro NFI Near Field Imaging Touch Screen System", Dynapro Systems Inc, Copyright 2000. www.dynapro.com Electrostatic/capacitive touchscreen digitizer with patterned conductive sensor (grid matrix?) [ElbaumS99a] Elbaum, Sebasian and Munson, John C. "Intrusion Detection through Dynamic Software Measurement", USENIX Workshop on Intrusion Detection and Network Monitoring, April 1999, pp 41-50 Disallowed Operational Anomaly system: dynamic modelling of program behavior using execution path patterns, deny operation when program behavior diverges from learned patterns. Technical reference for Cylant.com, intrusion detection contrasted to signatures for files or SNORT network patterns [Electrotextiles00a] Electrotextiles "Electrotextiles: The smart fabric of the future", electrotextiles.com, archived June 2000 electronic switches and sensors woven into wearable fabric: single-touch flexible tablet: appears to be resistive film equivalent. Switch matrix for multiple touch inputs? (Used for flexible keyboard) [EmD00a] Em, Davic "Fujitsu Stylistic 3400", byte.com, August 18, 2000 Pen tablet computer: not Tablet PC, uses CIC PenX handwriting-recognition software (trainable), with on-



screen soft keyboard. Runs modified Windows 9X / Windows98 OS [EmD00b] Em, Davic "Invasions of the Pen Computers, Part I", byte.com, August 18, 2000 Pen tablet computers: not Tablet PC, mostly Windows 95 / Windows98. Aqcess Technologies QBE tablet computer. [EricsonP00a] Ericson, Petter; Fahraeus, Christer; and Hugosson, Ola "A method and a device for matching images", European Paten Application EP 1050012 A1, November 8, 2000 Match overlap on adjacent images for best fit: applied to Anoto pattern [Ericsson99a] Ericsson Mobile Communications AB "Mobile Phone R380 - Design Guidelines for WAP Services", Ericsson Mobile Communications AB, November 1999 Touchscreen smartphone, on-screen virtual keyboard, handwriting recognition. User interface guidelines for WAP/WML (reduced HTML: Wireless Markup Language) environment for browser in very small handl-held display. [Ericsson00a] Ericsson Mobile Communications AB "Ericsson R380 Users Manual First Edition", Ericsson Mobile Communications AB, 2000 Touchscreen smartphone, on-screen virtual keyboard. three-point calibration of touchscreen with stylus. Flip closed, non-touchscreen display and physical keypad; flip open touchscreen desktop with icons. Handwriting recognition mostly single strokes: Grafitti II? Palm OS? Period is diagonal stroke, not tap; accent marks entered next to main character. Extended handwriting character set for oested, Yen symbol, OE/AE, etc. [Ericsson00b] Ericsson Mobile Communications AB "SecurID Ready Implementation Guide, Ericsson R380", Ericsson Mobile Communications AB, October 26, 2000 Implementation of RSA SecurID on touchscreen smartphone for use as two-factor security token.: "generate passcode" application. [Fingerworks00a] Fingerworks Inc. "Fingerworks - Inventor and Developer of MultiTouch Technology", Fingerworks.com, fetched Aug 17, 2000 Image of multitouch input of hand from Fingerworks input tablet Additional information in paper file [Fishkin00] Fishkin, Kenneth P.; Goldberg, David; Gujar, Anuj Uday; Harrison, Beverly L.; Mynatt, Elizabeth D; Stone, Maureen C; and Want, Roy "Zoomorphic Computer User Interface", United States Patent 6,160,540, December 12, 2000 Electronic teddy bear with various computer interactive technologies: specifically mentions tablet computer and gestures (e.g. shake) European patent also on file: same Goldberg of Unistroke [FitzmauriceGW99a] Fitzmaurice, George W.; Balakrishnan, Ravin; Kurtenbach, Gordon; and Buxton, Bill "An Exploration into Supporting Artwork Orientation in the User Interface", Proc. CHI 99, May 1999, pp 167-174 For drawing/sketching, permit user to turn or rotate touchscreen or pen-computer display, automatically rotate orientation of menus, pop-ups, and UI to keep correction orientation to user. [FitzmauriceGW99b] Fitzmaurice, G. and Buxton, W. "Temporal data control system", United States Patent 5,973,669, Oct. 26, 1999 "Scrubwheel": mouse with a rotational control, so that it can both be positioned (to select something, such as which video clip) and rotated (as a knob to control speed of rendering) See also http://www.billbuxton.com [FleckDC00a] Fleck, David C. "Digitizer system with cursor shape changing as a function of pointer", United States Patent 6,160,539, December 12, 2000 Change cursor on display depending on which area of a menu strip on separate tablet a stylus is hovering over in proximity. [Fonix99] Fonix Corporation "Fonix and Lucent Technologies Inferno Software Solutions To Offer Fonix Handwriting Recognition Software to Information Appliance Market", Lucent Technologies Shows graffitt-like unistroke character recognition, including unistroke "X" and German "Sharp-S". Mentions Inferno Operating System [Forcier99] Forcier, Mitchell D. "Script Character Processing Method and System with Bit-Mapped Document Editing", United States Patent 5,953,735, September 14, 1999 Editing of handwriting/electronic ink without recognition of characters Editing of electronic ink: word-wrap, inserting spaces, editing of handwritten lines of text. Compare with van Raamsdonk? Press reports from 1999 of Forcier accusing Microsoft of patent infringement for electronic ink editing [ForestDK99a] Forest, Donald K. "User interface method and apparatus", United States Patent 6,005,549, December 21, 1999 Perimeter menus for entering ideographics on a touchscreen (or other pointing device, mouse, headmouse for handicapped, etc.) for entering Chinese or other ideographic characters, or for speech generator input for motor handicapped. Items may be selected by a dwell time (press-and-hold), then move to submenu (which may be displayed). Show a visual indicator (brightness) in menu item on screen of how much dwell time is remaining. Discontinuous dwell time (aggregate dwell) for users who have hand



tremors. Also different dwell time thresholds for different keys. Compare with marking menus. Numerous references to dwell time (press-and-hold) in computer accessibility (e.g. HeadMouse), in addition to touchscreens. [ForestDK00a] Forest, Donald K. "Dwell Time Indication Method and Apparatus", United States Patent 6,160,536, December 12, 2000 Perimeter menus for entering ideographics on a touchscreen (or other pointing device, mouse, headmouse for handicapped, etc.) for entering Chinese or other ideographic characters, or for speech generator input for motor handicapped. Items may be selected by a dwell time (press-and-hold), then move to submenu (which may be displayed). Show a visual indicator (brightness) in menu item on screen of how much dwell time is remaining. Discontinuous dwell time (aggregate dwell) for users who have hand tremors. Also different dwell time thresholds for different keys. Compare with marking menus. [ForresterJE00a] Forrester, Justin E. and Miller, Barton P. "An Empirical Study of the Robustness of Windows NT Applications Using Random Testing", Proc. 4th USENIX Windows System Symposium, August 2000, Seattle Washington Fuzz-testing (random keyboard and mouse inputs) crashes or causes to fail most Windows NT application. Continuation of similar work done on Unix in the 1990s, that few programmers write software that check input for plausibility, so unanticipated inputs cause failure. [FranczekEJ99a] Franczek, Edward J; Bretscher, John Thomas; and Bennett, Ramond Walden III "Computer Virus Screening Methods and Systems", United States Patent 5,987,610, November 16, 1999 Scan data sent between two parties on telephone lines for a virus and block part of the data from being transmitted. Executing program that would be transmitted in a model of the receiving computer to detect virus before transmitting data. [FrankEH99a] Frank, Edward H.; Naughton, Patrick J.; Gosling, James Arthur; and Liu, John C. "Method and apparatus for presenting information in a display system using transparent windows", United States Patent 5,999,191, December 7, 1999 Transparent windows in a GUI, using alpha blending of pixel data. Example show menu boxes with buttons overlaid on other windows. [FreeSoftwareFoundation00a] Free Software Founcation "Drag Start Distance", http://140.116.2.3/Linux/CLE/CLE/devel/andrew/redhat/BUILD/kde-i18n/eo/messages/kdebase/kcminput.po Documentation of Drag Start Distance in open-source software. press-and-hold-and-drag distance. File contains additional discussion references to Drag Start Distance in KDE and other open-source software related to Linux O.S. [FreseV99a] Frese, Vincent II and Blevins, Brian "Method and system for on demand downloading of module to enable remote control of an application program over a network", United States Patent 5,909,545, June 1, 1999 Deploying remote UI over web using Java: display and application (interface) modules for remote display. Cites to X Over the Web. See also VNC? [Freund99] Freund, Gregor "System and Methodology for Managing Internet Access on a Per Application Basis for Client Computers Connected to the Internet", United States Patent 5,987,611, November 16 1999 Zone Labs: per-application firewall. Mentions intercepting process loading (via kernel hooks?) and checking application details, such as components, version, checksum. Not adaptive, rule-based. [FrinkL99a] Frink, Lloyd and Bishop, Bryon Dean "Method and System for Data Entry of Handwritten Symbols", United States Patent 5,946,406, August 31, 1999 Pen Windows emulation of keyboard input by placing invisible transparent handwriting input window penpalette over text-entry field, user perception is that writing is directly into the underlying text-entry field. Mouse-hook function detected stylus touch (WM_LBUTTONDOWN) in text field to start. Note: Feature appeared to be less prevalent in later versions of Pen Windows. [FuruhataT99a] Furuhata, Tomotake; Hirano, Toshiki; and Sawada, Chihiro "Touch panel double-touch input method and detection apparatus", United States Patent 5,943,043, August 24, 1999 Detect multi-touch (double-touch gesture by two fingers) on resistive or other digitizer by noting shift in position from tap by second finger, then position of first finger returning after 2nd finger's tap. Implementation of double-tap gesture to prevent problems of single finger/stylus moving between the taps. [Gadgeteer99a] The Gadgeteer "Palm VII Review", The Gadgeteer, August 29, 1999. Available at thegadgeteer.com, 2013 Review of Palm Pilot VII pen/stylus handheld PDA. Includes wireless antenna communications for access to portions of internet via "Web Clipping", wireless email. [GastonKR99a] Gaston, Kerry R. "Electronic Book System", United States Patent 6,956,048, September 21, 1999 Electronic book, with docking/charging station for downloading books. Touch input screen for making notes on displayed text or in margin, copy/digital-rights-management protection. Includes video. [GeigerFJ00a] Geiger, Fred J.; Wood, William K.; and Tandon, Sonjaya T. "Automated Post Office Based Rule



Analysis of E-Mail Messages and Other Data Objects for Controlled Distribution in Network Environments", United States Patent 6,073,142, June 6, 2000 Spam filtering using distributed post offices / mail servers/gateways, which may apply different business/filtering rules. [GelsingerPP99a] Icras, Inc. "Method for assisting window selection in a graphical user interface", United States Patent 5,892,511, April 6, 1999 Translucent windows in GUI so that you can see what is underneath. [GeneralMagic00a] Icras, Inc. "Using Magic Cap: Magic Cap 3.1. The User's Guide for the DataRover 840", General Magic, 2000 Full user guide for DataRover 840 stylus-input PDA, using resistive touchscreen digitizer and stylus. Pop-up virtual keyboard. Electronic ink data annotations and drawing, no handwriting recognition. Numerous screen shots of user interface shows simulated office rooms and objects: compare with Windows "Bob"? [GeneralMagic00b] Icras, Inc. "Magic Cap 3.1 User Interface Specification", General Magic/Icras, 2000 General Magic UI and operating system for tablet/pen computers: Includes flip orientation to rotate screen 180 degrees for left-handed users. [GeneralMagic00d] Icras, Inc. "Design and Magic Cap: Magic Cap SDK for Windows", General Magic/Icras, 2000 General Magic desktop is picture of desktop with office object: filing cabinet, rolodex, printer, fax, calendar, etc. Compare with Microsoft "Bob". [GeneralMagic00e] Icras, Inc. "MagicSDK Package Development Guide, version 3.1 "Rosemary"", General Magic/Icras, 2000 General Magic development: views are rotatable and flippable in 8 orientations. [GeneralMagic00f] Icras, Inc. "MagicSDK Tutorial for Microsoft Windows - Simulator", General Magic/Icras, 2000 General Magic development: SDK/IDE with windows-based simulator for General Magic pen PDAs [GerpheideGE99a] Gerpheide, George E. "Method and Apparatus for Data Input", United States Patent 5,861,875, January 19, 1999 Capacitive digitizer using X/Y matrix grid, interpolation between lines. [GersheneldN99a] Gershendel, Neil; Zimmerman, Thomas; Allport, David "Non-contact System for Sensing and Signalling by Externally Induced intra-body currents", United States Patent 5,914,701 June 22, 1999 Capacitive/electrostatic (quasi-electrostatic) signalling through human body. Transmission of signals from devices of one user to devices of another user then shaking hands/in contact. Multiple devices/transmitters can share by using different spread-spectrum pseudo-random codes for signalling. One hand and twohand (multi-touch) gestures, detect X/Y or 3D position of hands from signals received by receivers on edges of display. If transmitter also display, user touches transmitter with one hand, points/gestures with other. [GillespieD99a] Gillespie, David; Allen, Timothy P.; and Ferrucci, Aaron T. "Pressure Sensitive Scrollbar Feature", United States Patent 5,889,236, March 30, 1999 Capacitive touchscreen using X/Y grid, compute centroid (interpolation) of object in proximity or touching the touchscreen. Has particular algorithm for detecting tap and drag gestures. Refers briefly to multi-finger gestures because entire grid is measured, (and multiple centroids could be detected?) Scrolling speed faster if more pressure (force). Edge motion for scrolling: moving finger to outside margin of touchpad causes scrolling in that direction (for small touchpads). [GladstoneK99a] Gladstone, Kat, and Haber, Andrew S. "Licensing opportunity for technology which may be of interest", [email protected] Discussion List, Sep 29, 1999 posting of information on US Patent 5,018,208, "Input Device for Dynamic Signature Verification". Mentions use as a biometric automobile lock, with the benefit that neuromotor effects on handwriting variability would prevent drunk drivers from using their car. Contact information is Kate Gladstone - Handwriting Repair, Albany, NY [GoffL99a] Goff, Leslie "The S/360 Learning Process", ComputerWorld, Online News, 4/19/99, http://careers.computerworld.com/home/news.nsf/all/9904191964 Lists members of the GRAIL project at the RAND Corporation: Bill Sibley, Tom Ellis, Gabe Groner (Groener? ), Mal Davis Describes deficiencies of IBM S/360 interrupt processing, and TCBs / Task Control Blocks, to capture electronic ink in real time [Goldmine99a] Goldmine Software Corporation "Goldmine At-a-Glance: Goldmine 5.0", Goldmine Software Corporation, Colorado Springs, CO, October 1999 All-in-one Email, scheduling, Calendaring and contact-information manager PIM for DOS operating systems. Sales contact forcasting. Auto-dialing (via phone modem) of telephone number in contact database. [GoughML00a] Gough, Michael L.; MacDougald, Joseph J.; Venolia, Daniel S.; Gilley, Thomas S.; Robbins, Greg M.; Hansen, Daniel J. Jr.; and Oswal, Abhay "Method and apparatus for providing translucent images on a computer display", United States Patent 6,072,489, January 6, 2000



Updating a translucent/semi-transparent window [GouzmanR99a] Gouzman, Roman; Karasin, Igor; and Braunstein, Art "The Virtual Touch System by VirTouch Ltd: Opening New Computer Windows Graphically for the Blind", Proc. CSUN 99: Web Posted January 15, 1999, www.dinf.ne.jp/doc/english/Us_Eu/conf/csun_99/session0262.html Three-cell (4x8 pin braille display cells) tactile/haptic display built into a mouse, intended as graphical (vs. text) display for the visually impaired/blind. Four (two bits) shades of "grey". System includes speech output / screen reader. [GouzmanR99b] Gouzman, Roman; Karasin, Igor; and Rozenblum, Dmitri "Mouse-like input-output device with display screen and method for its use", United States Patent 5,912,660, June 15, 1999 Mouse with tactile feedback, multiple braille-like pins under fingers. One is fiduciary point indicating position of cursor on display, others display surrounding parts of image. Pins can be partially activated for gray scale. [GreggLE99a] Gregg, Leon Edward; Haugh, Julianne France; and Jaaskelainen, William Jr. "Hardware-selectable Mouse Movement", United States Patent 5,963,195, October 5, 1999 Mouse pointer with two rotary wheel adjusters built-in to set X and Y scale/gain for mouse motion/acceleration, separate from whether OS or application allows for adjusting any mouse-movement parameters [GriswoldGN99a] Griswold, Gary N. "Licensing Management System and Method in which Datagrams Including an Address of a Licensee and Indicative of Use of a Licensed Product are Sent from the Licensee's Site", United States Patent 5,840,504, August 17, 1999 DRM licensing system, secure software application monitor pings / datagrams to license server with requests periodially. Server can reply denying access: if server inaccessible for period of time, access also denied. Datagrams counted for billing. [HUSAT00] HUSAT "HUSAT: Background to the Institute", Presentation at Human Factors 2000 Symposium: Human Sciences and Advanced Technology at Loughborough University, U.K. HUSAT presentation human factors in user-interface technologies: history to Dimond Stylator and RAND Tablet: speech and handwriting seem natural to humans, but may not be most natural for humans interacting with machines States that using a sliding window in a recognizer reduces error rate by 50%, but does not give the point of comparison, or the description of how the error was computed (false positive, false negative, etc.) URL as of 2002: www.lboro.ac.uk/research/husat/news/proj/jn_presentation.html [HarrisDR99a] Harris, Daryl R.; Williams, Daniel L.; and Walczak, Thomas J. "Hand-held radiotelephone having a detachable display", United States Patent 6,009,336, December 28, 1999 Smartphone/radiotelephone with video camera and display: display is detachable so that usr can speak privately with rest of the phone. [HayesJJ00a] Hayes, John Joseph Jr. and Thornton, Curtis Wayne "Radiotelephones having contact-sensitive user interfaces and methods of operating same", United States Patent 6,131,047 (RE43931), October 10, 2000 Put touchscreen on a smartphone/radiotelephone, scroll by dragging finger. User can disable touchscreen (example devices also has phone keypad). [HeinerJM99a] Heiner, Jeremy M.; Hudson, Scott E.; and Tanaka, Kenichiro "Linking and Messaging from Real Paper in the Paper PDA", Proc. UIST '99, Asheville, NC. pp.179-186 Paper PDA: paper can be used as PDA for information management by scanning in pages of paper, archiving/indexing/processing electronically, and printing out new copy. StickerLink: on-paper hyperlinking using paper stickers with URLs. Paper includes circular registration/identification marks: Compare with Sekendur. Email by scanning, doing handwriting recognition on address field. Compare with Anoto notetaking applications, Slate Pen-Scheduler DayTimer application. [HinckleyK99a] Hinckley, Ken and Sinclair, Mike "Touch-Sensing Input Devices", Proc. CHI '99 Touch Trackball, Scrolling Touchmouse: used to change UI depending on what user is doing with devices. Mentions tap-and-hold / press-and-hold. [HinckleyK00a] Hinckley, Ken; Pierce, Jeff; Sinclair, Mike, and Horvitz, Eric "Sensing Techniques for Mobile Interaction", Proc. UIST '00, San Diego, CA, pp. 91-100 Add capacitive touchscreen sensor to Cassiopeia E105 Palm-sized PC, and second touch sensor for bezel. User can touch bezel for full-screen display. [Hong00] Hong, Jason I. and Landay, James A. "SATIN: A Toolkit for Informal Ink-based Applications", Proc. ACM UIST 2000, San Diego California Also mentions DENIM, sketch-based web site design tool using pen input. Makes comparisons with NewtonOS, Windows for Pen Computing, PenPoint. Describes sketch editing, and stroke simplification, examples are all the straightening of free-hand line segments in sketches and in characters. hold-select gesture: Holding pen down for a period of time enters select mode. Discusses press-and-hold (tap and hold) in Newton for select. (hold-and-select) [HorvitzEJ00a] Horvitz, Eric J.; Markley, Michael E.; and Sonntag, Martin L. "Method and system for transforming



the geometrical shape of a display window for a computer system", United States Patent 6,016,145, January 18, 2000 User interface GUI for displaying non-main windows in 3D perspective on the sides, as if they were walls in a room. One advantage is more windows can be seen by user for context. Trapezoidal projection simulating rotation out of the plane of the display. Cursor also resized to simulate distance. [HorvitzEJ00b] Horvitz, Eric J.; Markley, Michael E.; and Sonntag, Martin L. "System and method for resizing an input position indicator for a user interface of a computer system", United States Patent 6,023,275, February 8, 2000 User interface GUI for displaying non-main windows in 3D perspective on the sides, as if they were walls in a room. Also shows circular cyclorama-style wall, rather than room-type walls. Also projection of windows to the "floor". Trapezoidal projection simulating rotation out of the plane of the display. Cursor also resized to simulate distance. [HuL00a] Hu, Liangbing; Kim, Han Sun; lee, Jung-Yong; Peumans, Peter; and Cui, Yi "Scalable Coating and Properties of Transparent Flexible Silver Nanowire Electrodes", ACS Nano magazine, Vol 4 No 5, 2000, pp. 29552963 jumbled silver nanowires as replacements for ITO indium tin oxide in touchscreens etc: graphene electrodes in thin film; single-walled carbon nanotubes CNT, metal/silver nanowires. Compare with other wire mesh. [HuangX00a] Huang, X.; Acero, A.; Chelba, C.; Deng, L.; Duchene, D.; Goodman, J.; Hon, H.; Jacoby, D.; Jian, L.; Loynd, R.; Mahajan, M.; Mau, P.; Meredith, S.; Mughal, S.; Neto, S.; Plumpe, M.; Wang, K.; and Wang Y. "Scalable Coating and Properties of Transparent Flexible Silver Nanowire Electrodes", Proc. Int'l. Conf on Spoken Language Processing, Vol 3 pp. III-33-36, ISCA, Beijing, 2000 Prototype PDA with voice/speech recognition, touchscreen, and thumb roller user interface (modified Windows CE). Dr Who speech recognition engine. Speech recognition to completely point/tap touchscreen interface (Tap and Talk) pen touch selects objects and context, start/stop microphone, speech for text entry. [IBM00a] IBM "ThinkPad TransNote Digital Pen User's Guide digitalpen", IBM, November 2000 digital pen / stylus for IBM ThinkPad TransNote laptop with ThinkScribe digital notepad tablet. Says little more than to read the on-computer Help files. [IDEO99a] Ideo, Inc. "IDEO Eye Module User's Manual for Palm Pilot", Ideo, Inc. 1999 Third-party camera hardware module for Palm PDAs, plugs into expansion slot, PDA becomes a hand-held camera with display screen. [IDial00a] iDial Networks, Inc. "iDial Announces First Wireless Phone to Phone Access Service for Palm VII", Business Wire, March 27, 2000 VOIP internet network phone service on Palm VII using NetPhoneCard [IEEE00a] Institute of Electrial and Electronics Engineers "IEEE 100: The Authoritative Dictionary of IEEE Standards Terms, 7th Edition", Standards Information Network, IEEE Press, 2000 (hardcopy book) Comprehensive electronics and computer dictionary, IEEE, 2000. [IgarashiT99a] Igarashi, Takeo; Matsuoka, Satoshi; and Tanaka, Hidehiko "Teddy: A sketching interface for 3d freeform design", Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, pages 409-416, August 1999 Sketch recognition, drawing program converts rough lines to splined/mesh during drawing, for sketching 3D three dimensional images such as cartoons. Cites to Zeleznik, Landay. Cited in Arvo00 [IgarashiT99b] Igarashi, Takeo "Freeform User Interfaces for Graphical Computing", Doctoral dissertation, Graduate School of Information Engineering, University of Tokyo, December 1999 Sketch-based editing of freehand drawing: dragging and pulling operations on lines and line segments: stroke-based architecture for electronic whiteboards. Refers to "predictive drawing" for sketch editing/editor. Cites Unistroke character recognition, Cirrin and Quickwriting zone-based menu picking, mentions learning problem for gestures (see Greanias?) [Immersion00a] Immersion Inc. "MicroScribe 3D Desktop Digitizing Systems: User's Guide and Set-up Instructions", Immersion Corporation, San Jose, CA, 2000 Pantographic/mechanical 3D digitizer system. Primarily used for digitizing solid objects, medical modelling: no force feedback/haptic output [Intermec00a] Intermec Technologies Corporation "6642 Pen Computer Technical Reference", Intermec Technologies Corporation, Cedar Rapids Indiana, 2000 Pen computer using Windows 95 for Pen Computing / PenWindows 2.0: Pen Services 2.0, running on Windows 98 (later than Windows 95). "Lens" field for enabling text buttons in input fields. Adjustable area/distance for double-tap distance. Pen Recognition services include also CIC PenX Recognition. Shows gesture macro editor for assigning macros to PenWindows (1.0?) circle gestures consisting of circled letters: compare to Recognition macros of Pencept PenPad series. Toolbar option for right mouse click:



compare to Wang "Guide". PenX include Recognition Tuner to allow user to select which supported shapes appear in user's writing style, but not to train recognizer. Compare to recognizer in PalmOS for selection of shape prototypes. ActiveX Pen Controls, PenWindows 2.0 pop-up virtual keyboard from Start/Programs/Pen Services. Document is in two portions. [Intermec00b] Intermec Technologies Corporation "PenKey 6200 6300 Hand-Held Computer Programmer's Reference Guide", Intermec Technologies Corporation, Cedar Rapids Indiana, 2000 Pen computer using Windows 95 for Pen Computing / PenWindows 2.0. Includes definitions of SYSTEM.INI and WINDOWS.INI for Windows 3.1 to enable Windows for Pen Computing: pkrotate.drv [IrvineCE99a] Irvine, Cynthia E. "The Reference Monitor Concept as a Unifying Principle in Computer Security Education", Naval Postgraduate School, Monterey CA, June 1999 Report on graduate-level course program on computer security: reference monitor as example -- all accesses protected, access mediation tamperproof, subject to mathematical proof of correctness. Note: military students do not participate in design. [JaegerS99a] Jaeger, Stefan "Recovering Dynamic Information from Static, Handwritten Word Images", Abstract only: PhD Thesis, http://isl.ira.uka.de/ISL.publications.html#phd Describes attempt to re-construct dynamic information of on-line handwriting recognition from off-line static images: mentions Travelling Salesman Problem as part of algorithm. Abstract says this would allow merging of recognition methods between off-line and on-line algorithms, but does not mention any results Full text of thesis (at end) says they do not deal with pen-lifts or segmentation, assume one character at a time [JaegerS00a] Jaeger, S., Manke, S, and Waibel, A. "NPEN++: An On-Line Handwriting Recognition System", 7th International Workshop on Frontiers in Handwriting Recognition, Amsterdam 2000, page 249-260 handwriting recognition: Mentions normalizing size, inclination, and rotation; computing baselines; also refers to general problem of resampling to make points equidistant in space, and Bezier interpolation of "missing" points; Note: removes delayed strokes (e.g. crossing of a "T"). Compare with elastic matching by Tappert? [JaegerS00b] Jaeger, S., Manke, S, Reichert, J., and Waibel, A. "On-Line Handwriting Recognition: The NPen++ Recognizer", International Journal on Document Analysis and Recognition, IJDAR'00, volume 3 No 3, p. 169-180, 2000 Handwriting recognition using very large word dictionary: claims 91.2% recognition rate using 50000 word dictionary: tree-pruning to avoid exhaustive search [JannetteDA00a] Jannette, Daniel A.; Allen, Edwin M; Burnard, Mark F.; Crenshaw, Jamie L.; DeSaele, Curtis R.; Hill, Michael E.; Morrison, Gerald O.; Raheja, Sonia; Szuch, Willam G.; Vickers, Paul W; and Zaun, Mark S. "Design and Engineering Management System", United States Patent 6,036,345, March 14, 2000 Project management and organization system [JensenD99a] Jensen, Del "Textured tile Rotation System and Method", United States Patent 5,956,043, September 21, 1999 Algorithm for rotating (tiled) texture pattern when an object is rotated, so that there are no breaks in the texture when you try to do it with horizontal tiling of a supertile in the rotated image/object. [JohnsonNF99a] Johnson, Neil F. and Jajodia, Sushil "Exploring Steganography: Seeing the Unseen", IEEE Computer, February 1998, pp. 26..34 Overview of Steganography, including historical. Modifying bitmaps, color palettes on GIF image (S-Tools, StegoDos Black Wolf's Picture Encoder v0.90B, Jpeg-Jsteg V4, PictureMarc, SureSign). Frequency modification in audio files. StegoKeys, Stegomessage. [JosuttisNM99a] Josuttis, Nicolai M. "The C++ Standard Library -- A Tutorial and Reference", Addison-Wesley, 1999 (hardcopy book) C++ Standard template library, 1999. auto_ptr, reference counting. Cites to Boost C++ template library. [KamelHM00a] Kamel, Hesham M. and Landay, James A. "A Study of Blind Drawing Practice: Creating Graphical Information Without the Visual Channel", Proc. ASSETS 2000, Arlington VA pp. 34-41 Visual impaired / low vision / blind can draw, but need tactile feedback to do it in a computer user interface. Grid-based model to provide audio (speech) feedback on position and features. [KangBH99a] Kang, Beng Hong; Yeung, Chun Wun; Tedja, Eswin; and Tan, Jin-Meng "Dual Orientation display Handheld Computer Devices", United States Patent 5,949,408, September 7, 1999 HP OmniGo HP95a: Screen image rotation (portrait/landscape orientation) in a handheld PDA by interchanging row and column lines. [KangBHA00a] Kang, Beng Hong Alex; and Chung, Sun-Woo "Method for providing a cue in a computer system", United States Patent 6,160,555, December 12, 2000 Handwriting recognition user interface: show different cursors (alpha/num/upper-case/etc.) in text entry before user writes, so that user only need to look at one place to see both mode and text cursor position. Note: separate writing and text area, not direct text input / direct manipulation.



[KanoS99a] Kano, Shigeru; Nakada, Yasuhiro; and Usuda, Yutaka "Automated recognition of and distinction among graphics input text input and editing commands in a pen based computer", United States Patent 6,913,221, June 15, 1999 Gesture/sketch/text pen-computer user interface on graphics tablet, automatically classify recognition based on priorities of recognizers. Compare with W4PC? [KatsurahiraY99a] Katsurahira, Yuji and Sekizawa, Yasushi "Position Detecting Apparatus and Method for Detecting Position Pointing Device", United States Patent 6,005,555, December 21, 1999 Electromagnetic grid digitizer tablet, signal from tablet causes multiple pens or objects to resonate at different frequencies, simultaneous detection of distinct pointing devices (multi-touch) [KavehC99a] Kaveh, Chris "Fujitsu Pen Developer's Guide for Windows", Fujitsu Personal Systems Inc., May 1999 Programming guide for Pen computing on Windows platforms: post-Pen-Windows. PenRight! controls, Visual Basic 4.0 and 5.0, PowerSoft Powerbuilder, VBX Pen Edit controls, (16-bit and 32-bit), 32-Bit Pen API, Pen-aware and pen-unaware applications, CIC InkTools for handwriting recognition for Windows. Microsoft Pen Extensions 1.0 for Windows 3.1, Microsoft Pen Services 2.0 for Windows95, CIC PenX for Windows NT. [KawakamiM99a] Kawakami, Masahiko; Mamiya, Masaru; Nishiki, Tomonori; Tsuii, Yoshitaka; Okamoto, Akito; Fuita, Toshihiro "A New Concept Touch-Sensitive Display Enabling Vibro-Tactile Feedback", Proc. HCI '98 on People and Computers XIII, pp. 303-312 Vibration Touch: tactile haptic feedback, both click and vibration. Different vibration patterns for different operations (vibrate, one-shot click, pull-down slow click) via solenoid mount in touchscreen. [KawashimaS99a] Kawashima, Shosaku "Digital camera with cellular phone and portable computer", United States Design Patent 405457, Feburary 9, 1999 Cellphone/smartphone with digital camera and portable computer: compare with Palm telephone with photo attachment? Video? [KayA00] Kay, Alan "Character recognition", Posting to SQUEAK Mailing List November 5 2005, archived at http://lists.squeakfoundation.org Commentary on Goldberg Unistroke patent case: asserts that much of handwriting recognition technology prior art was done in 1960's, not known to U.S. Patent Office. Cites GRAIL system by Groner as Unistroke recognition: mentions QuikWriting and Ledeen recognizer [KentJ99a] Kent, Joel "Acoustic condition sensor employing a plurality of mutually non-orthogonal waves", United States Patent 5,986,224, November 16, 1999 Acoustic surface (shear?) wave touchscreen, waves are sent diagonally or along non-orthogonal axes, e.g. three 60-degree axes. May use angled reflectors in edge transducers or sensors. Thus can discriminate multiple-touches both for ambiguity, or reliable multi-touch input. [KimH99a] Kim, H.; Gilmore, C.M.; Pique, A.; Horwitz, J.S.; Mattoussi, H.; Murata, H.; Kafafi, Z.H.; and Chrisey, D.B. "Electrical, optical, and structural properties of indium-tin-oxide (ITO) thin films for organic light-emitting devices", Journal of Applied Physics, Vol 86, No 11, 1 Dec. 1999, pp. 6451-6461 ITO deposited on glass by vapor deposition, controlling for electrical and optical properties: optical transmission, reflectance, refractive index, band gap (color/tint). [KinrotO00a] Kinrot, Opher; Kinrot, Uri "Interferometry: Encoder measures motion through interferometry", Laser Focus World, March 2000 Optical Translation Measurement, motion detection, basis for OTM Technologies optical digitizer pen, development by GOU Lite Ltd for coherent encodings for one-two- or three-dimensional motion (stepper motor? servomotor? See Teledyne/TAC file) [KinrotO00b] Kinrot, Opher "Polarization Sensitive Detector", Internation Patent Publication WO/00/52763 A1, September 8, 2000 Light sensor for determining polarization using two stacked sensors with orthogonal polarization. Used in OTM/VPen? [KleinDA00a] Klein, Dean A. "Method and apparatus for selectively displaying a parameter in a separate status panel", United States Patent 6,014,141, January 11, 2000 Use one scroll/level bar to show level of whatever parameter is being adjusted: e.g. if adjusting volume (via icon), shows volume level, if checking battery (icon), shows battery level, contrast adjustment on laptop from keyboard. [KnowltonKC99a] Knowlton, Kenneth Charles and Miliefsky, Gary Steven "System for the Capture and Indexing of Graphical Representations of Files, Information Sources and the Like", United States Patent 5,973,692, October 26, 1999 User-interface linking of graphical areas on one display to graphical areas in another display [KobayashiF99a] Kobayashi, Fumiyuki "Alteration of the orientation of data on a display / Portable terminal device capable of controlling direction of screen display on display", GB Patent GB2331204A, Australian Patent Application AU 199891322B2, 1999



Handheld device with touchscreen or smartphone rotates display landscape/portrait based on pressure sensors indicating how it is being held in the hand. [KohtakeN99a] Kohtake, Naohiko; Rekimoto, Jun; and Anzai, Yuichiro "InfoStick: an interaction device for InterAppliance Computing", Proc. HUC '99 Int'l Symp. on Handheld and Ubiquitous Computing, 1999, pp. 246..258 Prototype small information device about the size of a large USB stick, small display and three-button user interface. Optical camera for 2D barcode/matrix code. Pick up information / deposit information / selectmenu. User points at object (digital slide display, web page or document on PC, post-it- note with digital pattern) to transfer to laptop, printer, etc. For larger display of information than fit on small screen, plug into base station. Can be used as pointing device on LDC display. [KoikeH00a] Koike, Hideki; Sato, Yoichi; Kobayashi, Yshinori; Tobia, Hiroaki; Kobayashi, Motoki "Interactive Textbook and Interactive Venn Diagram: Natural and Intuitive Interfaces on Augmented Desk System", Proc. CHI '2000, Den Haag, Amsterdam, pp. 121-128 Optical tablet sensing for digital desk: describes template matching for detecting multiple fingertips and hand for multitouch. [KortuemG99a] Kortuem, Gerd; Bauer, Martin; and Segall, Zary "NETMAN: The design of a collaborative wearable computer system", Mobile Networks and Applications, Vol 4, pp. 49-58, 1999 Wearable computer with video conference communication between field technicians, with speaker phone, or field technician and remote advisor/expert. [KrausePR00a] Krause, Philip R. "User interface for dynamic presentation of text with a variable speed based on a cursor location in relation to a neutral deceleration and acceleration zone", United States Patent 6,067,069, May 23, 2000 scroll text to match user's reading speed, using head/eye position. Inclused acceleration and stop zones. [KrivopalB99a] Krivopal, Boris "Pressure Sensitive Ink Means and Methods of Use", United States Patent 5,989,700, November 23, 1999 Pressure-sensitive conductive rubber: semiconductor particles in elastic, organic polymer binder: assigned to Tekscan [KrugerJ99] Kruger, Justin "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments", Journal of Personality and Social Psychology, December 1999 Vol. 77 No 6, pp 1121-1134 Self-evaluation of success/skill: subjects who know more or are more skilled, recognize their own performance more conservatively than low-skilled subjects. To paraphrase: a real expert will be more cautious in claiming to know exactly what is going on, because the real expert knows about the additional factors that may come into play, and what complex interactions can happen. Whereas a person who knows less will be more likely think they have it all figured out, because that person does not know about the other things they should include in their thinking, or that there might be additional ways they could interact. [Kurtenbach99a] Kurtenbach, Gordon P. "Display and Control of Menus with Radial and Linear Portions", United States Patent 5,926,179, July 20, 1999 Marking menus: pen-still (press-and-hold) gesture brings up menu. Flick gesture involves command without pop-up menu. Detects press-and-hold with movement and time-out thresholds. [Kurtenbach99b] Kurtenbach, Gordon; Fitzmaurice, George W.; Owen, Russel N.; Baudel, Thomas "The Hotbox: Efficient Access to a large Number of Menu-items", Proc. CHI '97, Pittsburgh USA, pp 231-239 Large menus by combining menubar, accelerator keys, modal dialog, pop-up menus, radial/pie marking menus, and menubars [LCSTelegraphics00] LCS/Telegraphics "Wintab Advanced Pointing Device Management for Windowed Environments", LCS/Telegraphics, Cambridge Massachusetts, July 28, 2000 Background information on WinTab specification, originated at 1991 Autodesk conference to define vendorneutral device driver specification for digitizing tablets [LaGrangeGW99a] La Grange, George W.; Gerpheide, George E.; Woolley, Richard D.; Donohue, Tom; and Layton, Mike "Computer input stylus method and apparatus", United States Patent 5,914,708, June 22, 1999 Passive stylus for capacitive touchscreen using transcapacitance: conductive foam on tip of stylus increases capacitive effect by deforming to cover more area. [LallicanPM00a] Lallican, P.M.; Viard-Gaudin, C.; and Knerr, S. "From Off-Line to On-Line HandWriting Recognition", International Workshop on Frontiers in Handwriting Recognition, IWFHR2000, September 2000, Amsterdam, Netherlands, pp 303-312 On-line works better than off-line, so simulate stroke order (and direction of strokes?) to recover timeorder of the handwriting. See also VisionObjects commercial development. [LambertB00a] Lambert, Brian "System and Method for changing the rotational orientation of display data", International Patent Application WO 00/68766, November 16, 2000 On laptop or touchscreen computer, rotate windows of a given application for landscape/portrait



orientation, but other applications can stay in the orientation best suited for that particular application. [LandonP00a] Landon, P.; Keates, S.; Clarkson, P.J.; and Robinson, P. "Using haptic feedback to enhance computer interaction for motion-impaired users", Proc. 3rd Int'l Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000, pp. 25-32 For motion-handicapped users (cerebral palsy), vibrotactile/vibrating haptic feedback not helpful, but forcefeedback feedback is. Feedback prototypes use Immersion Technologies and SensAble Technologies devices. [Lang99] Lang "GO Alumni Directory", www.apocalypse.org (no longer available) (partial copy) List of Alumni from GO Corporation: original list no longer available [LarmouthJ99a] Larmouth, Jojhn "ASN.1 Complete", Open Systems Solutions, May 31, 1999 Reference on ASN.1 Abstract Syntax Notation One encoding (1984), ASCII syntax representation for arbitrary data, XER standard (XML Encoding Rules); binary and extended forms. [LautzenheiserTG00a] Lautzenheiser, Ted. G. and Lacy, David R. "Computer based system and method for modelling activities of people in an organization", United States Patent 6,023,572, February 8, 2000 Project management and organization system [LeganchukA99a] Leganchuck, Andrea; Zhai, Shumin; and Buxton, William "Manual and Cognitive Benefits of Two-Handed Input: An Experimental Study", ACM Trans. on Computer-Human Interaction, Vol 5, No 4, December, 1998, pp. 326-359 Multiple-hand / multiple-touch user interface study: using two hands allows simultaneous manipulation of scale and zoom and position, versus one hand with menu/tool selection. WACOM two-stylus (puck and stylus) tablet used in study prototype. [LevinG99a] Levin, Golan and Yarin, Paul "Bringing Sketching Tools to Keychain Computers with an AccelerationBased Interface", Proc. CHI '99, May 1999, pp. 268-269 "shakepad": Accellerometer in keychain fob, 2D drawing program. Refers to keychain such as Tamagotchi computers distinct from PDAs with touchscreens. Compare with accelerometers in PDAs (e.g. iPhone), WII hand-held input device. [LewisC99a] Lewis, Chris "Cisco TCP/IP Routing Professional Reference, Second Edition", McGraw-Hill, 1999 (hardcopy book) TCP/IP reference for Cisco products. Includes internetworking with IPv6 [LewisJR99a] Lewis, James R.; Allard, David J.; and Hudson, Harold D. "Predictive Keyboard Design Study: Effects of Word Populations, Number of Displayed Letters, and Number of Transitional Probability Tables", Proc. of Human Factors and Ergonomics Society 43rd Annual Meeting, 1999, pp 429-432 Study of virtual predictive keyboard of IBM Simon marketed by BellSouth: text used for vocabulary/probability tables should be adaptive, or at least should reflect vocabulary of user application. [LewisJR99b] Lewis, James R. "Input Rates and User Preference for Three Small-screen Input Methods: Standard Keyboard, Predictive Keyboard, and Handwriting", Proc. of Human Factors and Ergonomics Society 43rd Annual Meeting, 1999, pp 425-428 Study of virtual predictive keyboard of IBM Simon marketed by BellSouth: showed that standard keyboard, even small, was faster than either predictive keyboard or handwriting. Cites that fast handwriting is about 25 wpm words per minute. Did not study graffiti/unistrokes or other later methods, but disadvantages were attributed to cognitive load of unfamiliar/unpracticed data input skill compared to standard keyboard. [LewisJR99c] Lewis, James R.; Kennedy, Peter J.; and LaLomia, Mary J. "Development of a digram-based typing key layout for single-finger/stylus input", Proc. of Human Factors and Ergonomics Society 43rd Annual Meeting, 1999, pp 415-419 5x5 keyboard layout for touchscreen virtual keyboard: keys not alphabetical, positioned for closeness based on di-gram frequency in English text. [LiY00a] Li, Yang; Guan, Zhiwei; Chen, Youdi; Dai, Guozhong "PenBuilder: Platform for the Development of PenBased User Interface", Proc. ICMI 00 Third Int'. Conf. on Advances in Multimodal Interfaces, pp 534..541 (partial copy) Pen-Book-Page-Paper stylus UI system for mobile systems, with universal gestures: research prototype in China. Cites Wintab. [LifchitzA00a] Lifchitz, Alain and Maire, Frederic "A Fast Lexically Constrained Viterbi Algorithm for On-line Handwriting Recognition", 7th International Workshop on Frontiers in Handwriting Recognition, Amsterdam 2000, page 313-322 Lexical constraint / dictionary for handwriting recognition context: uses directed acyclic word graph DAWG instead of trie tree-structure [LinJ99a] Lin, James; Newman, Mark; Hong, Jason I; Landay, James A. "DENIM: Finding a Tight Fit Between Tools and Practice for Web Site Design", UC Berkeley Technical Report UCB//CSD-99-1065, September 1999 Sketch editing with limited gestures, uses barrel button on stylus to change mode/function: compare to Recognition Macros of Pencept with AutoCad and other applications



[LongAC99a] Long, Allan Christian Jr.; Landay, James A.; and Rowe, Lawrence A. "Implications For a Gesture Design Tool", Proc. CHI '99, Pittsburgh PA USA, pp. 40-47 Study of satisfaction of PDA users (Apple Newton, PalmPilot) on gestures: users preferred gestures as user interface on touch screen, want to define own gestures by example, not tolerant of recognition errors. Test recognizer uses Rubine algorithm. [LouderbackJ99a] Louderback, Jim "Fujitsu PenCentra 130", ZDNet Publications: www.zdnet.com/zdtv/freshgear, May 21, 1999 Fujitsu rubberized/ruggedized pen computer running WindowsCE. Mentions pen tablets (portable handwriting computers) running Windows NT. [LouderbackJ99b] Louderback, Jim "Qubit", ZDNet Publications: www.zdnet.com/zdtv/freshgear, March 26, 1999 Internet Access Device stand-alone touchscreen/tablet computer for accessing the internet. [MacKenzieIS99a] MacKenzie, I. Scott and Chang, Larry "A performance comparison of two handwriting recognizers", Proc. Interacting with Computers 11, 1999 32-subject comparison of Microsoft (Pen Windows 2.0?) and CIC Handwriter 3.3 handwriting recognizers. Also mentions Unistroke, Graffiti (original, not CIC) and T-Cube unistroke handwriting recognition: T-Cube from Apple uses pie menus. Zhao incremental recognition. Interface characteristics (subsetting, context) have significant effect on performance. [MacKenzieIS99b] MacKenzie, I. Scott and Zhang, Shawn X. "The Design and Evaluation of a High-Performance Soft Keyboard", Proc. CHI 99, Pittsburgh PA USA, 15-20 May 1999, pp. 25-31 Study of soft/virtual keyboard layout: QWERTY keyboard better than "optimal" keyboard layout at first (familiarity?), optimal keyboard only slightly better over time. Testing on Windows for Pen Computing / Pen Windows, WACOM integrated touchscreen and display. Also mentions Unistroke, Graffiti (original, not CIC) and T-Cube unistroke handwriting recognition: T-Cube from Apple uses pie menus. Zhao incremental recognition. Interface characteristics (subsetting, context) have significant effect on performance. [MacLeanKE99a] MakLean, Karon E. and Roderick, Jayne B. "Smart Tangible Displays in the Everyday World: A Haptic Door Knob", Proc. IEEE/ASME Intl. Conf. on Advanced Intelligent Mechatronics AIM'99, Atlanta, Sep. 1999, 6 pages Haptic feedback in doorknob (everyday object): force, motion, heat/temperature, sound. Speculative uses include gesture motion on doorknob as an input password, whether inside lights are on/off, whether someone is/has been in room. [MachroneB00a] Machrone, Bill "Wacom's LCD Pen Tablet: Nearly Perfect, Pricey", PC Magazine, November 1, 2000 Integrated tablet and display touchscreen from Wacom PL500, pressure sensitive (actually force-sensitive) stylus, mis-called pressure sensitive tablet, price about $3000 [MakofskyS99a] Makofsky, Steve; Nottingham, Jason; and Tucker, Andrew "Teach Yourself Windows CE Programming in 24 Hours", Sams Publishing, 1999 (hardcopy book) Windows CE, small touchscreen hand-held computer/PDA [MankoffJ00a] Mankoff, Jennifer; Abowd, Gregory D.; and Hudson, Scott E. "OOPS: A toolkit supporting mediation techniques for resolving ambiguity in recognition-based interfaces", Computers and Graphics, Vol 24 No 6, 2000, pp 819-834 Toolkit for using context or providing additional UI features to handle ambiguous recognition or misclassification, including interactively in a graphical user interface: gestures, sketch recognition, annotation/recognition, sketches of UI elements (buttons, checkboxes, etc.). (mediation). Cites to soft keyboard on Newton MessagePad, Microsoft Pen for Windows (sic: Windows fro Pen Computing), Undo, "Put-that-there". [MargolisPE99a] Margolis, Philip E. "Computer and Internet Dictionary, Third Edition", Random House/Webster's, 1999 (hardcopy book) Technical dictionary 1999 [MarianettiR99a] Marianetti, Ronald II and Haitani, Robert Yuji "Method and apparatus for immediate response handwriting recognition system that handles multiple character sets", United States Patent 6,889,888, March 30, 1999 Palm Pilot user interface: separate handwriting areas for alphabetic and for numeric/arithmetic character recognition, or other multiple character sets. [MartinD00] Martin, David A. "Projection Display System with Touch Sensing on Screen, Computer Assisted Alignment Correction and Network Conferencing", United States Patent 6,141,000 Whiteboard system using a projection display with a touch-screen digitizer. Multiple pens/stylus, eraser. [MartinelliHK99a] Martinelli, John K.; Martinelli, Robert M; Pickett, Mark C.; Ritchey, Wendell W.; and Yaniger, Stuart J. "Force Sensing Semiconductive Touchpad", United States Patent 5,943,044, August 24, 1999 Tap, Tap-Drag and other gestures for emulating mouse button gesture (click-and-drag, etc.). Touchpad using two sheets and a resistance-changing material between. Critiques indirect measure of force/pressure



(capacitive area) vs. direct measurements of force in a digitizer/touchpad for pressure/force input for scrolling and acceleration. [MartinezEA00a] Martinez, Anthony Edward and Martinez, Sarka "Method and apparatus for altering a display in response to changes in attitude relative to a plane", United States Patent 6,137,468, October 24, 2000 Automatically rotate all windows and objects in a portable computer screen when computer is rotated, including particularly rotated. Uses accelerometer or level sensor. Includes rotation of all windows. [MassonT99a] Masson, Terrence "History of Computer Graphics (CG)", www.visualfx.com/milestones.htm Ken Knowlton 1950s. Primarily generation of computer graphics, mentions SketchPad with light pen, but not Stylator or Rand Tablet. Computer animation and CGI. [MasuiT99a] Masui, Toshiyuki "Text input device and method", United States Patent 6,002,390, December 14, 1999 Predictive keyboard: predicted words for auto completion are taken from a dictionary and displayed. [MatsushitaN00a] Matsushita, Nbuyuki; Ayatsuka, Yuji; and Rekimoto, Jun "Dual Touch: A Two-Handed Interface for Pen-Based PDAs", Proc. UIST '00, San Francisco, pp. 211-212 Resistive/capacitive touchscreen detecting multi-touches by shift in position. First touch (thumb) is fixed, sets one point, other touch (stylus) is other point and can be moved, for rotation and other gestures. [McCartyB99a] McCarty, Bill "Learning Red Hat Linux", O'Reilly, 1999 (hardcopy book) Red Hat Linux command line, user interface user guide [McCrickard99] McCrickard, D. Scott and Catrambone, Richard "Beyond the Scrollbar: An Evolution and valuation of Alternative List Navigation Techniques", IEEE Symposium on Visual Language, 1999, pp. 270-ff mural bar and pile bar: information about entries in lists are encoded with highlighting marks into trough/slider of a scrollbar, compare with Russian patent by Gorbunov [McGeeMR99a] McGee, Marilyn Rose "A haptically enhanced scrollbar: force feedback as a means of reducing the problems associated with scrolling", Proceedings for the First PHANToM Users Research Symposium (PURS'99), Heidelberg, Germany, pp. 17-20 Study of force feedback in cursor control / scrollbar: hardware was PHANToM by SensAble Technologies, mechanical actuators to pen-like or thimble-like user input device. [McGill99a] McGill University "A Two DOF Planar Haptic Interface: The Pantograph", McGill Univeristy, Canada, Annual report 1999 Note on 2D electromechanical pointing device with force feedback: Design dates to 1994, Commercial products are mouseCAT and PenCat/Pro from Haptic Technologies, Inc. [McKielFA00a] McKiel, Frank Albert Jr. "Method and system for enabling blind or visually impaired computer users to graphically select displayed elements", United States Patent 6,046,722, April 4, 2000 Accessibility for visually impaired/blind for touchscreen or mouse: auditory feedback of position, and graphical elements in quadrants of a specified radius around the cursor. Uses stereo/binaural sound. [MessmerHP99a] Messmer, Hans-Peter "The Indispensable PC Hardware Book: Your Hardware Questions Answered, third edition", Addison-Wesley, 1999 General hardware overview of PC: DOS, OS/2, Windows NT. Mentions mouse as oldest kind of pointing device, and equivalence of pointing devices (joystick, trackball, etc.) [MicroTouch99a] MicroTouch Systems Inc. "TouchWare for Windows User's Guide", MicroTouch Systems, Inc. Methuen Mass First copyright is 1997 [MicroTouch00a] MicroTouch Systems Inc. "MicroTouch ThruGlass Projected Capacitance Touchscreens", MicroTouch Systems, 2000 Electrostatic/capacitive touchscreen: states accuracy as "97%" (??), 190 points/second data rate. [Microsoft99a] Microsoft Corporation "Description of Windows for Pen Computing Pens", Microsoft Knowledge Base, 10/13/1999, KB85663 Pen/tablet drivers use special interface for higher data rate than mouse driver, special display driver for displaying electronic ink [Microsoft99b] Microsoft Corporation "Microsoft Computer Dictionary, Fourth Edition", Microsoft Press, 1999 General Reference [Microsoft99c] Microsoft "penwin.h Pen Windows functions, types, and definitions Version 2.0", Microsoft Corporation, 1999 #include definitions file for Windows for Pen Computing Version 2.0 -- 1996. CompactPenData tablet data information includes pressure (force), height (proximity), angle x/Y, angle Z, barrel rotation, compress or uncompressed. Gestures include select, clip (cut/copy/paste), whitespace (space, tab, return), edit/autocorrect, upper and lower case circle: different definitions for Japan. Further gestures select first/left/right/last, clear, help, insert context menu ("M"), undo ("U"), cut/copy/paste ("x","c","p", circle a-z and circle A-Z gesture See also penwin.h for version 2.0 [Microsoft99d] Microsoft "Microsoft Windows CE User Interface Services Guide: The ultimate reference and toolkit



for Windows CE", Microsoft Corporation, 1999 (hardcopy book) Windows CE programming and user interface guide. [Microsoft99e] Microsoft "Microsoft Computer Dictionary, Fourth Edition", Microsoft Corporation, 1999 (hardcopy book) Technical dictionary [Microsoft00a] Microsoft "Microsoft Tablet PC Announcement, Comdex 2000 Keynote Address by Bill Gates", Jonas Thulin, Youtube.com, posted in 2011 Video from audience of Tablet PC product announcement at Comdex 2000. Features shown are electronic ink and note-taking, no demonstration of gestures, handwriting recognition. Shows Tablet PC propped up on a table with small stereo speakers attached. [Microsoft00b] Microsoft "Microsoft MSDN Library, October 2000", Microsoft MSDN, October 2000, CD distribution of MSDN references (physical device) Distribution CD with knowledge base, other technical documentation for Windows systems (Windows 3.1, 95, NT, etc.), October 2000 [MillerJ99] Miller, James R.; Bonura, Thomas; Nardi, Bonnie; and Wright, David "System and Method for Performing an Action on a Structure in Computer-Generated Data", United States Patent 5,946,647, August 31, 1999 Scan displayed text with pattern matching to find phone numbers, addresses, etc., to highlight them and turn them into active links or buttons, clickable with a stylus. [MobilePoint99a] MobilePoint Corporation "Re-Inventing Face-to-Face Selling", MobilePoint.com, archive 1999 Forms-based pen-computing application for field sales [MohantySP99a] Mohanty, Saraju P. "Digital Watermarking : A Tutorial Review", Report, Indian Institute of Science, Bangalore 1999. Fetched 2014 from informatika.stei.itb.ac.id/~rinaldi.munir/Kriptografi/WMSurvey1999Mohanty.pdf Overfiew of digital watermaking and steganography as subset of information hiding, steganography vs. cryptography. Examples from digital images, not audio, multimedia, etc. Historical citations to Herodotus, Iliad, Indian literature. [MondenA00a] Monden, Akito; Iida, Hajimu; Matsumoto, Ken-ichi; Inoue, Katsuro; and Torri, Koji "A Practical Method for Watermarking Java Programs", 24th Comp. Softw and App. Conf. (compsac2000), Taipei Taiwan, Oct 2000 Watermarking / hidden signature in Java code by adding a dummy method, and recoding opcodes for arithmetic operations to represent the signature data. Cites to code obfuscators, method is robust when obfuscated. [MoranTP99a] Moran, Thomas P.; Pedersen, Elin R.; McCall, Michael K; and Halasz, Frank G. "Time-Space Object Containment for Graphical User Interface", United States Patent 5,404,439, April 4, 1995 When selecting objects with a drag rectangle, take into account temporal/time when objects were created by user -- user mostly likely wants to select objects created together in time. Uses threshold time to determine direction of reversing motion in a "wipe" (swipe?) gesture. [MufticS99a] Muftic, Sead "Smart token system for secure electronic transactions and identification", United States Patent 5,943,423, August 24, 1999 Logging on to a computer system for access using a smart card cryptographic token and a public certificate for the identity of the user [MunichME00a] Munich, Mario Enrique "Visual Input for Pen-Based Computers", Ph.D. Thesis, California Institute of Technology, January 21, 2000 Optical/Vision based pen-computing tablet, pen tracking by watching position of pointed stylus, pen/stylus up/down detection by checking for ink on paper, and for coincidence of shadow with object. Signature verification. Disambiguation of stylus tip from finger based on sharpness of end. Electronic file includes powerpoint slide of thesis defense [MyersB00] Myers, Brad; Hudson, Scott E.; and Pausch, Randy "Past, Present and Future of User Interface Software Tools", ACM Trans. on Computer-Human Interaction, Vol 7 No 1, March 2000, pp 3-28 Future UIs will employ recognition-based UIs (gestures?); dissimilarity of mouse-based vs. stylus vs. gesture UIs not equivalent [NagasakiK99a] Nagasaki, Katsuhiko; Shimada, Kazutoshi; Tatsumi, Eisaku; Suzuki, Norikyuki; and Sunakawa, Shinichi "Information Processor", United States Patent 5,936,619, August 10, 1999 Portable computer (with touchscreen and stylus): rotate layout of screen as device is rotated, for landscape/portrait. Shows layout being re-generated automatically (text flow). Physical orientation determined from how display is connected. [NakamuraS99a] Nakamura, Satoshi; Tsukamoto, Masahiko; and Nishio, Shojiro "Design and Implementation of Operating Mechanisms for Window System with Two Mice (in Japanese)", Information Processing Society of Japan SIG Notes, Vol 99 No 35, pp. 1..7, May 14, 1999.



Experimental system for bi-manual/two-handed (multi-touch) user interface in windowing system using two mice (relative position) [Netscape99] University of North Texas "Netscape Navigator 4.x and the UNT Home Page", University of North Texas, fetched from www.unt.edu/training/UNTHomePage/UNTHomePage.PDF Description of Netscape 4.x user interface, including back icon with press gesture: press-and-hold. [NewmanWM99a] Newman, William M.; Dance, Christopher R.; Tayloer, Alex S.; Tayloer, Stuart A.; Tayloer, Michael; Aldhous, tony "Camworks: A video-based Tool for Efficient Capture from Paper Source Documents", Proc. ICMCS99 IEEE Int'l. Conf. on Multimedia and Computing Systems '99, Florence, Italy, Vol 2, pp 647-653 Video camera for live scanning and capture of paper documents while user working on electronic document. Cites to Wellner, others. Shows algorithm for skew/tilt correction using horizontal projection profiles. [NicholsRK99a] Nichols, Randall K. "ICSA Guide to Cryptography", ICSA International Computer Security Association, 1999 (hardcopy book) General reference on cryptography/PKI practice [Nokia99a] Nokia "Nokia 9110 User's Manual: Nokia 9110 Communicator", Nokia, 1999 Nokia 9110 Communicator smartphone: cellphone in outer cover, opens landscape-clamshell to show keyboard and display. Calendar, Document editing, Fax, Contact address book, Email, WWW browser. No mouse/touchscreen, keyboard navigation only. [OguraT99a] Ogrua, Tsuyoshi and Itoh, Akihisa "Apparatus and Method Regarding Drag Locking with Notification.", United States Patent 5,907,327, May 25, 1999 Slide to unlock via tap-and-drag drag-lock gesture. [OkamotoM00a] Okamoto, Masayoshi; Ieda, Kazuhiro; and Horii, Hiroshi "Handwritten character input device allowing input of handwritten characters to arbitrary application program", United States Patent 6,088,481, July 11, 2000 Keyboard handwriting input to arbitrary programs, using transparent overlay window for input. Cites to Wang "Guide", Slate PenApps. Compare with transparent overlay inking window in Pen Windows? [OmniVision00a] OmniVision Technologies "OV7620 single chip video camera specifications", OmniVision Products Specifications, 2000 OmniVision OV7620 Single-Chip CMOS VGA color digital camera, OV6620 OV6120 CIF color and B/W digital video cameras Includes lens? [OmojolaO00a] Omojola, Olufemi; Post, E. Rehmi; Hancher, Matthew D.; Maguire, Yael; Pappu, Ravikanth; Schoner, Bernd; Russo, Peter R.; Fletcher, Richard; and Gershenfeld, Neil "An installation of interactive furniture", IBM System Journal, Vol 39 Nos 3/4, 2000, pp. 861-879 Capacitive/electrostatic (uses term electrostatic) touch sensors on furniture for active furniture -- M.I.T. Media Lab. Tabletop projected image on lazy-susan. 5x6 quasi-static capacitive sensor matrix on placement recognizes finger vs. palm gestures by area of contact. [OsawaN00a] Osawa, Noritaka; Asai, Kikuo; and Sigimoto, Yuji Y. "Immersive graph navigation using direct manipulation and gestures", Proc. Virtual Reality Software and Technology VRST 2000, Seoul Korea pp. 147-152 Virtual reality 3-D gestures including zoom (with two fingers of one hand), spotlight (using zoom), lock and unlock gestures. Using a gesture for unlock? [OverholtDJ00a] Overholt, Daniel James "The Emonator: A Novel Musical Interface", Master's thesis, M.I.T., Sep. 2000 Electromechanical 3D musical user interface, bed of rods. Cites to FingerWorks and TacTex multi-touch input touchscreens. [PCMonitor99a] Zanner, Peter "Husky FC-PX5: Harte Bursche", PC Monitor Zeitschrift, Vol 3 März, 1999, pp. 24-25 (in German) product review of Husky ruggedized PC with Windows 95, using touchscreen (stylus with side button) instead of mouse. [Palm99a] 3Com "3Com PalmVx PDA", PalmVx, 3Com, 1999 (physical device) Functional device: 3Com Palm Vx PDA running General Magic / MagicCap OS 3.1. Touchscreen: No separate keyboard. Includes CDs. [Palm00a] Palm Pilot / 3Com Corporation "Handbook for the Palm VII Handheld", 3com Corporation, 2000 (fetched 2005) Resistive-film PDA with wireless email via Palm.Net wireless service (pre-smartphone). Web browsing support via webclipping. [Palm00b] Palm Pilot / 3Com Corporation "Getting Started with the the Palm VII Handheld", 3com Corporation, 2000 (fetched 2005) Resistive-film PDA with wireless email via Palm.Net wireless service (pre-smartphone). Web browsing support via webclipping. [Palm00c] Palm Pilot / 3Com Corporation "Palm m500 Handheld Series User Manual", 3com Corporation, 2000 User manual for Palm PDA with resistive-film touchscreen. Graffiti handwriting recognition, User selects



character subset (numeric, alphabetic, etc.), pop-up virtual keyboard, electronic ink notes. Wireless network connectivity for email? [PanagrossiJ00a] Panagrossi, Jerry "Data Entry Device and Method", United States Patent 6,104,317, August 15, 2000 Stroke keyboard UI: touch in center of button (e.g. phone button), stroke in different directions to get different characters. Compare with marking menus keyboard of Buxton, Kurtenbach, etc. [PanditMS99a] Pandit, Milind S. "Recognition Of and Operation On Text Data", United States Patent 5,849,636, January 12, 1999 Recognize text (phone numbers, dates) in text, then perform operations on those pattern instances detected: e.g. pop-up menu for Send Mail / Add to Address Book. Highlighting of text by patterns, then clickable? [ParadisoJA00a] Paradiso, J.A.; Hsiao, K.; Strickon, J.; Lifton, J.; and Adler, A. "Sensor systems for interactive surfaces", IBM Systems Journal, Vol 39, Nos 3 and 4, 2000, pp. 892-914 Touchscreen/touch sensors for large surfaces: laser scanning rangefinder to track multiple hands; acoustic tap detection using differential time-of-arrival; piezoelectric wire to detect location and pressure (force), RFID tag reader for flock-of-birds of multiple tags in 3D three-dimensional coordinates. Gesture Wall with capacitively-coupled sensor to signal in body. Cites to Softboard for tracking multiple targets, computer vision for hand-tracking. Mentions conductive plastic as alternative to ITO for transparent touchscreen digitizers. Mentions Fingerworks multi-touch and and glidepoint [ParadisoJA00b] Paradiso, Joseph A.; Hsiao, Kai-Yuh; Strickon, Joshua; and Rice, Peter "New Sensor and Music Systems for Large Interactive Surfaces", Proc. of the Int'l Computer Music Conf. 2000 (ICMC 2000), Berlin, Germany, August 2000, pp. 277-280 phase-measuring laser rangefinder to create large multitouch (multiple hand tracking) touchscreen on large wall. Multimedia demonstrations [PebleyKM00a] Pebley, Kevin M. and Carlton, Todd D. "System and Method for Transferring Encrypted Sections of Documents Across a Computer Network", United States Patent 6,154,840, November 28, 2000 Encrypt document in sections so that user can fetch only the sections of interest from a document server. [PenComputing99a] Pen Computing Magazine "Pen Lab: Symbol Technologies SPT 1700: Ruggedized Palm III with laser scanner and integrated LAN radio", Pen Computing Magazine, June 1999, Page 44 Palm Pilot III touchscreen device with wireless LAN radio. File contains additional documentation. [PenComputing99b] Pen Computing Magazine "Palm VII", Pen Computing Magazine, 1999 Palm VII device with wireless LAN radio. [Perlin00] Perlin, Kenneth "Method and Apparatus for Writing", United States Patent 6,031,525, February 29, 2000 QuikWriting: refers specifically to it as using zones, ergo zone-based recognition, but the "writing" is not normal characters, it is a circular menu, no stylus-lift. See also press announcement "Look Ma, One Hand!" from Microsoft [PerryTS00a] Perry, Theresa Sue and Dillard, Pamela Ann "Wireless communication device adapted to generate a plurality of distinctive tactile alert patterns", United States Patent 6,160,498, December 12, 2000 Assign vibrating alert patterns (haptic vibration) to particular calls or numbers in a cell phone [PersidskyA00a] Persidsky, Andre "Self-contained pen computer with built-in display", United States Patent 6,130,66, October 10, 2000 Pen computer with no tablet: senses position by a "motion sensor" without special writing surface (Compare Anoto?), shows text (recognized?) on small display in housing of (thick) pen. Display flips orientation (rotate 180 degrees) for left-hand and right-hand usage. Compare with Anoto pen? [PetzoldC99a] Petzold, Charles "Programming Windows: Fifth Edition", Microsoft Press, 1999 (hardcopy book) Reference manual on Windows "C" Win32 API circa 1999. Described two and three-button mouse, WM_LBUTTON, WM_MBUTTON, WM_RBUTTON messages (p 275), Synchronous nature of messages in event que (p 329), system message queue [PfaffenbergerB00a] Pfaffenberger, Bryan "Webster's New World Dictionary of Computer Terms, Eighth Edition", IDG Books Worldwide, 2000 (hardcopy book) Technical dictionary of computer terms 2000 [PlamondonR00a] Plamondon, Rejean and Srihari, Sargur N. "On-line and Off-line Handwriting Recognition: A Comprehensive Survey", IEEE Trans. on Pattern Analysis and Machine Intelligence PAMI, January 2000, pp 63-84 Compare with Tappert/Suen survey articles on state-of-the-art in handwriting recognition for 1980 and 1990. Mentions Writer Identification (signature verification and identification), language models for incorporating grammar and dictionary models. [PocketComputing99] decas&integra.fr "Collection of pocket electronic devices", www.integra.fr/dreamteam/caz/computers/pocket/collection.htm, fetched 1999 Casio PB-100 scientific computer / palmtop with full keyboard, BASIC; Casio VDB-1000 touch-screen / touch sensitive wrist-watch computer; Casio IF-8000 touchscreen pocket computer, electronic ink on



screen, second touch pad buttons for calculator / alphabetic 1987; Casio NX-6000 Planeo touchscreen only palmtop, no physical keyboard; Sharp IQ-9200 touchscreen calculator, hierarchic list manager; full keyboard; Apple Newton Message Pad 110 PDA; NEC PC Engine GT / Turbo Express; Index portable keyboard-less / touchscreen computer with FM-Radio receiver, radio modem, GPS. Fetched 1999, refetched (water damage to print copy) from internetarchive.org, 2014 Sharp IQ-9200 also sold as OZ-9600 mk II? See also IQ-8920 [PocketPCLife99a] Windows CE Power Magazine (Clifford Brooks) "Calligrapher 5.3 handwriting Windows CE: Improve handwriting recognition with Calligrapher 5.3", www.pocketpclife.com/tocs/issue199212.html Produce review of Paragraph Calligrapher 5.3 handwriting recognition: full sentence run-on connected writing, not trainable, include pop-up keyboard for corrections and "special characters". Mentions confusion of single-stroke "1" and "I" (compare with Shillman?) [PorrasLN00a] Porras, Luis Narvaez and Goldberg, Isaac Rudomin "Embodied Modelling Tools in a 3D Environment", 8th Int'l. Conf. in Central Europe on Computer Graphics, Visualization and Interactive Digital Media'2000, Feb. 7-10, 2000, pp. 92-99 3D three-dimensional graphical editing using 2D touchscreen or mouse input user interface. To maximize viewing of 3D image on display, tools can be iconized, toolbars are transparent. [PostER00a] Post, E. Rehmi; Orth, Maggie; Russo, Peter R.; Gershenfeld, NNeil "E-broidery: Design and fabrication of textile-based computing", IBM Systems Journal, Vol 39 Nos 3/4, 2000, pp. 840-860 "No-Soap Radio" from M.I.T. Media Laboratory. Conductive metallic thread (organza) woven into fabric: fabric touch keyboard using matrix, conductive threads pressed to touch each other -- phantom contacts with multiple keys. Also capacitive keypads/touchscreen/touchpads. Organza: decorative silk thread wrapped in thin gold helix. Discussion of frabrication, stitching techniques for conductive yarn, other materials. Relate with multi-touch capactivie sensing. [PowellDE00a] Powell, Dennis E. "Practical KDE", QUE, Division of Macmillan Compute Publishing, 2000 (hardcopy book) Guide for KDE Desktop Environment/window manager for Linux/Unix [PrestiP99a] Presti, Peter "The Haptic Lens - A Tactile Sensor", Proc. IEEE/ASME Intl. Conf. on Advanced Intelligent Mechanis, Sep. 19-23, Atlanta, 1999, p. 787-ff (partial copy) Pressure-sensing 3D surface digitizer, translucent elastomer layer on clear rigid faceplate, flexible white layer on top. Pressing on surface is visible from underneath where translucent elastomer is compressed thinner. Capable of multi-touch? [PryorTR99a] Pryor, Timothy R. "Method for providing human input to a computer", United States Patent 5,982,352, November 9, 1999 Touchscreen (with rear projection), determine position optically by detecting Schlieren changes in polarization or other optical effects of force applied to (semi-)transparent screen. Multipoint (multi-touch). Force/pressure by degree of distortion. Tactile feedback -- actual touching of surface -- or vibration, inflatable bladders, air blasts, etc. Separate section uses optical targets on user's body: glove, hat, etc. [Psion99a] Psion, Inc. "Psion MX Series 5 Users Guide", Psion, Inc., 1998 Psion clamshell Palm-top (a bit large) with half-sized touchscreen and full-size abbreviated keyboard. Sketch drawing program, images can be inserted into Word documents. Voice note audio recording. [Psion00a] 3lib.UKOnline.co.uk "History of Psion", 3lib.ukonlin.co.uk/historyofpsion.htm, archive June 2000 History of Psion palm-top computers: original Organiser (PDA) with one line display. Later HC with modified DOS operating system. Series 5 in 1997..1999 with touchscreen, multi-tasking OS. Series 7 with touchscreen September 1999 included JVM Java machine for active web browsing pages. [Qualcomm99a] Qualcomm Incorporated "pdQ Applications Handbook", Qualcomm Incorporated, 1999 Handbook for applications on Qualcomm pdQ PALM/OS smartphone: address book, calculator, call history, dialer, email, expenses, modem emulation, browser . Calibration of digitizer / touchscreen (refers to "digitizer"). Gestures using upper-case alphabetics. One-touch dialing gesture. Refers to handwriting (write) input. [Qualcomm99b] Qualcomm Incorporated "pdQ Basics Handbook", Qualcomm Incorporated, 1999 User manual for Qualcomm pdQ PALM/OS smartphone: address book, calculator, call history, dialer, email, expenses, modem emulation, browser . Calibration of digitizer / touchscreen (refers to "digitizer"). Gestures using upper-case alphabetics. One-touch dialing gesture. Refers to handwriting (write) input. [RadleyPE99a] Radley, Petr Ernest "Hand-held radiotelephone having two-part construction", United States Patent 5,966,643, October 12, 1999 Radiotelephone/smartphone with separate earpiece/headphone and microphone. Include video camera and video display, infra-red link. [RakijasM99a] Rakijas, Michael and Archer, Benedict G. "Scientific Computing with Octave: Octave, GNUPlut and Other Tools", Southern California User Group, September 18, 1999 Introduction to Octave: MATLAB clone



[RekimotoJ99a] Rekimoto, Jun and Saitoh, Masanori "Augmented Surfaces: A Spatially Continuous Work Space for Hybrid Computing Environments", Proc. CHI '99, Pittsburgh, Penn. User interfaces for seamless integration of displays/computers via hyperdragging: integrate portable/small computer (pen computer, etc.) with larger display (virtually/physically adjacent) by dragging to edge of small computer, appears then in larger display. Larger display can be InfoWall Wellner-like desktop for integration with physical objects, or extension from desktop to InfoWall display. Used to link electronic files (e.g. annotations) to recognized physical object (e.g. book). Prototype uses 2D barcode marker/tags on physical objects for identification/targetting on Wellner desktop. Windows can be moved from laptop to InfoWall or InfoTable via hypderdragging off the original device onto the other. [RekimotoJ99b] Rekimoto, Junichi "Three-dimensional virtual reality space sharing method and system, an information recording medium and method, an information transmission medium and method, an information processing method, a client terminal, and a shared server terminal", United States Patent 5,956,038, September 21, 1999 3D virtual reality system over a network, similar in style to Second Life or MOOGs. [RekimotoJ00a] Rekimoto, Jun and Sciammarella, Eduardo "ToolStone: Effective use of the physical manipulation vocabularies of input devices", Proc. UIST '00, pp. 109-117 ToolStone: Small cube with 6 DOF 3D digitizer inside, used to simulate multiple (multi-use) input devices depending on which side is touched "down": french curve, eraser, etc. Bi-manual multi-touch input combined with stylus of Wacom tablet, [RheingoldH00] Rheingold, Howard "Tools for Thought, Chapter Eleven: The Birth of the Fantasy Amplifier", M.I.T. Press, 2000 History and interview with Alan Kay on Dynabook. Refers to Sketchpad electronic ink (?) graphics editing. Lengthy discussion of Dynabook, but all references are to mouse (not stylus/tablet) and keyboard, no mention of handwriting recognition. [Rocha00] Rocha, Manuel Antonio "Handwriting Template System", United States Patent 6,142,783, November 7, 2000 Physical/mechanical guide template, like those used for drifting lettering, as a guide to the correct writing of cursive script. Compare to restricted templates for doing Unistroke characters within a small mechanical box [RojasR00a] Rojas, Raul; Knipping, Lars; Raffel, Ulrich; and Friedland, Gerald; "Elektronische Kreide: Eine JavaMultimedia-Tafel für den Präsenz- und Fernunterricht", Technical Report B-17/2000, Freie Universität Berlin, Institut für Informatik, October 2000. Available at www.match.tu-berlin.de Research group investigating various whiteboard systems for university instruction and remote education, using commercial products. Real-time participation in class using web browser, including interactive audio and video streams. [Rolodex99a] Franklin Electronic Publishers "Rolodex Electronics RF-192 Personal Organizer User's Guide", Franklin Electronic Publishers, 1999 Small PDA device, with address book / phone numbers. PC sync. [RosenbergLB00a] Rosenberg, Louis B.; Schena, Bruce M.; Brave, Scott B. "Method and apparatus for tracking the position and orientation of a stylus and for digitizing a 3-D object", United States Patent 6,078,876, June 20, 2000 Pantographic 3-D stylus digitizer, including menu input by pantographic stylus, menu in work area. [SETBC99a] Special Education Technology of British Columbia "Visually Impaired Resource Guide: Assistive Technology for Students who use Braille", setbc.org, 1995 Resource/training notes on JAWS screen readers for visually impaired, description of features. Notes the particular difficulty of screen reader organization with Windows 95, compared to outSpoken on Macintosh. Loading bitmap recognizer to assign labels to unrecognised icons. [SchediwyRR99a] Schediwy, Richard R.; Pritchard, Jeffre O.; Kao, Ting; Allen, Timothy P.; and Platt, John C. "Object Position Detector", United States Patent 5,861,583, January 19, 1999 Capacitive grid tablet digitizer, senses capacitance changes at intersections in grid. Measures entire area of contact from multiple intersection nodes simultaneously (multi-touch). Produces profile of finger contact, multi-finger gestures. Infers pressure from outline of contact. [SchmalstiegD99a] Schmalstieg, Dieter; Encarnacao, L. Miguel; and Szalavari, Zsolt "Using Transparent Props for Interaction with the Virtual Table", Proc. 1999 ACM Symposium on Interactive 3D Graphics, Atlanta, Georgia, pp. 143-153, 232 Dual-handed input and manipulation of 3D graphics: user has both a 3D stylus/pen and a transparent touchscreen panel (a "prop") with 3D position sensors: stereoscopic images (shutter glasses) projected on surface: user can look through panel (tools and lenses, including MagicLens) are projected to appear to be on the panel, and use the tools on the 3D objects projected beneath. Described as Augmented *Virtual* reality: physical objects incorporated into VR image. Cites to MagicLens, Responsive Workbench. Ascension Flock of Birds 3D/6DOF sensors.



[SchmidtA99a] Schmidt, Albrecht; Beigl, Michael; and Gellersen, Hans-Werner "Sensor-based adaptive mobile user interfaces", Proc. HCI '99, pp. 251-255 Smartphone, PDA other devices adapt UI user interface to context of environment or user action: sensor for orientation change orientation of display (orientation-aware), ambient light for contrast, etc. Shows screen orientation of Apple Newton, but notes that it was not much used by users. Has picture of landscape/portrait on Newton. [SchneidermanR99a] Schneiderman, Ron "A Manager's Guide to Wireless Communications", American Management Association Press, 1999 (hardcopy book) Review of Wireless/smartphone communication circa 1999, including wireless internet access, wireless teleconferencing including web browser by DEC Mobile Computing and Communications Appliance, and Deutsch Telekom. Front cover: stylus/pen smartphone -- IBM Simon? Video: p. 126 and front cover [SciammarellaE00a] Sciammarella, Eduardo "Interactive control of audio outputs on a display screen", United States Patent 6,081,266, June 27, 2000 Touchscreen user interface for audio outputs: make icon larger for louder, move graphical object to change balance. Compare with Buxton audio control panel? [Scribens99] Scribens Research Laboratory of Ecole Polytechnique Montreal "Bibliography of publications", Available at http://www.scribens.polymtl.ca/publications.html Lists publications and patents by Plamondon, Privitera, Guerfali, Djezire, Nouboud, Barriere, Parizeau, Clergeau-de-Tournemire, Leclerc, Sabourin, Beaumier, Brault, Yergeau, Suen, Chouinard, Lorette, Clement, Stelmach, Schomaker, Baron, Beauregard, Gagne, Robillard, Poussart, etc. Signature verification, electronic ink data compression formats, Human recognition accuracy, Fuzzy syntax for script recognition, normalization of handwritten characters, more [SelkerT99a] Selker, Ted "Style and Function of Graphic Tools", Graphics Interface GI'99 Online Papers: www.graphicsinterface.org Essay reviewing graphical user interfaces, including position input, no mention of gestures or handwriting recognition. Speaks to advantages of pie menus (gestures) [SensorySoftware99a] Sensory Software Inc. "HandsOff: on-screen keyboard", Sensor Software, 1999 On-screen keyboard, rectangular grid display, intended for severely handicapped users. Any pointing device may be used -- e.g. head-mounted pointer, does select on "dwell" gestures as substitute for touch or mouse click. [ServanSchreiberF99a] Servan-Schreiber, Franklin "Display of multiple images based on a temporal relationship among them with various operations available to a user as a function of each displayed image size", United States Patent 5,973,691, October 26, 1999 User interface for photographs, more recent images are larger and in front. User can tap/select images. Larger images have more options available for labeling and editing. Mouse input. [ShawS99a] Shaw, Sandy "JISC Technology Applications Programme (JTAP): Overview of Watermarks, Fingerprints, and Digital Signatures", Report U. Edinburgh, August 3 1999, available at smartsheep.org/jisctechnology-applications-programme-jtap, fetched 2014 Overview and tutorial for technologies for copyright protection for digital documents and information sources. Notes that digital signatures (PKI) awkward for data subject to modification by compression, etc. Watermark indentifies originator, fingerprint identifies intended recipient, PKI/signatures authenticate origin, integrity of content, non-repudiation of origin, and confidentiality (encryption). Discussion of ownership assertion and rights management. [ShneidermanB00a] Shneiderman, Ben; Kang, Hunmo "Direct Annotation: A Drag-and-Drop Strategy for Labeling Photos", Proc. IV '00, Intl. Conf. on Information Visualisation, Wash. DC 2000 IEEE Computer Society Annotate/label personal photo collections by drag/drop of names onto correct part of image, and into list boxes on sides, etc. Mouse or touch screen (direct manipulation), text values only ever entered first time of use via keyboard. [SieverE99a] Siever, Ellen "Linux in a Nutshell: A Desktop Quick Reference, 2nd Edition", O'Reilly and Associates, 1999 (hardcopy book) Linux command and user interface user guide [SiioI99a] Siio, Iitiro; Masui, Toshiyuki; and Fukuchi, Kentaro "Real-world Interaction using the FieldMouse", Proc. UIST '99, Ashevill NC, pp. 113-119 FieldMouse: pen-mouse combined with bar-code reader. Determines position from known location of barcode ID, then determines absolution position by extrapolation from mouse-sensed relative movement. Example is active book, children's book with barcode tags in pictures (infra-red ink?). Compare with Anoto. [SinghM00a] Singh, Mona "Apparatus and Method of Formatting a List for Display on a Touchscreen", United States Patent 6,157,379, December 5, 2000 Automatic layout in user interface of a pick/menu list for touch screen input, with appropriate spacing for targeting.



[SinghM00b] Singh, Mona "Device Having Adjustable Touch-based Display of Data", United States Patent 6,154,194, November 28, 2000 Scrolling and window re-size areas in user interface window for touchscreen [Sklarew99a] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 6,002,799 assigned to AST Research, Inc., Irvine, California, December 14, 1999 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" Very long list of prior art [Sklarew99b] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 5,933,526, assigned to AST Research, Inc., Irvine, California, August 3, 1999 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" Specific to learning mode where adaptive recognition must be taught from initial state [Sklarew00] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 6,064,766, May 16, 2000 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" actions are confirmed specifically by reverse-video of the field operated upon [SmailagicA99a] Smailagic, Asim; Siewiorek, Dan; Iannucci, Bob; Dahbura, Anton, and Bass, Len "MoCCA: A Mobile Communication and Computing Architecture", Mobile Computing and Communications Review, Vol 3, No 4, 1999, pp. 39-45 Early smartphone/communicator prototype, included web server and servlets, pcAnywhere, Phone dialer (OTS software), browser for wireless communications. Display video, touchscreen interface. [SmailagicA99b] Smailagic, Asim "An evaluation of audio-centric CMU wearable computers", Mobile Networks and Applications, Vol 4, 1999, pp. 59-68 CMU Carnegie Mellon Wearable Computers: wireless phone (smartphone) with full-motion video. [SmithJR99a] Smith, Joshua Reynolds "Electric Field Imaging", Ph.D. Thesis, M.I.T. School of Architecture and Planning, February 1999 Electric field/capacitive sensing/imaging, sections on synchronous detection methods, spread spectrum,signal processing. Not directly about tablets or digitizer, examples of capacitive field sensing for hand and body parts in various applications. LaZmouse: 3D locator/pointing device not requiring user to hold the mouse. See also FishFace (general electric field proximity/position sensor.) [SmithJR99b] Smith, Josh "Electric Field Imaging: "Seeing" by Inverse Electrostatics -- Overview of Electric Field Sensing", MIT Media Lab, web.media.mit.edu/~jrs, 1999 (date approximate) Electric field/capacitive sensing/imaging. Review of touchpad and proximity capacitive sensing projects: LazyFish sensing board, School of Fish, NEC Passenger Sensing product Active Seat (airline), 3D/6DOF FieldMouse shows interpolating positions of two hands. FishFace touch/display device, Active Table, [SmithJR99c] Smith, Joshua R.; Salthouse, Christopher; and Gershenfeld, Neil "Code-Division Multiplexing of a Sensor Channel: A Software Implementation", IEEE Jnl. on Selected Areas in Communications, vol 17 no 4, April 1999, pp. 725-731 Capacitive (or other) field sensing (applicable to tablets) using multiple modulated signals, demodulated at back end. CDMA spread-spectrum gives more consistent noise rejection than TDMA modulation. Implementation is "LazyFish" electronics board. Compare with syncronous demodulation in Blesser, Rekimoto? [SmithiesS99a] Smithies, Steve, Novins, Kevin, and Arvo, James "A handwriting-based equation editor", Proceedings of Graphics Interface '99, pages 84-91, Kingston, Ontario, June 1999 Two-dimensional input with handwriting recognition for mathematics: features include showing parsing in real-time, with an eight stroke delay, showing bounding boxes on all characters/groupings as an aid to user understanding parsing. User can edit strokes, select alternate recognition results, and edit grouping/parsing of strokes. [Sony00a] Sony Corp. "Sony DataRover 840 PDA", General Magic/MagicCap DataRover 840, Sony Corp., 2000 (physical device) Functional device: Sony DataRover 840 PDA running General Magic / MagicCap OS 3.1. Touchscreen: No separate keyboard. Includes CD. [Spiegel99a] Der Spiegel "Der Palm VII - der erste "Online Palm"", Der Spiegel, 17 Nov. 1999 Review of early 1999 release of Palm VII, with live internet connections: email, multimedia, WAP-handy (cellphone) [SpitzerMB00a] Spitzer, Mark B. "Eyeglass Interface System", United States Patent 6,091,546, July 18, 2000 Video camera, display, microphone in eyeglasses. One use us is hands-free cellphone. Compare with Google Glasses.



[StefikM99a] Stefik, Mark "The Internet Edge: Social, Technical, and Legal Challenges for a Networked World", MIT Press, Cambridge, 1999 Excerpts, Chapters 3, 4, 8. Essays on digital rights management and digital payments via Internet. Stefik: Chapter 3. The Digital Wallet and the Copyright Box. The Coming Arms Race in Trusted Systems. Stefik: Chapter 4. The Bit and the Pendulum. Balancing the Interests of Stakeholders in Digital Publishing. Stefik: Chapter 8. The Digital Keyhole. Privacy Rights and Trusted Systems. [StewartJ99a] Stewart, Jason; Bederson, Benajamin B.; and Drui, Allison "Single Display GroupWare: A Model for Co-present Collaboration", Proc. ACM CHI'99 Conf., May 15-20, 1999, pp. 286-293 Multiple users sharing single display (whiteboard), but from separate workstations/PDAs with touchscreen. Asks question about accessibility using audio output for the blind/visually impaired for groupware/whiteboard. [StreitzNA99a] Streitz, Norbert A.; Geissler, Joerg; Holmer, Torsten; Konomi, Shin'ichi; Mueller-Tomfelde, Christian; Reischl, Wolfgang; Rexroth, Petra; Seitz, Peter; Steinmetz, Ralf "i-LAND: An interactive Landscape for Creativity and Innovation", Proc. CHI '99, May 15-20, 1999, Pittsburgh, pp. 120-127 i-LAND/BEACH: InteracTable touchscreen tabletop (rear projection), DynaWall. Shows windows rotated on tabletop display for varying user perspectives/orientation around table. Gesture input, Objects may be moved from one display to another via take-and-put or pick-and-drop. Virtual location is cooperation via system regardless of user physical location. [StreitzNA99b] Streitz, Norbert A.; Geissler, Joerg; Holmer, Torsten; Konomi, Shin'ichi; Mueller-Tomfelde, Christian; Reischl, Wolfgang; Rexroth, Petra; Seitz, Peter; Steinmetz, Ralf "i-LAND: An interactive Landscape for Creativity and Innovation", Proc. CHI '99, May 15-20, 1999, Pittsburgh, Video demonstration Video of InteracTable and related devices. [StrickonJA99a] Strickon, Joshua Andrew "Design and HCI Applications of a Low-Cost Scanning Laser RangeFinder", Master's Thesis, EECS, M.I.T, June 1999 scanning laser rangefinder for tracking multiple hands (multipoint touchscreen) in front of large rearprojected display wall. Refers to Gesture Wall, smart wallboards (whiteboards) with handwriting tracking. Tracking/rangefinding by triangulation, time-of-flight (speed of light delays measured), continuous wave phase. [StroustrupB99a] Stroustrup, Bjarne "An Overview of the C++ Programming Language", CRC Press, 1999, ISBN 0-8493-3135-8 Summary of C++ object-based programming language. Abstraction mechanisms, Standard Library, namespaces and exceptionhandling for large-scale programming, comparison with "C" programming model. Describes principles of development behind C++ (rationale). [SubrahmoniaJ00a] Subrahmonia, Jayashree and Zimmerman, Thomas "Pen Computing: Challenges and Applications", Proceedings of ICPR 2000 (pre-print) Short overview of pen-computing: refers to ultrasonic/acoustic digitizer, also barcodes like that of Anoto but different. Gives limiting factors for acoustic/sonic digitizer, at 50 dpi due to air currents, etc. Optical pattern digitizer building up mosaic of images from normal paper: not clear that they ever actually evaluated such a device [SynapseAdaptive00a] Synapse Adaptive "outSpoken Ensemble 3.0 Screen Reader Software with Braille and Speech Output for Windows ME/95/95", www.synapseadaptive.com, fetched 2014 (!) Packet of GWMicro outSpoken for windows and Office 2000: supporting older Windows versions in 2014. [SynapseAdaptive00b] Synapse Adaptive "outSpoken for MacIntosh 9.0", www.synapseadaptive.com, fetched 2014 outSpoken screen reader for MacIntosh, speaks icons and user interface elements [Synaptics00a] Synaptics Inc. "Synaptics TouchPad Interfacing Guide", Synaptics.com, January 18, 2000: original version 1998 Physical protocols for Synaptics Touchpad interfaces: PS/2 mouse, Serial RS-232C tablet, ADB Apple Macintosh. Optional dual sensing of pen and touch (capPen). Gestures for scrolling, mouse-button emulation (tapping) implemented in driver, tap-and-a-half drag gesture. Because of tap timing (time-out, no second tap) simulated mouse clicks are generated after a slight time delay. Edge Motion: movement to edge generates continuing motion in same direction. Hop gestures are two taps separate left or right. Drag gesture is tap-lift-hold (tap-and-a-half) [TacTex00a] TacTex Controls Inc. "MTC Express Guide multitouch touchpad: Owner's and Developer's Guide, Version 1.1", TacTex Controls Inc., April 20, 2000 optical-fiber multi-touch controller, shows map of fingertip images for five fingers as part of installation verification. Normalization process to calibrate touch response (taxel == touch pixel) with interpolation for higher resolution. "C" API reference. [TakahashiT99a] Takahashi, Tomoichi and Kuzuya, Mikio "3-D Input Device Using a Ball Rotation Interface", Proc. 8th Int'l. Conf. on Human-Computer Interaction, Munich Germany, Aug 22-26, 1999, Vol. 1 pp 397-401



3D three-dimensional input, using trackball (3D orientation) on top of joystick (X/Y) and up/down buttons (Z position) [TarrC00a] Tarr, Christopher; Salisbury, J. Kenneth Jr.; Massie, Thomas Harold; and Avilies, Walter A. "Method and apparatus for generating and interfacing with a haptic virtual reality environment", United States Patent 6,084,578, July 4, 2000 Interfacing a haptic device/display with a virtual reality environment, using a model of the compound objects of the virtual reality environment. Many references. [TealPoint00a] TealPoint Software "TealNotes User's Manual, Program Version 1.14", TealPoint Software, 2000 Annotation/sticky notes for Palm OS touchscreen computer: intercepts all text fields (field contains a special code), adds transparent rectangular overlay/window for electronic ink notes, via HackMaster systems extension. Can be activated by one-letter graffiti stroke/gesture. Note garbage-collection problem of "orphan notes" when original record is deleted, ink data still in database. On-screen ink editing (erase) or edit ink data images directly using TealPaint. [TealPoint00b] TealPoint Software "TealScript User's Manual, Program Version 2.21", TealPoint Software, March 16, 2000 TealScript replacement for Graffiti handwriting recognition on Palm touchscreen computer: trainable recognition, matches to example strokes in user profile (elastic matching?) User can also adjust match thresholds ("squelch"). Notes incompatibility with other 3rd party application that patch (hook) into the Palm OS. [TengJH00a] Teng, Jen-Hao; Chan, Shun-Yu; Lee, Jin-Chang; and Lee, Roy "A LabVIEW Based Virtual Instrument for Power Analyzers", PowerCon 2000, Proc. Int. Conf. on Power System Technolgy, Perth, Australia, 4-7 Dec. 2000, pp. 179-184 LabVIEW example: Virtual instrument for power management, shows data flow architecture of components. [TognazziniB99a] Tognazzini, Bruce "Linear Touch Input Device", United States Patent 5,859,629, January 12, 1999 one-dimensional linear touchpad, pressure sensitive material. Placement next to keyboard, use for inputting linear quantities. [TreeseW00a] Treese, Win "Putting it Together: Going Wireless", netWorker, June 2000, pp. 9-12 Overview of wireless smartphones circa 2000: OmniSky Palm Vx with NovAtel wireless modem, Palm VII, Spring PCS Wireless Web, NeoPoint 1000 RIM 950 Wireless handheld. Some use touchscreen but not cited in article. [TrinderM99a] Trinder, Michael "The computer's role in sketch design: a transparent sketching medium", in Computers in Building: Proceedings of the CAAD Futures 99 Conference, Kluwer Academic Publishers, Boston 1999 Includes overview/survey of sketch input systems: electronic cocktail napkin, Bill Buxton. Compares mouse to drawing with a brick, pressure-sensitive tablet and use of tilt in user interface. [TurkM99] Turk, Mathew "Chapter 10: Gesture Recognition", Draft manuscript for "Handbook" project: Available at vehand.engr.ucf.edu/handbook Gesture recognition, but primarily about hand and body gestures, not tablet/stylus gesture recognition for a computer GUI. One review section on Pen-based Gesture Recognition. PenPoint/GO is notable by not being mentioned: author's address is at Microsoft Research. Only citation before 1991 is to Sketchpad. [UKA99] Interactive Systems Labs, CMU and Uni Karsruhe "Publications on Neural Nets, Speech, Multimodal and Natural Language Processing", http://isl.ira.uka.de/ISL.publications.html Source of Jaeger abstracts on handwriting recognition: lip-reading, speech recognition and UIs mostly. [UchihashiS99a] Uchihashi, Shingo and Wilcox, Lynn "Automatic Index Creation for Handwritten Notes", Xerox Palo Alto Technical Report FXPAL-PR-99-135, 1999; Proc. ICASSP '99 IEEE Int'l. Conf. on Acoustics, Speech, and Signal Processing, March 15-19, 1999, Vol. 6 pp. 3453-3456 indexing of electronic ink notes without character recognition: most common groupings / index term clusters (or dominant groupings, appear like titles?) are put into hyperlink table with page numbers. [Unigraphics00] Unigraphics User Museum "Miscellaneous Junk", http://www.uguser.org/home/museum/hall/Misc_junk.htm Collection of early Unigraphics tablet/pointing devices: Textronix Data Tablet, Spaceball three-dimensional joystick/pointing device, UGDD Unigraphics Detail Drafting tablet overlay, showing menus for keystroke macros (CAD UI menus on tablet digitizer) or GRIP programs, McDonnel Douglas [UreMJ99a] Ure, Michael J. "Touch-sensitive keyboard mouse", United States Patent 6,982,302, November 9, 1999 multiple simultaneous touches used for touch-sensitive keyboard with chording: one-handed keyboard using multiple key depressions -- touch-tone keyboard layout, or for one-hand PDA-type device. Any kind of multi-touch sensor may be used. Automatic registration of fingers by having space (most frequent



character) be all four fingers. [UreMJ00a] Ure, Michael J. "Touch-sensitive keyboard mouse and computing device using the same", United States Patent 6,107,997, August 22, 2000 multiple simultaneous touches used for touch-sensitive keyboard with chording: one-handed keyboard using multiple key depressions -- touch-tone keyboard layout, or for one-hand PDA-type device. Touch sensor shows images of fingertips. Any kind of multi-touch sensor may be used. Haptic effects of real keyboard? [UyeharaL00a] Uyehara, Lance; Eberhard, Martin F.; and Tarpenning, Marc E. "Display orientation features for hand-held content display device", United States Patent 6,154,214, November 28, 2000 handheld PDA/ebook reader with touchscreen: orientation button brings up dialog to rotate screen to 0,90,180,270 degrees. Resets to 0 degrees when placed in cradle. Pop-up menus for other display options. [VanKleeckM99a] Van Kleeck, Michael "Method and system for entering data using an improved on-screen keyboard", United States Patent 6,008,799, December 28, 1999 On-screen virtual keyboard with alphabetic keys, common words (e.g. o: or, on of, over) on additional keys below, double-click and flick gesture to pop-up dictionary words or additional characters or punctuation. Cites to PenPoint and Windows for Pen Computing / PenWindows 1.0. Period character can automatically add additional two space characters. [Videotile99] Videotile: AT&T Laboratories "The Videotile", AT&T Laboratories, Cambridge Massachusetts Tablet digitizer and display, refers to sending raw analog video to the device rather than processing the video stream in the terminal: mentions Papyrus Associates and Allegro handwriting recognition. Recognition algorithms execute on separate server processor, not in local firmware [VigodaBJ99a] Vigoda, Benjamin William "A Nonlinear Dynamic System for Spread Spectrum Code Acquisition", Master's Thesis, M.I.T., September 1999 Low-power spread-spectrum modulation and decoding. Used in TouchTag encoder to send data through capacitive coupling (capacitive touchpad) see Zimmerman 1995. [VincentN00] Vicent, N. and Dorizzi, B. "A Fractal Justification of the Normalization Step for Online Handwriting Recognition", Proc. 7th International Workshop on Frontiers in handwriting Recognition, pp. 535-540 Cites to REMUS recognition software: handwriting recognition, application of fractal models to determine characteristics which do not vary based on tablet resolution, writing speed, or writing size. Automatic scaling of sizes? [VoelzH99a] Völz, Horst "Das Mensch-Technik-System: Physiologische, physikalische und technische Grundlagen Software und Hardware", Expert Verlag, Germany, 1999 (hardcopy book) (in German) Introduction and review tutorial of human-machine interfaces. Broad-ranging overview includes sensors and human output for Hall-effect sensing of magnetic fields, oder/smell, gas emissions, graphical dithering, not just traditional touch and key inputs with visual output. Page 164 discusses touchscreen inputs, cites to Ward/Schulz (!). [Wacom00a] Wacom, Inc. "Wacom Intuos 3 tablet (physical device)", Wacom, 2000 (date approximate) Electromagnetic tablet (Wacom), approximate letter-size, pressure/force-sensing cordless stylus, also includes "mouse" -- mouse-styled wireless puck. [WagnerK99] Wagner, Kirsten "Informations- und Wissensorganisation anhand räumlicher Ordnungsmodelle: Das "Spatial Data-Management System" der "Architecture Machine Group" als Fallbeispiel", Position Statement, Internationale Zeitschrift für Theorie und Wissenschaft der Architektur, ISSN 1439-8363, Wolkenkuckucksheim, 1999 Review of data-access models using graphical representations: shows Memex system of Vannevar Bush, showing stylus input (with digitizer tablet? Telautograph?) for hand-written electronic ink, possibly handwriting recognition to insert "code symbols" as annotation and linking information on a document. Photograph of Sketchpad by Sutherland, 1963. [WakerlyJF99a] Wakerly, John F. "Digital Design: Principles and Practices, Third Edition", Prentice Hall, 1999 Undergraduate textbook on digital design and logic [WalrathK99a] Walrath, Kathy and Campione, Mary "The JFC Swing Tutorial: A Guide to Constructing GUIs", Sun Microsystems, 1999 (hardcopy book) Java Swing: Layout Managers for positioning GUI elements at runtime (not an IDE), Limited support for Accessibility API com.sun.java.accessibility.util and AccessibleJComponent base class, documentation for accessibility on included CD-ROM. [WalshN99a] Walsh, Nancy "Learning Perl/Tk", O'Reily & Associates, 1999 Mapping between UI events and mouse buttons in Perl [WanderleyMM00a] Wanderley, M.M. and Battier, M, eds. "General Bibliography on Gestural Control in Music", Reprint from "Trends in Gestural Control of Music", 2000, Ircam - Centre Pompidou Very long bibliography: mostly about gestures of music conductors, but contains some references to written gesture recognition with stylus and digitizer. Also contains references to recognition of American



Sign Language gestures. [WellsND99a] Wells, Nicholas D. "Teach Yourself KDE 1.1 in 24 hours", SAMS Publishing, 1999 (hardcopy book) Guide for KDE window manager for Linux/Unix: Describes configuration UI for mouse, thresholds for acceleration on movement, and threshold for minimum distance in a short period of time for mouse draggesture movement to go into accelerated speed: compare with flick gesture? KDE 2.0 also has minimum drag distance, minimum number of pixels movement before a drag is started. [WellsND00a] Wells, Nicholas "Special Edition: Using KDE", QUE, Division of Macmillan, 2000 (hardcopy book) Guide for KDE Desktop Environment/window manager for Linux/Unix [WeltnerT00a] Weltner, Tobias "Windows Scripting Secrets", IDG Books, 2000 (hardcopy book) VBScript -- and by reference, CScript [WestermanW99a] Westerman, Wayne "Hand Tracking, Finger Identification, and Chordic Manipulation on a multitouch surface", Ph.D. dissertation, U. Delaware, Spring 1999 Refers to hotscroll continuous scrolling on OS/2 by Samuel Audet, and Autoscroll (scrolling momentum) by a four-finger gesture without decelerating. Multi-touch gestures / multi-touch input devices at Bill Buxton's lab 1980's, mentions LeeSK85. [WiVik99a] Synapse Adaptive "WiVik 2 V2.5 On-Screen Keyboard", SynapseAdaptive.com, 1999 WiVik (Windows Visual Keyboard) on-screen virtual/software keyboard for Windows 3.1, 95, NT, version 2.5. Accessibility for motor-impaired. Double-click gesture on key for shift/upper-case, selection/button activation on button-down/touch instead of button-up/release/lift. Expanding-quadrant selection (compare with marking menus): keyboard image shown in four segments, dwell on segment expands segment further divided into four quadrants, etc. Highlighting of keys as pointer passes over. Intelligent "always-ontop" when pointing to keyboard or application. [WilsonAD00a] Wilson, Andrew David "Adaptive Models for the Recognition of Human Gesture", Ph.D. Thesis, M.I.T, February 2000 Gesture recognition (hand and body gesture) for interactive demonstration "Swamped!". Gesture of hand inside a puppet. [WilsonJY99a] Wilson, James Y. "Remote Occlusion Region", United States Patent 6,005,533, December 21, 1999 Remote-control touchscreen terminal for computer, shows screen of host computer with local virtual keyboard. Defines a blocking/occlusion window on screen of host computer to match local virtual keyboard image. Omnibus specification for CruisePAD pen computer. Wake-up from low-power sleep mode when touchscreen is touched: Compare with Wang Freestyle. [WinogradT99a] Winograd, Terry and Guimbretiere, Francois "Visual Instruments for an interactive Mural", Proc. CHI 99, 15-20 May, 1999, pp. 234-235 Optically-tracked laser pointer digitizer: both a stationary click (turn on laser while close) and marking menu by sweeping laster pointer across an action bar (side of octagon for pie menu). Five gestures like Xerox STAR menu items (cut, copy, paste, undo gestures: sweep gesture across action bar. Universal menu gesture can be used anyplace for large-size display: press-and-hold? it refers to pie menus and Kurtenbach [WolfeA00] Wolfe, Andrew Lawrence and Barrett, Gary Lloyd "Multi-modal touch sensitive peripheral device", United States Patent 6,037,930, March 14,2000 Resistive film digitizer used to implement various virtual devices: joystick mode, mouse mode, absolute mode, keypad mode Compare to Kaplow? [YuW00a] Yu, Wai; Ramloll, Ramesh; and Brewster, Stephen "Haptic Graphs for Blind Computer Users", Proc. 1st Intl. Workshop on Haptic Human-Computer Interaction, pp. 41-51 Three-dimensional 3D force-feedback electromechanical digitizer for visualizing simple graphs and 3D surfaces for the blind and visually impaired. Adding auditory feedback and texturing aids in the perception of the surface: compare with SciAm article on multi-sensate integration. [ZettsJM99a] Zetts, John Mark and Desrosiers, Maurice Roger "Distinguishing gestures from handwriting in a pen based computer by size discrimination", United States Patent 5,862,256, January 19, 1999 Size/dimension threshold for tap gesture is smaller if character recognition is also enabled, to allow user to use both tap and small characters like periods. Effect on press-and-hold? two thresholds, but no pressand-hold [ZettsJM99b] Zetts, John Mark and Desrosiers, Maurice Roger "Distinguishing gestures from handwriting in a pen based computer by stroke analysis", United States Patent 5,864,635, January 26, 1999 Bounding rectangles to segment strokes into likely words: use this information plus size to distinguish writing from gestures. [ZettsJM00a] Zetts, John Mark and Desrosiers, Maurice Roger "Method of and System for Facilitating User Input into a Small GUI Window Using a Stylus", United States Patent 6,049,329, April 11, 2000 When user starts writing on text field, bring up larger rectangle to write in. If just tap in text field, select it.



[Zowie99a] Zowie Entertainment "PirateQuest Zowie PlayZone", Zowie Entertainment, 1999 Multi-touch character game using physical tokens (of pirate characters) whose position and identity on a ship model determined electronically (electromagnetic?). Compare with M.I.T. physical tokens. biblio98 (Prev page)

Main Page







Main Page


[AFB01a] American Federation of the Blind "Product Evaluation -- The Tools of the Trade: Advanced Screen Reader Features", AFB AccessWorld Magazine, March 2001 Screen readers: automatic labeling of graphics in Window-Eyes, Window Bridge, JAWS [AMUG02] AMUG "AMUG's Newton Page", http://www.amug.org/amug/sigs/newton List of links to Newton pen-computing software, long list of Newton applications and companies [AdvancedRecognitionTechnologies02] Advanced Recognition Technologies "smARTspeak NG, simpliWrite, smARTWriter product descriptions", http://www.artcomp.com, 43 Brodezky Street, POB 39918, Tel-Aviv, 61398, Israel Product literature on 'Noise Robust', handwriting and speech recognition products. Mentions special support for left-handed handwriting input. Says special writing areas and gestures are not required, but also mentions support for macro gestures. Appears to be the former ParaGraph recognition technology. Speech recognition in TI/Texas Instruments C55x DSP chips. Multilingual handwriting recognition, Roman/Chinese/special symbols. Presents user with a recognition result at the end of every character: similar technique used in Wang Freestyle laptop with simulated devices in 1987 (see Martin/Ward/et-al patent: though not mentioned there). [AlbinssonPA03a] Albinsson, Par-Anders and Zhai, Shumin "High Precision Touch Screen Interaction", Proc. CHI 2003, April5-10, 2003, Ft. Lauderdale, USA Touchscreens advantageous over physical keys and switches because control surface is overlaid on display. Mentions targeting accuracy limits of fingers, especially small targets. Suggests zooming, and second-stage fine adjustment of location by cursor keys (may be virtual) or cross-hairs, or selecting on finger lift: selection on finger lift enhanced by displaying a "precision handle" to let user adjust fine position before lifting. [AllanRA01a] Allan, Roy A. "A history of the personal computer: the people and the technology", Allan Publishing, London, Ontario, Canada, 2001 Mentions Sidekick phone dialer for DOS by Borland Internationa, 1984. On-screen highlighting of phone numbers for dialing.



[AmentoB02a] Amento, Brian; Hill, Will; Terveen, Loren "The Sound of One Hand: A Wrist-mounted Bio-acoustic Fingertip Gesture Interface", Proc. CHI 2002, pp. 724-725 Hand gestures picked up by sound/acoustic sensor on wrist via bone conduction: finger-tapping, rubbing, flicking, detected by sound [AmroHY01a] Amro, Hatim Yousef and Dodson, John Paul "Touch screen with random finger placement and rolling on screen to control", United States Patent 6,278,443, August 21, 2001 Rolling gesture: roll fingertip or stylus on touchscreen tablet, such as to scroll for fine positioning. Finger tip *not* placed over cursor. May also scroll whole image on screen. Not clear how rolling is detecting: small vector? pressure? (Resistive-film digitizer) [AndersonEC01a] Anderson, Eric C. and Dalke, George W. "Method and System for Auto Rotating a Graphical User Interface for Managing Portrait and Landscape Images in an Image Capture Unit", United States Patent 6,262,769, July 17, 2001 Rotate/flip orientation of user interface, and of camera image, when a digital camera is turned landscape/portrait mode. [Anoto03a] Anoto AB "Anoto Software Development Kit: Development Guide for Services Enabled by Anoto Functionality", Anoto AB, Sweden: [email protected] Applications software development kit for Anoto "smart paper" patterned optical digitizer technology. File contains additional (and more recent) materials. [Anoto03b] Anoto AB "Logitech and other vendor Anoto pens, with pattern paper ca. 2003", Anoto AB (physical device) Optical digitizer using "smart paper" patterned optical digitizer technology, included working pens, assortment of paper, applications software for Windows. Optical stylus (inking pen) determines position, which paper form, which instance of paper form from pattern. Able to determine height, angle, rotation of stylus/pen. [Apple03a] Apple Computer "Using Ink Services in Your Application", Apple Computer, July 24, 2003 Documentation of electronic ink and handwriting recognition in Apple Carbon/Cocoa, Objective-C with graphics tablet and a stylus. Automatic (pen-enabled) handwriting recognition in Mac OS X version 10.2. Ink services version 10.3: gestures for editing text. Gestures similar to PenPoint: cut (pigtail), clear (rubout), escape, join, etc. Untargetted (use of) gestures applies to current selection or text insert point, targetted (use of) gestures uses hot spot location: join ("v") has two hot spots. Compare with PenWindows and Tablet PC (contemporaneous). Handwriting/gestures anywhere on screen, except in specific mouseinput areas (title bar, etc.). Toggle button between ink/handwriting recognition modes (compare: PenCAD). Discusses mouse-event coalescing, disabled for electronic ink (no buffering). [Apple03b] Apple Computer "Ink Services Reference", Apple Computer, 2003 API documentation for Ink Services: electronic ink, (text) gestures, and handwriting recognition on Apple Mac OS X. Ink Phrases: collection of strokes -- compare with PenPoint Scribbles. [ArdS02a] Ard, Scott "Tablet PC's troubled heritage", CNET news.com.com, November 7, 2002 Review of some history of pen-computing: does not mention Pencept or CIC/Handwriter of mid-1980s. T.L. Dimon Stylator, RAND tablet Grafacon. Cites Alan Kay dynabook (however it did *not* include handwriting input). [ArfibD02a] Arfib, Daniel and Dudon, Jacques "A digital emulator of the photosonic instrument", Proc. 2002 Conf. on New Instruments for Musical Expression (NIME-02), Dublin, Ireland, May 24-26, 2002, pp. NIME01-NIMI04 Computer simulation of historical (electronic) musical instrument with waveforms printed concentrically around rotating optical disk. Two-handed user interface uses two styli (mouse and stylus) on same Wacom tablet: multi-touch. [ArvoJ02a] Arvo, James "Jim Arvo's List of Publications", www.cs.caltech.edu/~arvo/list.html, 2002 Publication list for James Arvo, mostly on photorealistic rendering of graphical images. [AscensionTechnology02a] Ascension Technology Corporation "Flock of Birds Installation and Operation GuideRevB", Ascension Technology Corporation, January 31, 2002 Installation and operation guide for 6DOF/3D position tracking digitizer. First revision of documentation 1992. One example is tracing (with one sensor) 3D position and orientation of electronic game pistol. Generated magnetic fields can affect CRT monitors. Nearby metals affect magnetic fields, cause errors, transmitter must be mounted using non-magnetic materials. [AvrahamiD01a] Avrahami, Daniel; Hudson, Scott E.; Moran, Thomas P. and Williams, Brian D. "Guided Gesture Support in the Paper PDA", Proc. UIST '01, Orlando, Florida 2001 paper PDA -- user writes on paper (with pre-printed marks/forms), scans into computer, which recognizes and processes/stores information, prints out new pages. Semi-structured templates or guides for user to make gestures/marks on the paper: e.g. orientation dots, angled figure-8 pattern of guidelines for arrows/check-marks/cross-out/asterisk. [BabitskyS02a] Babitsky, Steven and Mangraviti, James J. jr. "Writing and Defending your Expert Report: The



Step-by-Step Guide with Models", SEAK, Inc. 2002 (hardcopy book) Guide to writing expert reports for internal purposes, accidents, medical, etc. [BaeckerR03a] Baecker, Ron "KMDB1001F: Interacting with Knowledge Media", Lecture/course notes Refers to videos of "Gestures as Data" (handwriting recognition on tablet, sketch recognition), Genesys, Baecker, 1968, and "Gestures as commands", Kurtenbach/Buxton, 1987 [BalakrishnanR02a] Balakrishnan, Ravin and Hinckley, Ken "Symmetric Bimanual Interaction", Proc. ACM CHI2002, CHI Letters 2(1), pp. 33-40 Study of two-handed manipulation, degree of parallelism using two cursors (styli) in two hands to track/dock two objects. Two pens (styli) on a single Wacom Intuos digitizing tablet. multi-touch [BallareDE01a] Ballare, Daniel E. "Supplemental Mouse Button Emulation System, Method and Apparatus for a Coordinate Input Device", United States Patent 6,208,329, March 27, 2001 Mouse emulation on tablet touchscreen: one finger touch is left button, two finger touch is right button, detects double finger by getting strong signal. Other method is double-tap for right button. Cites to Freestyle patent for mouse emulation. [BanerjeeBR01a] Banerjee, Biswa R.; Gladwin, S. Christopher; Maskatia, Arif; and Soucy, Alan "Structure and method for controlling a host computer using a remote hand-held interface device", United States Patent 6,292,181, September 18, 2001 Remote control touchpad for a computer, handwriting recognition performed on host computer. Switch between pen and mouse (writing and pointing) modes by pressing barrel side button on stylus. Compare with Microsoft display/tablet? On-screen icons to select stylus as right/left mouse button input: compare with Freestyle "The Guide"? Cites to Pen Windows: barrel button means erase ink rather than write ink. Compare with Freestyle "erase" button on stylus? [Barcodes03a] Bar Code History Page / The History of Bar Codes "Bar Code History Page / The History of Bar Codes", educ.queensu.ca/~compsci/units/encoding/barcodes/history.html, fetched 2003 History of bar codes back to Wallace Flint 1932, Bernard Silver 1948 [BarkleyJ01a] Salas, Pito; Beir, Jeffrey; Leffler, Melissa; Glenn, McDonald; Kleppner, Paul; Morrissey, Craig; and Tonra, James E. Jr. "Implementation of role/group permission association using object access type", United States Patent 6,202,066, March 13, 2001 Project management and organization system [BarrettGL03] Barrett, Gary L. "Light Transmission and Touch Panels", Touch International, www.touchinternational.com Whitepaper on light transmission through touch panels: 40%, versus 95% for window glass. Describes light transmission for resistive, capacitive, SAW Surface acoustic wave, and infrared touch panel digitizers [BarrientosFA02] Barrientos, Francesca A., and Canny, John F. "Controlling expressive avatar gesture using pen gesture", Proc. 4th Intl. Conf. on Collaborative Virtual Environments, Bonn, Germany, 2002, pp. 113-119 Using pen gestures -- not recognized shapes, but motion to control joints of simulated avatar -- for a virtual reality system. Tailored to hand-held displays, with voice input and one-handed input to control avatar. Users required practice to be able to do pen gestures and interact at the same time. [BartlettJF01a] Bartlett, Joel F. "A Simple CMOS Camera for Itsy", Compaq Western Research Laboratories Technical Note TN-48, February 2001 Add-on camera for Itsy small palm-held/hand-held computer. Video recording at 30 frames/second with VGA. Interpolation/image-recovery to recover color data by averaging pixels which each have a single R/G/B value. [BartlettJF02a] Bartlett, Joel F. "Apparatus and Method for Detecting and Executing Positional and Gesture Commands Corresponding to Movement of Handheld Computing Devices", United States Patent 6,347,290, February 12, 2002 Commands/gestures by rotating entire hand-held device (dynamic or sequences of positions/orientations), using inertial (or accellerometer) sensors. Slide-to-unlock? [BaudischP02a] Baudisch, Patrick; Bederson, Benjamin B.; and Zierlinger, Alexander "Drag-and-Pop: Accellerating Drag-and-Drop on Large Screens", Proc. UIST 2002 Drag-and-drop is direct manipulation, eliminates interaction with (hidden) clipboard for cut-and-past or copy-and-paste. Xerox Liveboard, SmartTech.com Smartboard whiteboard systems. Multi-display whiteboard systems such as DynaWall. Drag-and-pop rubber bands extend towards object from possible targets, only need to drop on nearest rubber band instead of far distance. [BaudischP03a] Baudisch, Patrick; Cutrell, Edward; Robbins, Dan; Czerwinski, Mary; Tandler, Peter; Bederson, Benjamin; and Zierlinger, Alex "Drag-and-Pop and Drag-and-Pick: techniques for accessing remote screen content on touch- and pen-operated systems", Proc. INTERACT 2003, pp. 57-64 user-interface on a large whiteboard touchscreen: when user picks a menu item or icon for an action, automatically move possible targets temporarily towards the stylus/pen so that they user can reach them more easily. Also: electronic video file of demonstration.



[BeatonBF01a] Beaton, Brian Finlay; Smith, Colin Donald; Blouin, Francoise; Comeay, Guillaume; and Craddock, Arthur Julian Patterson "Intelligent Touch Display", United States Patent 6,310,610, October 30, 2001 Transparent graphical user interface tools on touchscreen of smartphone. Press-and-hold of one to two seconds. Analog (resistive sheet?) touchscreen, but can determine pointer size (stylus, finger) from contact area / touch point distribution, or by speed of pressure change. One feature is showing translucent/transparent navigation arrows on small screen all the time, can be enabled when needed (reduces screen real estate needed). [BeatonBF02a] Beaton, Brian Finlay; Smith, Colin Donald; Blouin, Francoise; Comeay, Guillaume; and Craddock, Arthur Julian Patterson "Contextual Gesture Interface", United States Patent 6,340,979, January 22, 2002 Transparent graphical user interface tools on touchscreen of smartphone. Press-and-hold of one to two seconds. Analog (resistive sheet?) touchscreen, but can determine pointer size (stylus, finger) from contact area / touch point distribution, or by speed of pressure change. One feature is showing translucent/transparent navigation arrows on small screen all the time, can be enabled when needed (reduces screen real estate needed). [Beaudouin-Lafon01a] Beaudouin-Lafon, Michel "Novel Interaction Techniques for Overlapping Windows", Proc. UIST '01, Orlando Florida, 2001, pp. 153-154 Show overlapping windows with tabs: user has better overview of windows in context. Compare with PenPoint? Also rotated windows, peeled-back windows to see what is underneath. If Window rotated less than threshold 10 degrees, snaps to orthogonal unrotated perspective. [BeaverK02a] Beaver, Kathleen "JAWS for Windows tutorial", SUNY / Buffalo, Assistive Technology Training Online, USDE# H324M980014 Screen reader, speaks ALT tags/labels on graphics if present. [BeckerCH02a] Becker, Craig Henry; Glass, David Wayne; Hocker, Michael David; McLean, James Gordon; Pickoveer, Clifford Alan; and Winarski, Daniel James "Method and system for variable speed scrolling within a data processing system", United States Patent 6,337,694, January 8, 2002 Scroll faster speed/velocity depending on what is in view (larger objects faster). [BedersonB03a] Bederson, Benjamin B.; Clamage, Aaron; Czerwinski, Mary P.; and Robertson, George G. "A Fisheye Calendar Interface for PDAs: Providing Overview for Small Displays", CHI 2003 Demonstration papers, April 5-10, Ft. Lauderdale Florida, pp. 618-619 Zoom (figuratively) into region or information of interest in a small display: e.g. tap on a date in a calendar month view, get the day information with the rest of the calendar month pushed out to the sides. Mentions work on DataLens and fish-eye techniques back to 1982. [BensonG01a] Benson, Glenn "Computer system for protecting a file and a method for protecting a file", United States Patent 6,301,660, October 9, 2001 Siemens e-license: challenge/response with asymmetric keys (compare with HTTPS/PKI) for DRM rights managements for file access, partial file access, rendering [BerqueD01a] Berque, Dave A.; Johnson, David K; and Jovanovic, Larry "Teaching theory of computation using pen-based computers and an electronic whiteboard", ACM SIGCSE (Computer Science Education) Bulletin, Vol 33, no 3, pp 196-172, 2001 Use of Wacom or Smart Technologies whiteboard in groupware application with network of pen-based computers. Original works cites to 1993. [BiD01a] Bi, Depeng; Cohen, Gary Steven; Cortopassi, Michael; George, Jose T.; Gladwin, S. Christopher; Hsiung, Harry; Lim, Peng; Parham, John Allan; Soucy, Alan Joseph; Voegeli, Derick W.; and Wilson, James Y. "Mouse Emulation with a Passive Pen", United States Patent 6,262,719, July 17, 2001 Emulate two button mouse with single stylus/pen: toggle icon to change state of switch tip from left mouse button emulation to right mouse button operation. Includes displaying on-screen keyboard. [BiD03a] Bi, Depeng "Multi-user on-screen keyboard", United States Patent 6,664,982, December 16, 2003 On-screen keyboard in handwriting recognition tablet devices, devices connect to a server wirelessly (thin client?). On-screen keyboard protected by occlusion window from being overwritten by server display of data. Refers to "shadowing" of keyboard on display: transparent virtual keyboard? [Billinghurst02] Billinghurst, Mark and Buxton, William "Gesture based Interaction: Chapter 14 of "Haptic Input" Draft Documents", www.billbuxton.com Gesture as hand gesture, not as handwriting gesture for pen-computing. Mentions Theremin as gestureinput device (non-computer): File also contains 2009 Draft [BjoerkS01a] Bjoerk, Staffan "The ScrollSearcher Technique: Using Scrollbars to Explore Search Results", Poster at Interact 2001, IFIP TC.13 Conference on Human-Computer Interaction, July 9-13, Tokyo, Japan Show marks in scrollbar for where search terms are found in a document. Compare with information/tags in sliders, esp. Russian patent. [BjoerkS02a] Bjoerk, Staffan and Redstroem, Johan "Window Frames as Areas for Information Visualization", Proc. NordiCHI '02, Oct 19-23, 2002, pp. 247..250



Display shades of color in frame/slider bar of window to show how much document has been changed at that point, readability score of text, tags of different colors for different search terms, etc. Cites to Alphaslider, SeeSoft, Data Visualization Sliders, Scrollsearcher. Compare with information/tags in sliders, esp. Russian patent. [Blickenstorfer02] Blickenstorfer, Conrad H. "Commentary by Pen Computing Magazine's editor-in-chief", Pen Computing Magazine, Issue 47, December 2002 Refers to new generation (2002) of Tablet PCs using ink, not handwriting recognition: compare to Slate note on "deferred recognition" and "ink as a data type" from 1992. [Blickenstorfer03] Blickenstorfer, Conrad H. "Commentary by Pen Computing Magazine's editor-in-chief", Pen Computing Magazine, Issue 50, 2003 Historical summary of pen computing: cites 1914 and 1938 patents, may have been taken from Jean Renard Ward on-line information on "History of Pen Computing": mentions Pen Services 2.0 for Windows95, successor to Windows for Pen Computing [BobrowDG01a] Bobrow, Daniel G; Everett, John O.; and Virros, Andrew "Paper intermedium: using paper to support interactive educational experiences", IEEE Computer, 2001, March, Vol 34 No 3, pp. 52-53 Part of larger article by Hecht on Dataglyph patterns hidden in printed material to encode information. Include data collection or links to further information in the information encoded in the pattern. Compare with Anoto and LiveScribe pen pattern. File includes bio sketch of Daniel G. Bobrow [Bodnar01] Bodnar, Eric O; Lee, Jennifer J; Kahn, Philippe R.; Feague, Roy W; Jorgensen, David E.; and Liu, Gwhoho H. "User interface methodology supporting light data entry for microprocessor device having limited user input", United States Patent 6,232,970, May 15, 2001 six-button menu input system for limited input, by giving most recently used options first: compare to pop-up menus. [BowesJ03a] Bowes, James; Dearman, David; and Perkins Ryan "Transparency for Item Highlighting", Poster Presentation, Vision Interface '03, Dalhousie University, Halifax, June 2003 Compare with greying out disabled menu items, toolbar items: show infrequently used items in transparent color, so not to obscure view of user interface objects underneath. [BraunAC01a] Braun, Adam C.; Beamer, Jonathan L.; Rosenberg, Louis B.; and Chang, Dean C. "Force feedback system including multi-tasking graphical host environment and interface device", United States Patent 6,300,936, October 9, 2001 Force-feedback haptic mouse for windows-based GUI, feedback defined by profiles for display resolution (which may change dynamically), different feedback in different application windows, [BreidenbachJ03a] Breidenbach, Jeff "GlyphChess: What is GlyphChess?", Available at http://www2.parc.com/asd/members/jbreiden/glyphchess.html Test application using printed Xerox glyphchess pattern on the bottom of chess pieces: pieces' position can be read with chess pieces on top of a flatbed scanner. [BrewsterS01a] Brewster, Stephen "Overcoming the Lack of Screen Space on Mobile Computers", Technical Report TR-2001-87, Department of Computing Science, University of Glasgow. Also Personal and Ubiquitous Computing, Vol 6 No 3, May 2002, pp 188-205 Targeting action on small touch screens, lack of tactile feedback on simulated buttons, use auditory sound feedback in the user interface instead to ensure accuracy. [Bricklin01] Bricklin, Dan "Look to the Past to Envision the Future", CACM, March 2001, Vol 44 No 3, page 44 Essay that IT technology / human interface components are are already developed (like the wheel), but the systems of the future will apply them in substantially more sophisticated way. [Bricklin02] Bricklin, Dan "About Tablet Computing Old and New", Available at http://www.bricklin.com/tabletcomputing.htm Historical essay of pen-computing development from about 1990 onward: Slate Corporation, GO Corporation, Microsoft Pen-Windows. Pen-centric UI (term coined by Slate), deferred translation of recognition rather than handwriting recognition in real time. Slate pen-computing applications: At-Hand, PenPower, Day-Timer Pen Scheduler, LooseLeaf Notetaker, PenApps, PenBasic. Mentions Pensoft, Aha!, digital ink, Windows Journal program. Cites Wang Freestyle (synchronized voice and ink recording, no handwriting recognition or gestures) as "totally major", even without handwriting recognition or gestures. [Bricklin03a] Bricklin, Dan "Tablet PC: First Impressions", Available at http://www.danbricklin.com/log/tabletpc.htm States that Tablet PC, "tablet" implies a pen, therefore relating to pen computers / pen computing. Mentions problems of over-filtering / over-smoothing of electronic ink data in pen drivers. [BrighthandNews03] Brighthand News "Graffiti's Dead, Long Live Graffiti 2", www.brighthand.com, January 13, 2003 Press article on replacement of Graffiti with Jot recognition from CIC/Handwriter. Mentions that Grafitti handwriting recognition had to be in special area of screen, Jot/Graffiti2 can be anywhere on screen. Cites Palm/Xerox patent lawsuit



[BrowneHA01a] Browne, Hilary; Bederson, Benjamin B.; Plaisant, Catherine; and Druin, Allison "Designing an Interactive Message Board as a Technology Probe for Family Communication", Tech. Report CS-TR-4284, HCIL2001-20, UMIACS-TR-2001-63, Univ. Maryland, 10-Oct-2001 Shared note-taking electronic message board / shared whiteboard for families, using electronic ink and post-it notes. Cites to InterLiving Project of EU Disappearing Computer Initiative. Prototype uses Wacom or Sony VAIO SlimTop pen tablet. Cites note taking to Lotus telenotes. [BruwerFJ01a] Bruwer, Frederick J. "Intelligent electrical device comprising microchip", United States Patent 6,249,089, June 19, 2001 Using touch sensors (touch pads: see touchscreen; carbon coated membrane type switches) to control a flashlight/light by adding a MMI (man-machine interface: user interface). Touch pads / membrane switches can be sealed, unlike mechanical switches, do not expose user to high current/voltage in mechanical switch. Switches sensed by microchip that adds intelligent functions to otherwise dumb device: emergency flashing, time schedule, etc. See Bruwer 2010 on proximity sensing. [BurrellGL01a] Burrell, Gary L. "Global Positioning Device with Alignment Control of Displayed Information", United States Patent 6,330,149, December 11, 2001 GPS touchscreen device, automatically rotates screen map and information windows for orientation. Indicia on buttons are angled to be usable in either orientation landscape/portrait. [BusinessPeople03a] Business People Magazine "Ascension Technology: They Do the Locomotion", Business People Magazine, March 2003, pp. 2-7 History of Ascension Technology founded by ex-employees of Polhemus, 3D digital tracking systems. Mentions patent disputes between Polhemus and Ascension [ButterA02a] Butter, Andrea and Pogue, David "Piloting Palm", John Wiley and Sons, Inc., 2002 History of GRiD/GRiDPad, Palm Computing / Palm Pilot, Handspring, Zoomer, U.S. Robotics, Psion, Newton, Microsoft WinPad hand-held palmtop portable touch computers / PDAs. Interviews with Jeff Hawkings, Donna Dubinsky, Ed Colligan. Acquisition of GEOS from GeoWorks. Touchdown pen computer with ComPaq. Microsoft Pocket PC, but not Tablet PC. [Buxton01a] Buxton, W. and Fitzmaurice, G.W. "User interface system and method for controlling playback timebased temporal digital media", United States Patent 6,191,784, Feb. 20, 2001 See also http://www.billbuxton.com. Marking/pie menus? [CNET02a] CNET News.com "Tablet PC: Scribbling into the future", CNET News.com, November 8, 2002, available at http://news.com.com Magazines to public on Tablet PC, Brief history of pen computing to 1957 patent, citing interview with tablet visionary Alan Kay (Dynabook) [CappsS01] Capps, Stephen et al "Tactile character input in computer-based devices", United States Patent 6,326,947, December 4, 2001, Assigned to Microsoft Corporation "Tactile" character recognition, user moves finger or stylus across raised guides that are similar to 7segment or 17-segment character displays Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton. Patents cite little or no prior art before 1990: citations to van Rammsdonck, Pencept, and CIC may have been appropriate [Casio01] Casio "Casio E-Pen: Handschrift für den Computer: Kugelschreiber für die Direkteingabe von Schrift auf dem PC", www.plugged.de/news/computing/a1005556375.shtml, dated 12.11.2001 Infrared digitizing pen, infrared transmitter in pen, receiver unit plugs into USB port, resolution 0.25 mm at 200 points/second, handles up to A4 size paper. Handwriting recognition from various packages: Lexicus, PenPower, Paragraph. For use with Casio Pocket PC and Cassiopaia Fiva Subnotebook. Compare with Anoto? [Casio02a] Casio "Casio VDB-1000 wristwatch, detail page (VDB5000(", Collection from Pocket Computing Museum, http://cdecas.free.fr/computers/pocket/vdb1000.htm, 2002 Casio Wristwatch with touchscreen input [Casio02b] Casio "Casio VDB-1000 Virtual Databank Watch: Touchscreen DataBank and Organizer: wristwatch, technical details (VDB5000)", Collection from Pocket Computing Museum, http://cdecas.free.fr/computers/pocket/vdb1000/index.html, 2002 Casio VDB-100 Wristwatch with touchscreen input circa 1991: shows touch screen of menus, no character recognition. File contains additional materials. [CaslonAnalytics03a] Caslon Analytics "Caslong Analytics profile: Digital Paper", www.caslon.com/au/digitalpaperprofile.htm, fetched 2003 Business overview of technologies collectively described as "digital paper" -- electronic paper, electronic ink. SmartPaper/Gyricon paper-like display using microscopic spheres in transparent membrane at Xerox PARC in 1975; Electronic ink paper-like display similar by E-Ink, DataGlyphs and DigiPaper printed pattern similar to Anoto [CertCo02] CertCo, Inc. "CertSigner (TM) FAQs", Available at http://www.certco.com/apki/shtml



Description of Agreement-based / Account-based / Acceptance-based PKI model for CertSigner, a webforms signing and automatic credentialing product incorporating cryptographic authentication and other authentication [ChangKHP01a] Chang, Kenneth H.P. "Encoding and Decoding a Message Within an Image", United States Patent 6,265,398, July 1, 2001 dataglyph encoding of URL and other addresses: glyph pattern involved contrast of pixels: grey on white background, grey on black background: pattern is visible in examples. Compare with Anoto? [ChattyS01a] Chatty, Stephane et al "DigiStrips and related projects / DigiStrips: humaniser les interfaces", http://www.tls.cena.fr/divisions/PII/digistrips/index_e.html, 2001 Part of GRIGRI project: electronic flight progress strips for air traffic control, using touchscreens, limited gesture recognition. [ChenH03a] Chen, Huamin and Mohapatra, Prasant "A Novel Navigation and Transmission Technique for Mobile Handheld Devices", Proc. of GroupWare 92, also Technical Report, CSE-2003-1, Univ. California at Davis, 2003 Alternative to WML and mobile mark-up languages for the days of low bandwidth, small screen PDAs for web browsing: send structural part of web page (layout) to PDA with only first part of the (larger) text sections. User clicks / acts on a piece of text, and PDA sends request for an update to that portion of the HTML. Based on a presumption that the reduced text (by itself) will help by sending less data, and only what the user requests, rather than the whole page. (Also that the reduced text will be intelligible.) [ChenTHT03a] Chen, Timothy T.H.; Fels, Sidney; Min, Saehee Sarah "FlowField and beyond: applying pressuresensitive multi-point touchpad interaction", Proc. 2003 Int. Conf. on Multimedia and Expo, ICME'03, July 2003, pp. 49-52 UI using pressure-sensitive multitouch touchpad digitizer (MTC Express). Details of interpolation and image processing of Taxels (touch pixels): bilinear, bicubic, Gaussian, nearest-neighbor interpolation to get higher resolution than sensors. Other hardware include Lee multi-touch, Fingerworks TouchStream, optical vision tracking, SmartSkin. Multi-touch interactive art installations. [ChenX02a] Chen, Xinlei; Koike, Hideki; Nakanishi, Yasuto; Oka, Kenji; and Sato, Yoichi "Two-Handed Drawing on Augmented Desk System", Proc. ACM AVI 2002, Trento, Italy, pp. 219-222 Two-handed gestures using video tracking. In some gestures one hand supports the other: Close-hand gesture of left hand brings up structural radial menu (pop-up: compare with marking menu) to be selected by a left hand finger. Two-handed rotate gesture, center of circle or point of rotation with left hand. Cites to Krueger/VideoDesk. Compare with Wellner. multi-touch [ChewCH03a] Chew, Chee h.; Bastiaanse, Elizabeth A.; Blum, Jeffrey R.; Keyse, Greg A.; Lui, Charlton E.; Parker, Kathryn L.; Sharpe, Timoth D.; and Zuberec, Sarah E. "Method and Apparatus for providing context menus on a pen-based device", United States Patent 6,664,991, December 16, 2003 pop-up a menu based on a context gesture (any kind of gesture) rather than tapping or clicking on a handle. Cites to Buxton and Agulnick. [Chip03a] Chip Magazine "Testsieger: Sony Ericsson P800 / Ein Gerä - das gesamte Office R380s", Chip Magazin, 31.10.2003 / 14.04.2001 Sony Ericsson P800: smartphone/PDA with camera and touch-screen interface. Ericsson R380s flip phone with keys on outside, touchscreen on inside with Symbion EPOC OS. Physical keys press directly on touchscreen. [ChrisC03a] ChrisC(?) "From Whence It All Began", www.whatisnew.com/article1130.html Various quotes from about 2002/2003 about the history/origin of pen computing, by Bill Gates and others. Imply that original Dynabook was a tablet PC pen computer: note that original sources do not back this up much. [CobbleyDA02a] Cobbley, David A. and Deleeuw, William C. "On-Screen Transparent Keyboard Interface", United States Patent 6,501,464, December 31, 2002 virtual keyboard: "A graphical user interface in the form of a transparent keyboard may be positioned over an existing computer display. The user may input textual data through the keyboard by selecting keys in the transparent keyboard display. The text entry may then appear on the computer display in nontransparent or conventional format." [CompuPen03a] Hopkins Capital Group / Pen2Net, Inc. "CompuPen: Bluetooth Wireless Networked Pen", The Hopkins Capital Group, 2003 Pen2Net CompuPen: ballpoint pen that captures anything written on paper, without specially patterned paper (see Anoto). [ComputerMuseum03] Computer Museum / Mark Greenia "The History of Computing: Encyclopedia of the People and Machines that Made Computer History", http://www.computermuseum.li/Testpage/0000WELCOME.htm Toshiba Dynapad T100X Tablet computer / pen computer (1992), ran Microsoft Windows for Pen Computing, or PenPoint / GO, retail price $3500. [ConferW02a] Confer, William and Chapman, Richard "Handwritten Character Recognition for Cheap Devices",



Technical Report CSSE02-06, Auburn University, Alabama, November 18, 2002 Small, trainable handwriting recognition using chain codes and angles [CongdonB03a] Congdon, Bob "Gesture Recognizer", Available at http://www.bobcongdon.net/blog/2003_09_01_congdon_archive.html Describes early Applicon Ledeen/Teitelman character recognizer of 1973: single-stroke Z as a gesture using the bounds to for a Zoom area -- compare with Recognition Macro gestures of Pencept? [CostanzaE03a] Costanza, E.; Shelley, S.B.; and Robinson, J. A. "D-Touch: A Consumer-Grade Tangible Interface Module and Musical Applications", Proc. Conf. on Human-Computer Interaction (HCI03), Bath (UK), 2003 Tangible objects -- small blocks with recognition marks -- used for a tangible user interface. Musical input by placing blocks on an optical touch surface. [CouturierJM02a] Couturier, Jean-Michel "A Scanned Synthesis virtual intsrument", Proc. NIME-02, 2002 Conf. on New Instruments for Musical Expression, Dublin Ireland, May 2002 virtual musical instrument (like plucked strings) using Wacom tablet, gestures on pressure-sensitive multitouch surface (Tactex), and keypad (for chords). [CouturierJM03a] Couturier, Jean-Michel, and Arfib, Daniel "Pointing Fingers: Using Multiple Direct Interactions with Visual Objects to Perform Music", Proc. NIME-03, 2003 Conf. on New Instruments for Musical Expression, Montreal, 2003, pp 184-187 virtual musical instrument using Pointing Fingers (multi-touch trackers), but any multi-touch input device on the screen may be used (SmartSkin, etc.) [CouvreurJ03a] Couvreur, Julien "Curiosity is bliss: Pen stroke recognition", http://blog.monstuff.com/archies/000012.html Commentary on Rubine as early Unistroke alphabet character recognizer, chain codes, Graffiti implementation in C# programming language [DavisRC02a] Davis, R.C. "Sketch Understanding Demonstration: M.I.T. Project Oxygen (video)", M.I.T. CSAIL Project Oxygen, 2002 demonstration of interactive sketch understanding, with automatic automation of drawing (wheels on sketch of automobile turn, etc.). Compare with DavisRC07, which describes a Wizard of Oz prototype for sketch understanding. [DeAmicisR02a] De Amicis, R; Bruno, F., Stork, A., and Luchi, M.L. "The Eraser Pen: A New Interaction Paradigm for Curve Sketching in 3D", Int'l. Design Conference DESIGN 2002, Dubrovnik, May 14-17, 2002 six degree-of-freedom DOF digitizer for sketching 3D surfaces, using stereoscopic display and 3D digitizer. Permits erasing/deleting of portions of surfaces/lines while sketching (3D erase gesture?) by moving backwards along curve just drawn, integration of editing and creation phases. Earlier references back to 1999. 2D input allows for extruded curves. Accuracy of curve affected by velocity/speed of drawing/sketching motion (slower motion, more control points for the curve to be fitted.) [DeBoorA01a] De Boor, Adam and Eggers, Michael D. "Wireless communication device with markup language based man-machine interface", United States Patent 6,173,316, January 9, 2001 simplified/extended HTML mark-up language for small-screen portable devices. Extensions add navigational aids to touchscreen/small-display interface. [DeHerreraC02a] de Herrera, Chris "Digitizer and Styli - Differentiating the Tablet PC", www.tabletpctalk.com, November 10, 2002 Notes that TabletPC can support multiple digitizers at the same time: reference to integrated resistive touchscreen and electromagnetic digitizer tablet from FinePoint, versus Wacom? [DeHerreraC03a] de Herrera, Chris "Tablet PC Hardware Comparison 2002-2003", www.tabletpctalk.com, 2003 Hardware feature comparisions for Microsoft Tablet PC vendors: almost all use Wacom electromagnetic touchscreen/digitizer, some use FinePoint combined resistive touchscreen. [DeLaHametteP02a] de la Hamette, P.; Lukowicz, P.; Troester, G.; and Svoboda, T. "FingerMouse: A Wearable Hand Tracking System", Proc. UbiComp 2002, 4th Int'. Conf. on Ubiquitous Computing, September 2002, pages 15-16 FingerMouse: optical finger tracker (not actually worn) using two closely-spaced stereo cameras, and presumes that user is holding a pointer finger at approximately the same close distance. Called wearable because it is very low power consumption. [DecurtinsC03a] Decurtins, Corsin; Norrie, Moira C.; and Signer, Beat "Digital Annotation of Printed Documents", Proc. CIKM '03, November 3-8, 2003, New Orleans Electronic ink/Multimedia annotations on paper documents, user can mark area for annotation: multiple layers of annotation for whole document, particular part, finer details. Implemented with Paper++ [DelaneyK01a] Delaney, Kalen ("based on the first edition by Ron Soukup") "Inside Microsoft SQL Server 2000", Microsoft Press, 2001 (hardcopy book) General reference on SQL database system and language as implemented by Microsoft [Dell03a] Dell Computer "Dell Axim X5 User's Guide: Mobile Computer", Dell Computer, 2003 (hardcopy book)



User guide for Dell version of Microsoft Pocket PC with touchscreen and handwriting recognition (tap gesture to recognize electronic ink: compare with deferred recognition and Slate). Soft keyboard / virtual keyboard. [DemartinesP03a] Demartines, Pierre and Seni, Giovanni "Automatically scrolling handwritten input user interface for personal digital assistants and the like", United States Patent 6,661,409, December 9, 2003 Automatic scrolling of handwriting ink in a handwriting area during handwriting recognition: compare with Freestyle? [DenBoerW03a] den Boer, Willem; Abileah, Adi; Green, Pat; Larsson, Terrance; Robinson, Scott; and Nguyen, Tin "Active Matrix LCD with Integrated Optical Touch Screen", Society for Information Display SID 03 Digest, paper 56.3, 2003 TFT LCD display with integrated photosensor optical sensors for touch input: shows XY location of shadow (ambient light) cast by stylus or finger touch. multi-touch. [DentPW02a] Dent, Paul W. "Wireless Subscriber Terminal using Java Control Code", United States Patent 6,418,310, July 9, 2002 Communications device (e.g. smart phone) with Java: keyboard and mouse [DeviM01a] Devi, Prof.M "Basic Windows: Message Queue Events", Available at http://comsci.liu.edu/~murali/win32gui/BasicWindows.htm, fetched 2011 Shows action of WM_RBUTTONUP as posting WM_QUIT to exit current window (as contrasted to other right-click behaviors) [DiazMarinoRA03a] Diaz-Marino, Roberto Arturo; Tse, Edward; and Greenberg, Saul "Programming for Multiple Touches and Multiple Users: A Toolkit for the DiamondTouch Hardware", Proc. UIST '03 Poster/paper to accompany demonstration of applications using GroupLab DiamondTouch Toolkit SDK/APIs. Example SquiggleDraw, using multi-touch inputs from multiple users in drawing. States that DiamonTouch did multiple touches per user. Tracks/identified individual users on each input. [DietzP01a] Dietz, Paul and Leigh, Darren "DiamondTouch: A Multi-User Touch Technology", Proc. UIST '01, November 11-14, 2001 pp. 219-226 Multi-touch digitizer on a table (compare with Entertaible), using separate capacitive coupling for each human user, so that you can tell which user/stylus corresponds to which input. Mentions Human-Guided Simple Search as a prototype groupware application (whiteboard). Also: electronic video file of 2001 demonstration. [DietzP02a] Dietz, Paul H. and Leigh, Darren L. "Multi-user Touch Surface", United States Patent 6,498,590, December 24, 2002 multi-user, multi-touch touchpad/touchscreen, separate (AC) electrostatic/capacitive signals through body of each user, detection is capacitive junctures of X/Y matrix grid. For multiple touches by single user, X/Y array of separate antennas. [DietzP03a] Dietz, Paul and Leigh, Darren "DiamondTouch: A Multi-User Touch Technology", Mitsubishi Electric Research Laboratories Technical Report TR-2003-125, October 2003. Available at www.merl.com Multi-touch digitizer on a table (compare with Entertaible), using separate capacitive coupling for each human user, so that you can tell which user/stylus corresponds to which input. Mentions Human-Guided Simple Search as a prototype groupware application (whiteboard) [DietzP03b] Dietz, Paul; Yerazunis, William; and Leigh, Darren "Very Low-Cost Sensing and Communication Using Bidirectional LEDs", Mitsubishi Electric Research Laboratories Technical Report TR-2003-35, July 2003. Available at www.merl.com Simultaneous use of LED as optical transmitter and as optical receiver/sensor: opto-isolator? LED as photoresistor [DirectMeasurements03a] Direct Measurements, Inc. "Direct Measurements -- Circular SSR circular bar code", www.directmeasurements.com/circularsymbol.html fetched 2003 Circular bar code, no preferred direction, uses arc segment patterns in two concentric "data rings" inside border. Compare with illustrations in Sedaker. [DirectMobile02] DirectMobile.com "Refurbished Pen Computers", http://www.directmobile.com/DHHome/Fujitsu Pen.htm; http://www.directmobile.com/DHHome/Refurb_Nonfujitsu.htm; 2002. Telephone 1-800-687-6543 x 27 (Chris Cole) See also http://www.cadigital.com/1000.htm parent company, Surplus Technology Group, purchased assets of Grid System Inc. After running out of Grid units, went with Fujitsu Personal Systems Inc. product: Ricoh G1200 Windows 95 pen-computer with CD tray product: IBM 730 T Windows 95 pen-computer product: Kalidor K2000 Windows 3.1 (!) pencomputer, waterproof product: Fujitsu Stylistic LT Windows 98 pen-computer product: Fujitsu Stylistic 500 Windows 3.1/95 pen-computer product: Fujitsu Stylistic 1000 Windows 3.1/95 pen-computer product: Fujitsu Stylistic 2300 Windows 95/98/NT pen-computer product: Fujitsu Point 1600 Windows 95/98/NT



pen-computer product: Fujitsu Stylistic 1200 Windows 95/98/NT pen-computer product: Fujitsu Point 510 Windows 95/98/NT pen-computer product: Fujitsu PenCentra CD WindowsCE pen-computer [DolceTechnica03a] DolceTechnica.com "Peter on Tech: An early look at Windows XP Tablet PC Edition 2004, codename "Lonestar"", DolceTechnical.com, Nov. 17 2003 Public demonstration of Tablet PC 2004: context-sensitive syntax rules for recognition of URLs and other formatted text in input fields, electronic ink in Web Browser input through web controls, In-place TIP (Tablet PC Input Panel) with writing pad mode, write-anywhere: see also Microsoft file. [Dolphin01a] www.yourdolphon.co.uk, 2014 "SuperNova Screen Reader - Version History", www.yourdolphin.co.uk, fetched 2014 Dolphone screen reader, has graphics labels / graphic labeling. [DourishP01a] Dourish, Paul "Getting in Touch: Chapter 2 of Where The Action Is: The Foundations of Embodied Interaction", MIT Press, 2001 Review of touch, direct manipulation interfaces, tangible interfaces: cites to Wellner, metaDESK/phicons, [DraperGM03] Draper, Geoffery M. and Egbert, Parris K. "A Gestural Interface to Free-Form Deformation", Proc. of Graphics Interface, Canada 2003, pp 113..120 Gestures for shaping drawings (stretch, twist, size, etc.) [Dynapro01a] Dynapro Systems Inc. "Dynapro Near Field Imaging Touch Screen System integration guide", Dynapro Systems, Inc. Copyright 2001, www.dynapro.com Electrostatic/capacitive touchscreen digitizer, NFI [EChalk03a] Freie Universität Berlin "The E-Chalk Project: Electronic Chalk for the Classroom", http://kazan.inf.fuberlin.de/echalk/docs/2003-brochure.pdf Freestyle-like electronic whiteboard for university teaching: records/streams audio, whiteboard image, video, and any slides etc. for university lectures. [EasttyPC03a] Eastty, Peter Charles; Thorpe, Peter Damien; and Sleight, Christopher "Data processing apparatus utilizing proximity sensing to determine whether user's hand is within a predetermined distance", United States Patent 6,583,801, June 24, 2003 Capacitive touchpad sensor with proximity sensing: when user's hand/finger gets close in distance to an icon or widget object, activate or highlight the object. Compare with context-sensitive cursors and highlighting of objects to show drop locations using proximity in PenPoint? [ElbingKE01a] Elbing, Kristofer E. and Foote, Jonathan T. "Tactile Computer Interface", United States Patent 6,219,034, April 17, 2001 Software to map touch regions of the screen to tactile signals in an actuator in a pointing device (mouse, touch tablet, etc.) based on a profile definition. [ElledgeDD01a] Elledge, Dennis D. "Securing Restricted Operations of a Computer Program using a Visual Key Feature", United States Patent 6,192,478, February 20, 2001 On-screen simulated switch/key to enable an operation: slide-to-unlock? [EmD03a] Em, Davic "Beyond the Tablet PC", byte.com, June 16, 2003 Fujitsu Lifebook P1000: Windows XP Home edition rather than Tablet PC edition: integrated touchscreen, not proximity tablet. Apparently touchscreen used as best alternative to mouse (versus touchpad) to get very small form factor. [EricsonP01a] Ericson, Petter and Edsoe, Tomas "Identification of a virtual raster pattern", WIPO International Patent WO 01/75783 A1, 11 October 2001 Describes Anoto technology for an optical "smart paper" digitizer, using modified virtual raster pattern marks: position determination using fourier analysis [EricsonP03a] Ericson, Petter and Hugosson, Ola "Position Determination", United States Patent 6,470,104, May 27, 2003 Mathematical position-encoding pattern using different sized (large/small) dots on paper etc. for optical stylus digitizer [FaaborgAJ02] Faaborg, Alexander J. "Using Neural Networks to Create and Adaptive Character Recognition System", Research Report, Cornell University, ithaca NY, May 14, 2002 Report on back-propagation neural network: performance recognition accuracy went down when more than 12 characters were in the handwriting test set. Handwriting styles had drastic effects on recognition accuracy. Somewhat unencouraging about neural networks: training of a neural network can be laborious. [FarcyR01a] Farcy, Rene and Damaschini, Roland "Guidance-Assist System for the Blind", Proc. SPIE International Society for Optical Engineering, Vol, 4158, p. 209-214, Jan. 2001 Page 1 of 1 Mobility aids for the blind: Teletact hand-held laser telemeter, Vigitact infra-red proximity/bump-avoidance sensor. vibratory/haptic vibration output better than audio for beginners, not affected by ambient noise environment. Experienced users prefer audio, finer results. [FeinsteinDY02a] Feinstein, David Y. "View navigation and magnification of a hand-held device with a display", United States Patent 6,466,198, October 15, 2002



RotoView (product) tilt user interface for hand-held devices with orientation sensors (gyrocompas, accelerometers). Tilting does a scroll function for small displays. Non-linear tilt profile for fast and slow scroll. Cites to Rekimoto 1996 on Tilting Interface, Rock 'n Scroll by Bartlett. [FelleisenM01a] Felleisen, Matthias; Findler, Robert Bruce; Flatt, Matthew; and Krishnamurti, Shriram "How to Design Programs: An Introduction to Programming and Computing", MIT Press, 2002 Tutorial on software design and design patterns, using the Scheme object-oriented programming language. Intended as software design course for liberal-arts majors. Defines method == procedure. [FinePointInnovations03a] FinePoint Innovations "FinePoint Innovations: MagicPoint pen input digitizers", FinePoint Innovations, www.finepointinnovations.com, 2003 FinePoint electromagnetic digitizer, former engineering team from Kurta/Mutoh America OEM Pen-Based Products group. MagicPoint TV-14 Video Tablet: integrated transparent touchscreen, LCD video display, drivers. [FingerSystem03a] FingerSystem Inc. "FingerSystem i-pen mouse User's Guide", FingerSystem Inc. 2003 Optical mouse in (thick) pen housing: includes handwriting recognition software, shape recognition software riteShape for sketching. Compare with Anoto. Web Memo Plug-in for Internet Explorer for electronic ink annotation on web pages. ritePen handwriting recognition. [Fingerworks03a] Fingerworks Inc. "iGesture Game Mode Guide", www.fingerworks.com Multi-touch Touchpad digitizer tablet with built-in gesture recognition. Gestures for electronic games multiple gestures. multi-finger game gestures include trigger, activate, point and fire gestures. Additional materials in paper file. Shows multi-touch/multi-finger gestures/hotkey mappings for Windows: Editing/Search (cut, copy, search, redo, etc.), File and Window manipulation (up directory, save, open file, maximize/minimize window, etc.), Scroll and Zoom. [Fingerworks03b] ExtremeTech.com "Typing With Gestures on the TouchStream ST", www.extremetech.com, 2003 Review of Fingerworks multi-touch "TouchStream ST" keyboard [Fingerworks03c] Fingerworks Inc. "Multitouch Overview: Frequently Asked Questions", www.fingerworks.com, fetched Feb 15, 2003 Multitouch surface from Fingerworks: multiple-finger gestures, zero-force. Proprietary technology. [Fingerworks03d] Fingerworks Inc. "Fingerworks Support Forrums: Public Info. on Multi-Touch Technology", forums.fingerworks.com, August 15, 2003 Explanation of Fingerworks multi-touch technology: 2D grid permits "paw-print" images of all touching on surface, unile non-scaleable 1-D projection sensor arrangement of touchpad. Refers to Wayne Westerman Ph.D. thesis 1999, or two Fingerworks patents. [FingoldJ03a] Fingold, Jonathan "Persisting Ink with Attached Recognition Data", Microsoft Corporation, MSDN documentation, April 2003 compare with deferred recognition by Slate [FischettiM01a] Fischetti, Mark "Working Knowledge - Touchscreens", Scientific American, April 2001, pp. 102-103 Mentions Elographics resistive-film design 1977, Infrared touchscreen used underwater: cites advantages of touchscreen technologies for electronic voting, but does not mention other security considerations. [FishDE02a] Fish, Daniel E. "Force feedback computer input and output device with coordinated haptic elements", United States Patent 6,377,678, 2002 (see also 6,819,312, November 16, 2004) Combined touch tablet sensor and haptic pixels "haptels". Can show textures, and since combined with touch sensor, only needs to actuate haptel actuators where there is a touch. Compare display portion with optacon? Cites to Kaplow, 4,202,041. Also cites to active multi-touch input surface (AMTIS) by Fish. See also RE42,064. [FitzmauriceG03a] Fitzmaurice, George; Khan, Azam; Pieke, Robert; Buxton, Bill; and Kurtenbach, Gordon "Tracking Menus", Proc. UIST '03, Vancouver BC, Canada, 2003, pp.71-79 Point-and-press with stylus/pen tip: Pop-up tracking menus that follow cursor/stylus, compare with earlier work on marking menus: avoids tool palette round trips to a menu button at a particular location. Bull'seye menus. Dwell behavior optional (gesture). Dwell/press-and-hold: may have second threshold in "tracking boundaries", or in "drags in the outside region" to engage a function. Movement of pen without contacting edge [FlahertyG03a] Flaherty, Gopi "Gesture Toolkits", Carnegie-Mellon University, UI Course 830, Technical Presentation April 15, 2003 Overview of gesture-based interfaces: Describes retro-fitted gesture interfaces on exiting applications a "Boring Gesture Systems", gesture-based user interfaces will require new forms of interaction: PenInputPanel of Microsoft TabletPC Ambiguity of recognition, and of segmentation of handwriting/sketch strokes [FleckDC02a] Fleck, David C. "Digitizer system with on screen cue indicative of stylus position", United States Patent 6,473,073, October 29, 2002



Change cursor to indicate where a stylus is in "hover" over a fixed menu on a hardware tablet surface (versus displayed menu) [Fraunhofer01a] Fraunhofer-Institut für Rechnerarchitektur und Softwaretechnik "Fraunhofer-Institute Jahresbericht 2001: Touchglobe und Touchpad", Fraunhofer-Institut, Berlin Touchglobe and Touchpad: multi-touch input devices using sensor skin of piezo-electric polymers that can be formed on any surface: voltage potential from pressure detected by grid-structured foil [FreedomScientific03a] Freedom Scientific (formerly Hunter-Joyce) "JAWS for Windows 5.0 installation file", www.freedomscientific.com, 2003 (PDF Portfolio) installation file .exe for JAWS Screen Reader for windows version 5.0, with extracted Help files and manuals (also as PDFs). Compare with outSpoken, VoiceOver for Mac/IPhone, WindowEyes. Accessibility for visually impaired: keyboard navigation only, multiple voices, unrecognized window classes can be assigned to be handled as different (known) class. [FreedomScientific03b] Freedom Scientific (formerly Hunter-Joyce) "JAWS for Windows 5.0 screen reader Features and Enhancements: label graphics", www.freedomscientific.com, 2003 JAWS screen reader: automatic language detection in HTML; Virtual cursor / place marker; personalized settings per web site; Navigate directly to UI elements by type (next button, etc.); labeling of graphics (example: check/unchecked graphics can be labeled to be spoken) [FriedI02] Fried, Ian "Microsoft turns a new page on Mira tablet", ZDNet News, march 13, 2002 Mira tablet: display with touchscreen/digitizer, wireless link to PC -- not a Tablet PC, separate hardware. Demonstrated by Bill Gates at Consumer Electronics Show [FriedlandG01] Friedland, Gerald; Knipping, Lars; Rojas, Raul "E-Chalk Technical Description", Technical Report, Freie Universität Berlin, Institut für Informatik, August 28, 2001. Available at www.match.tu-berlin.de Research group investigating various whiteboard systems for university instruction and remote education, using commercial products. Points out useful things of traditional chalkboard/whiteboard: high contrast, large size, easy to use. Handwriting recognition to enter mathematical equations for plotting easier than keyboard during a lecture. Termination of handwriting is a down-left gesture / new-line symbol. Clients software is java-enabled browser (downloaded Java client.) Mimio (from Virtual Ink) ultrasonic digitizer pen not usable due to delays (from filtering to remove digitizer jitter). Wacom tablet with LCD projector: disadvantage is visual parallax with rear-mounted digitizer. Numonics and Hitachi digitizer whiteboards best. Cites Xerox Park Ubiquitous Computing whiteboard; Zombieboard (camera reading whiteboard, electronically subtract image of human). M.I.T. Transboard: electronic ink for collaboration without handwriting recognition. Electronic whiteboard with voice (similar to WANG Freestyle, but real-time?) by Thomas Ottman of U. Freiburg "Extended Whiteboard". eBeam whiteboard similar to Mimeo, but records voice. [FroebaB02] Froeba, B. and Kueblbeck, C. "Rapid hand gesture detection for human computer interface", CGIM: Proc. IASTED International Conference on Computer Graphics and Imaging, Anaheim, California 2002 (abstract only) Fraunhofer Institute: video recognition of where hand gestures are pointing (by moving a mouse cursor) and simulating mouse clicks. Thus would be compatible with standard computer. [Fujitsu01a] Fujitsu PC Corporation "Fujitsu Stylistic 3500 Technical Reference Manual", www.fujitsupc.com, 2001 Fujitsu Stylistic 3500 tablet computer / pen computer. Resistive-film touchscreen (no proximity/hover), optical liquid between two layers of digitizer (sic). Documentation refers to side-button on stylus, instead a right-mouse-button hotpad icon just off screen (compare with The Guide). One user-definable off-screen hotpad, generates Windows messages in OS via DLL. Windows98 with Windows Pen Services 2.0 (formerly known as PenWindows) with CIC HRS 5.0.5 HandWriter recognition. Also includes CIC PenX 2.02. Both with on-screen virtual keyboards, handwriting pad. CIC PenX has circle-letter gestures like Pen Windows 1.0, copy gesture generates control-X windows keyboard short-cut, etc. [Fujitsu01b] Fujitsu PC Corporation "Fujitsu Stylistic 3500 Pen Tablet User's Guide", www.fujitsupc.com, 2001 Fujitsu Stylistic 3500 pen tablet computer. Off-screen hotpads, similar to off-screen hotpads for The Guide. Resistive-film digitizer, warranty on scratches on display invalidated if used without a "screen protector". [Fujitsu02a] Fujitsu PC Corporation "Fujitsu and the Tablet PC", www.fujitsupc.com, 2002 Position paper by Fujitsu on Microsoft's Tablet PC initiative: handwriting recognition accuracy barely improved since 1989 and first pen tablets (sic). Compares Tablet PC hardware compatibility requirements with pre-existing Fujitsu pen tablets: "hover"/electromagnetic digitizer versus resistive "passive" digitizer tablet. Notes Microsoft's emphasis on electronic ink / digital ink instead of handwriting recognition. Overall cautious about Tablet PC initiative. [FukuchiK02a] Fukuchi, Kentaro and Rekimoto, Jun "Interaction Techniques for SmartSkin", Proc. UIST '02, Demostration Compantion, pp. 49-50 User interface gestures for SmartSkin multi-touch capacitive tablet: has hardware illustration of SmartSkin mesh sensor. Two-finger to rotate an object, two finger pinch/zoom, two hand for rotating/moving two



grapical objects at the same time. Also shows swepping multiple objects ("marble market") with two forearms/hands. Compare with Kruger videoplace? [FukuchiK03a] Fukuchi, Kentaro and Rekimoto, Jun "Massively Parallel Manipulation Techniques with SmartSkin", Proc. FIT 2003, September 2003 (in Japanese) Multi-touch (actually, full-contact) user interface technique using Rekimoto SmartSkin capacitive table: user can swepp objects together with whole hand or whole arm, simulated repulsive force gradient to bulldoze objects. [FukumotoM01a] Fukumoto, Masaaki and Sugimura, Toshiaki "Active Click: Tactile Feedback for Touch Panels", Proc AHI 2001, 31 March, 2001, pp. 121-122 Haptic transducer for adding click/vibration haptic feel to a touchscreen or touch panel digitizer. Refers to UnMouse by MicroTouch as using whole touch panel as a large push switch that can be added to any touch panel/touchscreen. Physical click sensation (versus vibration) particularly helpful in noisy environments, where a simulated audible click cannot be heard. Mounting transducer on rear of hand-held PDA reduces power used, because tactile feedback conveyed to contact position of grasping hand. [Gaca02] Gaca, Christian "Virtuelle Maus und Tastatur", Article published at www.philipkdick.de, October 2002 Citation to Fingerworks multi-touch input as similar to in-air gestures shown in movie "Minority Report", based on Philip K. Dick novel [GatesB01a] Gates, Bill "Bill Gates, Keynote Speech, COMDEX 2001, Las Vegas", KNNArchive, available on youtube.com 2014 www.youtube.com/watch?v=dXTZxX-XECQ (excerpts) Bill Gates Keynote speech COMDEX 2001. Show Tablet PC products from multiple vendors: notebook convertible TabletPC, switches from landscape to portrait. Portfolio (slate). Small form factor (compare with UMPC and OQO?). [Gateway02a] Gateway Computers "Microsoft Windows/XP Tablet PC Edition convertable laptop circa 2002", Gateway Computer (physical device) Tablet PC convertible: obtained as working device, may need stylus. [GeaghanBO03a] Geaghan, Bernard O. Moshrefzadeh, Robert S. Cordeiro, Craig A. Hagermoser, Edward S.; Hart, James J.; Hatin, Paul M.; Hauck, Karl P.; Peterson, Richard A. Jr.; Schulz, Stephen C.; and Taylor, Gordon F. "Touch Panel System and Method for Distinguishing Multiple Touch Inputs", United States Patent Application 2003/0063073 A1, April 3, 2003 On a single-touch touchpanel (e.g. resistive, acoustic, etc.) determine two touches (double multi-touch) by inferring the position and timing of the overlapping second touch by the rate of change of the input. [GerpheideGE02a] Gerpheide, George E. "Apparatus and Method for Tactile Feedback from Input Device", United States Patent 6,473,069, October 29, 2002 Haptic feedback output to touch-sensitive tablet surface indicating where user is touching on the surface, so that user need not look at the display. Drag-extend gesture using different portions of the display. [GerpheideGE03a] Gerpheide, George and Taylor, Biran "Touchpad having Find and Coarse Input Resolution", United States Patent Application 2003/0210286, November 13, 2003 Circular (arctuate) gestures, large diameter for coarse input value, small diameter for finer adjustment. Direction of circular motion (clockwise / counter-clockwise) determines whether valuator is increased or decreased. Figure-S motion also accepted for up/down. [GettysJ01a] Gettys, Jim and Packard, Keith "The X Resize and Rotate Extension - RandR", 2001 USENIX Annual Technical Conference Boston, Massachusetts, USA June 25-30, 2001 X-Windows window manager extension for rotating orientation of display (landscape/portrait) in particular for small portable PDA handheld computers. Refers also to accelerometer gestures ("throw") from one computer to another. Cites to Itsy. [GillespieDW02a] Gillespie, David W.; Allen, Timothy P.; Wolf, Ralph C.; and Day, Shawn P. "Object Position Detector with Edge Motion Feature and Gesture Recognition", United States Patent 6,380,931, April 30, 2002 Capacitive grid touchscreen using interpolation for fine position determination, recognition of double-tap gestures, others, based on pressure (contact area of finger) and time. Cites to several earlier capacitive grid touchscreens. [GillespieDW02b] Gillespie, David W.; Allen, Timothy P.; Wolf, Ralph C.; and Day, Shawn P. "Object Position Detector with Edge Motion Feature and Gesture Recognition", United States Patent 6,414,71, July 2, 2002 See 6,380,931: Capacitive grid touchscreen using interpolation for fine position determination, recognition of various drag gestures. [GoldbergD02] Goldberg, David; Want, Roy; Weiser, Mark D. "Rotationally Desensitized Unistroke Handwriting Recognition", United States Patent 6,366,697 B1, April 2, 2002 Unistroke single-stroke character recognition. Cites Gabelsberger Shorthand and Organk PenPut character recognition. Has very long recitation of other publications, which appear to be taken from this bibliography. Also cites responses in Xerox v. 3Com patent lawsuit: the Unistroke case concerning Palm? Earlier filings from 1993 abandoned?



[Golem03] Golem.de "Gestenbasierter PointScreen nutzt Körper als Eingabemedium", www.golem.de/0310/27910.html Gesture-based PointScreen uses human body as input medium / device Gesture input capacitive whiteboard from Fraunhofer Institute, technology similar to Theremin electrostatic device [GonzalezR01a] Gonzalez, Roberto "Access without limits: An all new approach to screen access for the blind and visually impaired", Proc. CeSun 2001 Conf. Paper on outSPOKEN World1 screen reader, developed specifically for GUI interfaces. [GoodL02a] Good, Lance and Bederson, Benjamin B. "Zoomable User Interfaces as a Medium for Slide Show Presentations", Information Visualization Vol 2002, No 2, pp. 35-49 PhotoMesa semantically zoomable user interface (ZUI) for viewing photographs on small-format PDA display. Cites to Pad++ etc. [GordonGB02a] Gordon, Gary B. and Wallace, Hugh "Impedance Sensing Screen Pointing Device", United States Patent Application 2002/0130273, September 19, 2002 Pointing device using a single finger: basically a small patterned touch tablet with electrostatic sensing, fine enough to detect image of ridges of fingerprint on finger, and correlate images to see how the fingertip is moving. [GouldEJ03a] Gould, Eric Justin; Risnadi, Paulus W.; Ho, Tina Chia; and Wilkins, Todd "Virtual Force Feedback Interface", United States Patent 6,583,782, June 24, 2003 Move cursor with stylus/touchscreen/tablet different distances depending on what the cursor is over: e.g. on a map, different slipperyness if over simulated water, slope, sand oil slick, boat in water [GoyinsGS01a] Goyins, Gregg S. and Resman, Mark F. "Palm Pressure Rejection Method and Apparatus for Touchscreens", United States Patent 6,246,395, June 12, 2001 Divide touch-sensitive tablet into zones, so can do palm rejection. Detect touches in different zones simultaneously for multiple simultaneous activation points (multi-touch): sort them out by actual activation time to determine which inputs were unintended. Dividing into zones can be done algorithmically by only scanning a portion of the tablet. [GrassetR02a] Grasset, Raphael and Gascuel, Jean-Dominique "MARE: multiuser augmented reality environment on table setup", Proc. SIGGRAPH 02, p. 213 Tabletop mix of real and virtual objects. Real objects examples are pen (both physical ink, and electronic stylus for interacting with virtual objects), laptop, cup, etc. Prototype uses 6DOF Wacom tablet with stylus. Dragging/lifting gestures and unistroke gestures. Magnetic trackers on physical objects. [GrossmanBM02a] Grossman, Beertrand M and Pickover, Clifford Alan "Video display and selection on a graphical interface", United States Patent 6,335,737, January 1, 2002 window management: show windows as rotated 3D projections (sideways) on a 3D carousel image, user can scroll/flick/rotate carousel image to see application windows [GuerreroJF01a] Guerrero, Jaime F. "Method and Apparatus for Controlling the Display of Hierarchical Information", United States Patent 6,236,400, May 22, 2001 GUI for scrolling through hierarchical/tree structured information display [GuhaA02] Guha, Angshuman, Haluptzok, Patrick M., and Pittman, James A. "Handwriting and Speech Recognizer using Neural Network with Separate Start and Continuation Output Scores", United States Patent 6,393,395, May 21, 2002 Time-delayed neural network TDNN for handwriting/gesture and speech recognition. Refers to dynamic time warping applied to lexicon dictionary, looking for least-cost / highest-probability (plausibility) path through a set of words. Important feature is points with high change in derivative of position i.e. points of high curvature (corners): compare with feature extraction of systems such as CIC and PenPad, IBM elastic matching. Some reference to the segmentation problem (cutting electronic/digital ink into strokes). [GuimbretiereF01a] Guimbretiere, Francois; Stone, Maureen; and Winograd, Terry "Stick it on the Wall: A Metaphor for Interaction with Large Displays", Submitted to SIGGRAPH 2001, available from the authors on-line FlowMenu, variation on pop-up pie menus / marking menus on a large wall display, flowmenu brought up by barrel button on chalk/stylus: menu selections can be multiple passes through section of single pie menu. Describes large display with marking (pie) menus as "digital wallpaper". Hierarchical menu effectively done in one stroke. Handwriting recognition on graphical sheets, electronic tracing paper for drawing. Items on display (desktop) may be minimized by moving to top of display, where they become thumbnails but remain active. [GuimbretiereF01b] Guimbretiere, Francois; Stone, Maureen; and Winograd, Terry "Fluid Interaction with Highresolution Wall-size Displays", Proc. UIST '01, November 11-14, 2001 pp. 21-30 Interactive Mural, large whiteboard tablet display: prototype used back-projection display. FlowMenu: popup menu appears at pen location when use presses stylus pen, so no UI clutter on screen. Automatic zooming of virtual writing/drawing sheets when moved into working area (ZoomScape). Recognition (Calligrapher) performed on all electronic ink strokes, showing in lower left of display. Logging of previous



state (compare with Wang Freestyle "Forever Desktop"). Cites to PenPoint, Momenta, Palm, marking menus. [GuimbretiereF03a] Guimbretiere, Francois "Paper Augmented Digital Documents", University of Maryland Technical Report CS-TR-4470, 2003. Also UIST '03 PADD / Paper Augmented Digital Documents. Stroke collector for ink annotation applications on paper documents using Anoto digitizer pattern: also cites to Xax, DigitalDesk, Ariel, and others. [GuimbretiereF03b] Guimbretiere, Francois "CMSC 838G Course Notes: People Paper and Computers", Univ. of Maryland, 2001/2003, www.cs.umd.edu Course notes, tablet and electronic paper computing: tabletop systems, Anoto pattern paper, bimanual/two-handed interactions, tactile/haptic feedback, tablet PC, Wang FreeStyle electronic ink annotation. Cites to FingerWorks and SmartSkin for multi-touch for bi-manual/multi-touch. Recal system for aotumatci indexing of stroke, audio, voice notetaking. [HITLab03a] Human Interface Technology Lab "HITLab Publications: Virtual Retinal Display", www.hitl.Washington.edu/pubs/hitlpub.php Publication list for virtual retinal display: projection directly onto human retina. Primary focus is as accessibility technology for visually impaired. [HP02a] Hewlett-Packard "hp Jornada 920 series wireless digital assistant user's guide", hp.com, fetched 2010 Windows Mobile PAD and cell phone: slide-to-unlock in hardware mentioned on pages 10, and 31 (for the battery) [HalseyE02a] Halsey, Eugene IV and Getz, Catherine A. "Reduced contrast improved transmission conductively coated transparent substrate", United States Patent Application Publication 2002/0086188 Transparent conductive grid tablet digitizer. Silicon dioxide covers non-conducting areas to reduce visibility of pattern. Mentions dummy features (for optical fill-in), transparent grid, near field capacitive touchscreen. [HamburgenWR01a] Hamburgen, William R.; Wallach, Deborah A.; Viredza, Marc A.; Brakmo, Lawrence S.; Waldspurger, Car A.; Bartlett, Joel F.; Mann, Timothy; and Farkas, Keith I. "Computing Practice: Itsy: Stretching the Bounds of Mobile Computing", IEEE Computer, April 2001, pp. 28-36 Embodied user interface: Hand-held device with accelerometer in a display: user input is by gestures moving device: pitch, roll, slosh, snapping, tapping, fanning. Device automatically changed from portrait to landscape mode based on how it was held: compare with iPhone accelerometer. Cites to earlier work on accelerometers, graspable user interfaces. Speech output (DECtalk), TalkSoft speaker-independent smallvocabulary speech recognition used in multimedia e-mail user interface. [HansenJP03a] Hansen, John Paulin; Johansen, Anders Sewerin; Hansen, Dan Witzner; Itoh, Kenji; and Mashino, Satoru "Command Without a Click: Dwell Time Typing by Mouse and Gaze Selections", Proc. INTERACT '03, pp. 121-128 Dwell time using human gaze/eye-tracking (accessibility) compared with mouse clicking using an on-screen predictive keyboard. Mentions adaptive dwell time, shorter time for more skilled users. Adjustable pressand-hold as "dwell time" [HaraK03a] Hara, Kenji "GTEdit: A Gesture-based Visual Programming Environment for Teaching Lisp", Senior thesis, U. Tokyo, Department of Information Science, February 12, 2003 Handwriting/gesture recognition by nearest-distance template matching -- compare with elastic matching? Editing of Lisp programs by editing node/tree structure of program (LISP): compare with Plamondon? GTEdit: Gesture-based Tree Editor. [HaradaT03a] Harada, Tomoki "Input Processing Method and Input Processing Device for Implementing Same", United States Patent 6,657,615, December 2, 2003 Different mouse input operations simulated based on touching time on a touch panel. Direction of scrolling depending on press-and-hold time. [HarrisonBL01a] Harrison, Beverly L.; Buxton, William A.S.; and Zhai, Shumin "Graphical user interface with antiinterference outlines for enhanced variably-transparent applications", United States Patent 6,317,128, November 13, 2001 Touchscreen/mouse GUI with transparent/translucent widgets (menus, tools) etc. Outlines on translucent elements make them easier for user to see. [HawkinsJC01a] Hawkins, Jeffrey Charles "Keynote Address, Comdex 2001", www.waystation.net/stillman/ComDexReporting/Comdex2001 Cites tablet computer as starting in 1989, focus includes portability, not just handwriting. WinPad as Microsoft effort *after* Windows for Pen Computing [HawkinsJC02a] Hawkins, Jeffrey Charles; Sipher, Joseph Kahn; Marianetti II, Ron "Multiple pen stroke character set and handwriting recognition system with immediate response", United States Patent 6,493,464, December 10, 2002 Refers to single stroke glyphs used as modifiers to characters already recognized, or to next stroke: e.g. to



make be recognized as a punctuation mark; presented as alternative to Unistroke recognition, especially for single-character (palmtop) input with limited writing area -- Unistroke, Graffiti Has extensive bibliography, including at least one reference to an internal/non-public (Litvin?): appears to have numerous references taken from this bibliography. Cites to PenDOS, Linus, Blatt gestures for spreadsheets at Wang in 1989. File has notes on Unistroke. [HayesJJ01a] Hayes, John Joseph Jr. and Thornson, Curtis Waynes "Radiotelephones having contact-sensitive user interfaces and methods operating same", United States Patent 6,278,888, August 21, 2001 Smartphone with graphical objects activated by input via contact-sensitive transducer (resistive, electrostatic, or acoustic touchscreen digitizer, or touch strip), momentary number of contacts during time interval (e.g. double-tap?), moving contact (for scrolling) [HechtDL01a] Hecht, David L. "Printed Embedded Data Graphical User Interfaces", IEEE Computer, March 2001, pp 47-55 DataGlyphs: PEDGUI (Printed embedded data graphical user interfaces). Information encoded on the pixels in a passive printed -- or also active -- display can be read by a stylus optical sensor. Refers specifically to "address codes" embedded in a pattern, such that the sensor can read position (like a digitizer?) from the pattern underneath: uses term "address carpets": contrast with Sekendur, Anoto. "paper intermedium" by Bobrow et al. MagicLens (which see) on printed documents: encyclopedia pictures, etc. [HechtDL01b] Hecht, David L.; Jared, David A.; Flores, L. Noah; Stearns, Richard G.; and Chang, Kenneth H.P. "Glyph address carpet methods and apparatus for providing location information in a multidimensional address space", United States Patent 6,327,395, December 4, 2001 Self-clocking glyph codes: Encoded pattern (modified stipling, small diagonal marks in two orientations for 0/1) in printed material, optical image capture pen reads patter, takes action on decoded value. Cites to Anoto, Rekimoto pick-and-drop [HeutschiT02a] Heutschi, Theodor "Electronic Device, Preferably and Electronic Book", United States Patent 6,335,678, January 1, 2002 Electronic book / tablet computer / laptop (?) with touchscreen or stylus input, wireless access and video camera [HimbergJ03a] Himberg, Johan; Kaekkilae, Jonna; and Maentyjaervi, Jani "On-line personalization of a touch screen based keyboard", Proc. IUI '03, January 12-15, 2003, Miami, Florida, pp. 77-84 For virtual/on-screen keyboard, automatically shift/adjust fine positions of keys based on frequency and patterns of use. Consistency based on user handedness and hand extensions. Basically a slight shift in target boundaries to match user's consistent offsets. Compare with stroke/ink targeting in PenPoint/PenApps/W4PC. [HinckleyK01a] Hinckley, Kenneth Paul "Technique for implementing an on-demand display widget through controlled fading initiated by user contact with a touch sensitive input device", United States Patent 6,333,753, December 25, 2001 Microsoft assignee, abstract says "Tool Glass" UI. Semi-transparent Tool window is blended in when nonpreferred hand/finger touches touchscreen, fades out when non-preferred finger removed, so that tool window does not clutter/obscure when not used / on-demand. [HinckleyK02a] Hinckley, Ken "Input technologies and techniques", "The human-computer interaction handbook", pp. 151-168, 1992, ISBN 0-8058-3838-4 Survey of input technologies: includes multi-touch, touchscreen, gestures. [HinckleyK03a] Hinckley, Ken "Synchronous Gestures for Multiple Persons and Computers", Proceedings of UIST 2003 Symposium on User Interface Software and Technology Gesture of tapping two tablets (Tablet PCs) with accelerometers together to join them into one extended virtual display: synchronous gesture refers to gesture involving two separate devices [HiramatsuK02a] Hiramatsu, Kazunori "Coordinate Input System and Display Apparatus", United States Patent 6,339,748, January 15, 2002 Rear-projection display touchscreen using image rays -- looks similar to display scan for CRT. Infrared light projected on rear of screen, passive optical stylus with a reflective tip (in infrared), CCD camera to detect reflection, calculate position coordinates. Not sure why it needs a CCD camera, if this is projecting a ray, seems like position could be determined only from timing. Option of multiple infrared rays. Spot ray projector for infrared and projector for visual image at right-angle incident angles to reduce interference. [HirshbergD03a] Hirshberg, David "Multifunctional Keypad on Touch Screen", United States Patent 6,597,345, July 22, 2003 Telephone-like virtual keypad on touchscreen, direction of stroke from key determines character/function input. Compare with Buxton keyboard? [History.Handy.Ru01] History.Handy.Ru "AT&T EO Communicator 440/880", http://history.handy.ru/museum/eo.html Museum article on PenPoint, EO Personal Communicator from 1991, 1993 (in Russian)



[HopkinsM03a] Hopkins, Mark "Disabling Press and Hold in Applications Written for Tablet PC", msdn.microsoft.com remove property MicrosoftTabletPenServiceProperty from dialog window on Tablet PC to disable press-andhold gesture, because it can be confused with mouse hold on button, such as for audio recording -example of conflict between mouse and stylus input , or of different behavior for press-and-hold (e.g. audio recording) [HoshinoT02a] Hoshino, Takeshi; Yokoyama, Yoshimasa; Minemoto, Takeshi; Aoki, Masahide; Osawa, Atsuo; and Umeharda, Kunio "Display operation method to change the number of images to be displayed and to independently change image direction and rotation of each image", United States Patent 6,396,506, May 28, 2002 Rotate windows showing image (e.g. of a menu) to be right-side-up (portrait/landscape) for multiple users surrounding a display surface table [HourcadeJP02a] Hourcade, Juan Pablo; Bederson, Benjamin B.; Druin, Allison; and Taxen, Gustav "KidPad: Collaborative Storytelling for Children", Demonstration. Proc. CHI 2002, April 25-25, 2002, Minneapolis, MN, pp. 500-501 KidPad: drawing program for children using graphical tools, tool boxes instead of graphical menus or simple icons. Tools can be picked up in GUI. [Hullender03] Hullender, Gregory and Gounares, Alexander "Ink Gestures", European Patent Application EP 1335272, United States Patent 6,938,222 B2 August 30 2005 Ink gestures recognized using Chebychev Polynomials and Bayes net. Refers to an ink gesture being included in a stroke: gesture is part of a larger stroke? Examples include gestures similar to PenPoint "flick" gestures. Claims specific to Chebychev polynomials used in recognition. Compare with "unimplemented glyphs" in Microsoft Tablet PC. [Hullender03a] Hullender, Gregory and Gounares, Alexander "Ink Gestures", United States Patent Application 2003/0156145/A1 Ink gestures recognized using Chebychev Polynomials and Bayes net Refers to an ink gesture being included in a stroke: gesture is part of a larger stroke? [HwangF01a] Hwang, Faustina; Keates, Simeon; Langdon, Patrick; Clarkson, P. John; and Robinston, Peter "Perception and Haptics: Towards More Accessible Computers for Motion-Impaired Users", Proc. ACM PUI, Orlando Florida, 2001 Haptic feedback used in pointing/mouse with motion-impaired users, using Logitech Wingman forcefeedback mouse generating both vibration and force-feedback. Gravity well around pointing targets helpful, velocity and acceleration damping. [IBM01a] IBM "Ink Manager SDK for ThinkPad TransNote "tablet pc"", http://www.ibm.com Announcement of ink-manager SDK in Java and C++ for electronic ink time-stamped ink strokes, ink annotation, character/handwriting recognition. Was announced about the time of PenWindows, PenPoint, before PenDOS from CIC. Note that "ThinkPad"in 1991 referred to pen PCs, not to generic laptops. [IBM01b] IBM "ThinkPad TransNote", http://www.ibm.com ThinkScribe digital notepad: separate digitizer, combined with ThinkPad Tablet PC, sold as TransNote product. Thus two digitizer tablets? [IBM01c] International Business Machines "Ink Manager Pro 1.0 and ThinkScribe Digital Notepad User's Guide", IBM Corp. Electronic ink / ink editing applications: My Ink Files, Ink Viewer, Ink Clipboard, note-taking with Lotus Notes and Microsoft Outlook. Mentions Graphics Tablet Mode, operates as absolute-position mouse input for pen-enabling via mouse emulation. Uses separate TransNote/ThinkScribe digital notepad for capturing ink and later transfering to PC -- compare with Anoto application. [IRIS01a] I.R.I.S. "IRISPen II (Executive) User's Guide", I.R.I.S., 2001 Hand-hold OCR scanner, includes handwriting recognition (numerals/digits only), text-to-speech voice output, bar-code scanner. Text recognition uses syntax pattern recognition for specific languages. Accessibility aid? Contrast sensor with Anoto. [Iggers02] Iggers, Bryan / Bryan's Tech Corner "Bryan's Tech Corner for the T100X", http://www.geocities.com/SiliconValley/Vista/4927/t100x_bigs.html Technical information, software, support on 1997 Toshiba T100X pen-windows pen-computing tablet computer [IlinskiR03a] Ilinski, Roman "Interface with pre-typing visual feedback for touch-sensitive keyboard", Proc. CHI 2003, April 5-10, 2003, Ft. Lauderdale, pp. 750-751 (short talk) Touch-sensitive (touchpad) keyboard cover, cover is touch-sensitive but shaped/contoured for the separate character "keys" [Immersion02a] Immersion Corporation "MicroScribe G2 User's Guide", Immersion Corporation, San Jose, California, 2002 Electromechanical 3D digitizer, desktop sized. Software drivers for DOS and Windows.



[Immersion02b] Immersion Corporation "Immersion TouchSense-enabled computer peripherals", Immersion Corporation, San Jose, California, 2002 List of third-party products using Immersion haptic feedback components, touchpad sensors: HP Force Feedback Web Mouse, Logitech iFeel Mouse, Saitek Touch Force Optical Mouse, Kensington Orbit 3D trackball. [Immersion02c] Immersion Corporation "Immersion MicroScribe-3D and 3Dl desktop digitzing systems data sheet ", Immersion Corporation, San Jose, California, 2002 Data sheet for mechanical-linkage 3D digitizer, for taking data for 3D modeling software (Solidworks, AutoCAD, Rhino3D etc.) Also used as 3D "mouse" to control camera views, define animation trajectories. (Would need 6DOF input?) [InoueH01a] Inoue, Hisashi; Nagayama, Keiji; and Yatsugi, Tomishige "Apparatus for Recording and Reproducing Digital Image and Speech", United States Patent 6,226,449 B1, May 1, 2001 Recording and displaying videos and speech on a hand-held device. Compare with video editing with Buxton/Kurtenbach? [Intel02] Intel Corporation "Intel Mobile Platform Vision Guide for 2003", Intel Corporation technical document, September 2002 Refers to Slates and Handled devices using stylus-based input (pen-computing), but not covered in this guide [InterLink03a] InterLink Electronics, Inc. "ePad from InterLink Electronics, electronic signature solutions", InterLink Electronics, Inc. 2003 Electronic signature touchpad, cites to ESIGN Act 2000, Model Notary Act 2002, HIPAA and Patriot Act requirements. See also CIC "sign-it" software. [Interaccion01a] Gonzalez, Julio Abascal; Penalvo, Francisco Jose Garcia; and Gonzalez, Ana B. Gil, eds. "Interaccion 2001: Segundo Congreso Internacional de Interaccion Persona-Ordenador", Conference Proceedings, May 16-18, 2001, Universida de Salamanca, Salamanca, Spain (In Spanish)Conference on user interface future, includes review articles on touchscreens (pantallas tacti'les), tablet/pen computers (tabletas), electronic whiteboards (pizarras electro'nicas), ubiquitous computing (computacio'n ubi'cua), digitizer tablet(tableta digitalizadora) etc. [IsenmanCC02a] Isenman, Colin C. "Method and apparatus for displaying translucent overlapping graphical objects on a computer monitor", United States Patent 6,346,281 March 12, 2002 Filing date 1995. GUI with transparent/translucent overlapping windows or objects, on pen computer with touchscreen and stylus. A vestigial outline or simplified form of the object underneath is shown on the obscuring window. [IwamotoT01a] Iwamoto, Takayuki; Maeda, Taro; and Shinoda, Hiroyuki "Focused Ultrasound for Tactile Feeling Display", Proc. ICAT 2001, Dec, 2001, Tokyo, Virtual Reality Society of Japan Haptic feedback / tactile display: focused ultrasound at 1MHz to 5Mhz works both by radiation pressure and direct stimulation of somatic nerves. Neurophysiological/engineering study, not computer application per se. See later work. [IwamuraR02a] Iwamura, Ryuichi "Motion sensing interface", United States Patent 6,498,628, December 24, 2002 Eyes-front gesture user interface using image detection of hand and fingers [IzadiS03a] Izadi, Shahram; Brignull, Harry; Rodden, Tom; Rogers, Yvonne; and Underwood, Mia "Dynamo: A public interactive surface supporting the cooperative sharing and exchange of media", Proc. UIST '03, CHI Letters Vol 5 No 2, 2003, pp. 159-168 Multi-user shared tabletop display with multiple users: each user uses separate mouse and keyboard, or remote connection. [JacobsS02a] Jacobs, Steve "The Electronic Curb-Cut Effect: Disability and Development", www.icdri.org Nov 22, 2002: fetched 2011 International Center for Disability Resources on the Internet. Disability/Accessibility design often useful to other users in other ways. First PDA intended for the deaf (US Pat. 3746793), W3C Web Content Accessibility Guidelines. [JaegerS03a] Jaeger, Stefan and Nakagawa, Masaki "The Callpaper Concept: Turning Paper into Computer Terminals", Seventh International Conference on Document Analysis and Recognition Vol II p. 1065, Aug 03-06, 2003 Edinburgh Scotland Speculative concept paper (?) for using pen and paper as interface to computer: does cite to Anoto, not to Digital Desk or Wellner. Refers to model file for each paper document (definitions of fields, their type -- tick vs. dat vs. action -- and location on the paper): compare with Anoto paper forms definition? Speculative about hardware: refers to optical/infrared tablet digitizers as "Sensor clamp", cites to OTM Technologies VPen laser 3-D digitizer pen for tracking motion. Confuses three-point and four-point calibration of digitizer. [JaredD01] Jared, David A.; Florws, L. Noah; Hecht, David L.; Stearns, Richard G.; and Chang, Kenneth H.P.



"Methods and apparatus for robust decoding of glyph address carpets", United States Patent 6,208,771, March 27, 2001 Use of patterned data glyphs/ two-dimensional address codes, printed on paper to encode position information, first examining the pattern to determine the orientation of the paper, and then to again analysing to determine the position. Compare to Anoto in particular, also to similar Microsoft patent. [JarrettR03a] Jarrett, Rob and Su, Philip "Building Tablet PC Applications", Microsoft Press, 2003 (hardcopy book) Chapter 2 discusses digitizer technologies, classifies them as resistive, electrostatic (which can determine tilt angle of the pen???), electromagnetic. Discusses nonlinearity distortions in digitizer due to ferrous metal and nearby electronic components. Discusses visual parallax due to refraction from protective glass, not from front-mounting of digitizer tablet over the display. Special problems of double-clicking versus double-tapping gesture, hand motion during pointing and hovering in proximity. Mentions barrel buttons on pen. Section "Tablet Computing Comes of Age" quotes Kaplan book on dramatic history of GO and Microsoft on PenPoint and pen computing, but gives no mention to Marlin Eller book on same history from Microsoft point of view to eliminate PenPoint. [JhaveriN03a] Jhaveri, Natalie "Two Characters per Stroke - A Novel Pen-Based Text Input Techniques", Proc. New Interaction Techniques '03, Spring 2003, Univ. of Tampere, Finland Text entry similar to QuikWriting: user touches on key on virtual keyboard, gets pop-up pie-menu for next character (e.g. vowels after consonants) [JonesR01a] Jones, Randall "Review: MTC Express Multi-touch Controller", Computer Music Journal, Vol 25 No 1, 2001: pp. 97-99 Review of TacTex MTC Express Multi-touch Controller: Touchpad with X/Y/Pressure at data rate of 200 points/second, resolutions approximately 1 mm or 25 points/inch. Proximity (in X and Y?) effect affects accuracy when touch points are close together. [JostenM03a] Josten, Marcus; Rutschmann, Dirk; and Massen, Robert "Messbar einfach: Mobiles und wirtschaftliches 3D Body Scanning in der Medizin mit dem MagicalSkin Scanner", CEUR Workshop Proceedings, Vol 80, 2003, pp. 216- 219 (in German) Corpus.e commercial product: User puts on "magical skin" tight body suit with pattern of targets, single camera products 3D model of body using structure-through-motion. Compare with Chen 2010. Application area is fitting of medical prosthesis, etc. [JungT02a] Jung, Thomas "Haptische Systeme ("Haptic Systems")", Lecture notes 27.06.2002, "Entwicklung von Multimediasystemen", Sommersemester 2002, Hochschule für Technik und Wirtschaft Berlin (in German) Lecture notes on haptic and multimedia input systems. Devices without computer-generated feedback include mmouse, pressure-sensitive tables, and touchpads (multi-touch) with gestures. Cites 3-D force mouse, Logitech ultrasonic mouse with 6 DOF degrees of freedom. MTC Express/Tactex multi-touch devise with interpolated values of taxels (touch pixels). Describes haptic-somatic systems using somatic sensing of body position as the "feedback". Datagloves include Cyberglove, Cybergrasp, CyberForce products. Motion tracking, neuronal implants. [KajewskiSL01a] Kajewski, Stephen L.; Tilley, Paul; Crawford, John; Remmers, Todd; Chen, Swee-Eng; Lenard, Dennis; Brewer, Graham; Gameson, Rod; Kolomy, Richard; Martins, Rui; Sher, Willy; Weippert, Achim; Caldwell, Geoff; and Haug, Mark "Handheld Technology Review", Queensland University of Technology Report 2001-008-C06, CRC for Construction Innovation, 2001. Available from icon.net Review of handheld computers / PDAs for use in construction industry: Palm OS, Symbian OS, Windows CE Os, Java. Bar code reading, RFID. GPS mapping software (Pharos, Delorme), GPS data logging. Biometric security (built-in fingerprint sensor: thermal). RotoView streaming video, and orientation sensor to allow viewing of larger virtual displays by tilt/scroll. Report date is 2001, but appears to have (forward looking?) references to July 2003. [KamijohN01a] Kamijoh, Noboru; Inoue, Tadanobu; Olsen, C. Michael; Raghunath, M.T.; and Narayanaswami, Chandra "Energy Trade-offs in the IBM Wristwatch Computer", Proc. IWSC '01, 5th IEEE International Symposium on Wearable Computers, 2001, pp. 1-9 Wristwatch computer (Linux, X11) with touchscreen. Focuses on power management -- reducing the number of turned-on pixels, etc. [KamiyamaK02a] Kamiyama, Kazuto; Mizota, Terukazu; and Kajimoto, Hiroyuki "A Vision-based Tactile Sensor II", Proc. ICAT 2002, December 4-6 2002, Tokyo Optical tactile sensor. Duplicate elasticity of human finger in deformable plastic, with optical markers. Internal camera detects displacement of markers, determines X/Y/Z forces on the sensor body. Application is telepresence, tactile sensing similar to human. Compare to TacTex and similar touchscreens / tablets. [Karpenko02] Karpenko, Olga; Hughes, John F., and Raskar, Ramesh "Free-form Sketching with Variational Implicit Surfaces", Eurographics 2002, Vol. 21 No. 3. Also available as Technical Report TR2002-27, Mitsubishi Electric Research Laboratories, June 2002 Three-dimensional solid-form drawing using two-dimensional input from a mouse, using a projected



silhouette of the two-dimensional input. www.merl.com [KeelyLB02a] Keely, Leroy Bertrand Jr.; Young, Douglas Alan; Palay, Andrew James "Pen-Based Interface for a Notepad Computer", United States Patent 6,337,698, January 8, 2002 Highlight/fade-in image of a tool in proximity of a stylus, depending on height in proximity. Or when stylus is in proximity, bring up a pie-menu (pop-up marking menu). Shows rotatable ribbon window with options for editing text, matches rotation of next and maintains relative position. Compare with movable tools of Buxton et al? [KentJ03a] Kent, Joel and Wilson, Geoffrey D. "Dual sensor touchscreen with projective capacitive sensors and force sensors", European Patent EP 2261781 B1, October 17, 2012. US patent co-filings 5,907,327 and 6,054,979 date to 2003 X/Y grid touchscreen/tablet sensor, combine capacitive sensor and force-resistor sensor in each sensing point. Has numerous references to operating in "drag mode". Cites to other dual-sensor tablet/touchscreens, e.g. to recognize stylus and finger independently in one device: multi-touch. [Kheog01] Kheog, John Michael "Circular Transportation Facilitation Device", Australian Innovation Office, Innovation Patent AU 2001100012 A4, Application Date 2001.05.24 Patent (under Australian IP law) for the Wheel. Patent application made to point out difficulties in examination standards of new Australian patent law. Granted, later retracted. Inventor, along with the Australian Patent Office, received an IgNobel award for this patent. [KinrotO02a] Kinrot, Opher; Kinrot, Uri; and Lederer, Gilad "Optical Translation Measurement", United States Patent 6,424,407, July 23, 2002 Motion detection (not absolute position) using laser light source and speckle pattern of surface, detect with diffraction grating to measure movement of speckle pattern. Part of OTM digital pen/tablet technology? Does not require any particular pattern on the surface. [KiralyJ01a] Kiraly, Jozsef and Dobler, Ervin "Method and System for Gesture Category Recognition an Training using a Feature Vector", United States Patent 6,249,606, June 19, 2001 Trainable gesture recognition and command definition for mouse gestures, or any cursor positioning / cursor directing devices (tablet, touchpad, finger pad, stylus, etc.). Slide-to-unlock? [KlatzkyRL03a] Klatzky, Roberta L.; Lederma, Susan J. and Langseth, Sara "Watching a Cursor Distorts Haptically Guided Reproduction of Mouse Movement", Journal of Exp. Psych: Applied, Vol 9 No 4, 2003, pp. 228-235 If visual feedback in cursor movement does not match haptic feedback/feel, it throws user dragging performance off, even with practice, even if users are forewarned [KlemmerSR01a] Klemmer, Scott R.; Newman, Mark W.; Farrell, Ryan; Bilezikjian, Mark; and Landay, James A. "The Designer's Outpost: A Tangible Interface for Collaborative Web Site Design", Proc. UIST '01, Orlando, Florida, 2001 Early stages of web design (like other graphical user interfaces) dominated by paper sketches of pages and links between pages. DENIM mock-up Paper post-it notes placed on whiteboard touchscreen / computer vision, used as tangible object (electronic lines, tap to get pop-up context menu, move post-it to change position). Physical post-it note can be replaced by electronic representation. Gestures include drawing relationship links between notes. Multiple electronic pens (color). [KnaapiM01a] Knaapi, Matti; Saaren-Seppälä; and Vuoristo, Kalle "Method for organizing and presenting visual information on a computer screen", United States Patent 6,222,543, April 24, 2001 Automatic layout of graphical objects, user-interface elements. Compare to PenApps? [KohtakeN01a] Kohtake, Naohiko; Rekimoto, Jun; and Anzai, Yuichiro "InfoPoint: A Device that Provides a Uniform User Interface to Allow Appliances to Work Together over a Network", Personal and Ubiquitous Computing, 2001, Vol 5 pp. 264..274, Springer-Verlag Drag-and-drop from one display to another (or from printed material to display/computer) using pointing wand with CCD camera to recognize 2D-matrix bar codes. Small display on hand-held unit to confirm "get" (pick) operation. [KrugerR03a] Kruger, Russell; Carpendale, Sheelagh; Scott, Stacey D.; and Greenberg, Sauls "How People Use Orientation on Tables: Comprehension, Coordination and Communication", Proc. GROUP '03, November 9-12, 2003, Sanibel Island, Florida, pp. 369-378 Rotation necessary in tabletop touch display surfaces: Cafe' Table, InteracTable, ConnecTable, InfoTable. Rotation can be done by user via gesture, direct manipulation etc. or automatic (e.g. as user drags virtual object to their location) [KurtenbachG02a] Kurtenbach, Gordon and Fitzmaurice, George W. "System for Accessing a Large Number of Menu Items using a Zoned Menu Bar", United States Patent 6,414,700, July 2, 2002 Pop-up context menus (variation on pie menus and marking menus), menu is brought up by pointing with cursor and pressing space bar or other key on keyboard. Avoids use of press and hold. Note: press-andhold counterexample, not tied to right mouse button. Filing date 1998. [KurtenbachGP03a] Kurtenbach, Gordon P. "Method and Apparatus for Producing, Controlling, and Displaying



Menus", United States Patent 6,618,063, September 9, 2003 Pop-up radial marking menu combined with linear menu, menu is brought up by pointing with cursor and pressing space bar or other key on keyboard. Avoids use of press and hold. Note: press-and-hold. [KuzunukiS01a] Kuzunuki, Soshiro; Aral, Toshifumi; Kitamura, Tadaaki; and Shojima, Hiroshi "Information Processing System", United States Patent 6,266,057, July 24, 2001 Wellner-like SmartDesk/DataDesk tabletop touchscreen user interface combining virtual documents and physical documents (electronic and paper). Compare with Wang Freestyle, e.g. stacking of papers/objects. [LakeM01a] Lake, Matt "Lifting the Cover of those Ubiquitous Touch Screens", New York Times, Thursday March 15, 2001, page G7 top Resistive overlay screens, capacitive overlay digitizers, scanning infrared touch screens, surface acoustic wave SAW. Overview of resistive film for PDAs such as Palm. [LandauS03a] Landau, Steven and Wells, Lesley "Merging Tactile Sensory Input and Audio Data by Means of The Talking Tactile Tablet", Proc. Eurohaptics '03, 2003, Dublin, Ireland Audio touch tablet user interface for the blind, example is geography teaching applications. A fixed tactile map is placed on a touch sensitive tablet (not specified, but perhaps capacitive since it must send through the map paper). Computer gives different responses or software for the same touch areas: for example, user touches on a country in a (tactile) map for software about where different animals live, or the names of the countries, etc. Tactile == static haptics? [LandayJA01a] Landay, James A. and Myers, Brad A. "Sketching Interfaces: Toward More Human Interface Design", IEEE Computer, March 2001, pp. 56-64 Overview of Sketch recognition systems for designing UIs: SILK (Sketching Interfaces Like Krazy), Electronic Cocktail Napkin, Etchapad. Unprocessed electronic ink images can be animated to try out the interactions of the user interface. Silk: rub-out/squiggle, lasso/group, unistroke X, spiral and caret gestures. [LaplantePA01a] Laplante, Phillip A. "Dictionary of Computer Science, Engineering, and Technology", CRC Press, 2001 (hardcopy book) Technical Dictionary of computer terms [LapstunP02a] Lapstun, Paul; Silverbrook, Kia; and King, Tobin Allen "Universal Pen with Code Sensor", United States Patent 6,474,888, November 5, 2002 Pen with multiple features.Mmarking properties: can write in different colors, line widths or patterns (e.g. dotted lines) based on control buttons on pen. Also an optical camera for reading an encoded pattern or images on the paper -- cites to earlier "netpage" application (Compare with Anoto?), or motion information (compare with Hugh Crane?). Very long list of co-pending applications. Cites to Dymetman. [LeeJC01a] Lee, Jin-chul "Software Keyboard System Using Trace of Stylus on a Touch Screen and Method for Recognizing Key Code Using the Same", United States Patent 6,292,179, September 18, 2001 Virtual keyboard using strokes to select different characters Compare with Buxton95 and implementation in WardJR96 [LeeperAK02a] Leeper, A.K. "Integration of a Clear Capacitive Touch Screen with a 1/8-VGA FSTNLCD to form an LCD-based TouchPad", SID 02 Digest, 2002. pp. 187-189 Synaptics ClearPad transparent touchscreen/tablet over small LCD display as an "enhanced" touchpad for a laptop: can show calculator buttons, app launcher buttons, clock, dialog boxes, etc. Compare with Kaplow dynamically reconfigurable keyboard. ClearPad functionality like Synaptics TouchPad, X/Y grid of conductors, junctions form capacitive sensors. Can detect multiple fingers (multi-touch), indirectly measure force from area of contact with finger. Unlike resistive screen, cPad adhered to entire display surface: new methods required for alignment, bubble-free adhesion, delamination. [Lemelson02a] Business Wire "Cognex Wins Appeal in Lemelson Lawsuit; Ruling Allows Cognex to Introduce the Doctrine of Prosecution Laches to Bar Enforcement of Lemelson Patents", Business Wire, January 31, 2002 Cognex wins right to assert Laches (excessive delay in bringing claim) in patent lawsuit regarding machine vision ()Cognex pattern/OCR recognition) brought by Lemelson [Leszynski01a] Leszynski Group "Summaries of Recent Projects: Tablet PC PowerToys / Snipping Tool", Leszynski Group, www.leszynski.com Snipping Tool applet for Tablet PC of Microsoft: clips desktop image (or window) and allows it to be annotated with electronic ink. Compare with Freestyle. Also InkShortcut (compare with PenCept recognition macros) mapping gestures to application's custom action, shape and symbol recognition, electronic ink indexing based on "perceptual" similarity of shapes: Emotive shape guides (e.g. write a sad or happy face mark) [LeventhalJ01a] Leventhal, Jay and Earl, Crista "Screen Readers and Speech Interfaces on the Internet automatically label graphics", Summary of meeting session on AFB testing of screen readers: Proc. CSUN 2001 Screen Readers Tested: JAWS for Windows, Freedom Scientific, Inc. outSPOKEN for Windows, Alva Access Group. Window Bridge, Syntha-Voice computers, Inc. Window-Eyes, GW Micro, Inc. WinVision, Artic



Technologies International. Automatic labelling of graphics included in test criteria. [LevyKL02a] Levy, Kenneth L. "Using Embedded Data with File Sharing", United States Patent Application 20020052885, May 2, 2002 Embedding data (attached) to files to identify, show provenance/source, untampered. Interface to central server for sharing/access rights, purchase. States watermarks cannot be read until a file is decrypted. [LiebenowFW01a] Liebenow, Frank W. "Computer keyboard display device control", United States Patent 6,323,873, November 27, 2001 Computer keyboard control of display (brightness, contrast, etc.) via computer BIOS, independent of operating system functions, or custom hardware in keyboard: BIOS only reads scan codes. [LongAC01a] Long, A. Chris; Landay, James A; and Rowe, Lawrence A. "quill: A Gesture Design Tool for PenBased User Interfaces", Paper submitted for publication, based on PhD dissertation with same title at UC Berkeley, 2001 User-defined/trainable gestures, with automatic system to tell defining user when gestures are similar/confusable, or may have other problems such as poor training set. Cites to other gesture system (Sensiva, KGesture, Opera web browser) for use with applications that are not gesture-aware: compare with PenCept recognition macros. Only single-stroke/unistroke gestures. [LongAC01b] Long, A. Chris; Landay, James A; and Rowe, Lawrence A. ""Those Look Similar!" Issues in Automating Gesture Design Advice", Proc. PUI 2001, Orlando, Florida pp. 1..6 User-defined/trainable gestures, with automatic system to tell defining user when gestures are similar/confusable, or may have other problems such as poor training set. Cut/CopyPaste gestures like caret, insert and pig-tail. Refers to gestures as location-indepedant user interface components (whiteboard?). [LuJ03a] Lu, Jin "Multi-point Touch Pad", United States Patent Application US 2003/0085882 A1, May 8, 2003 Array/matrix of strain gauges for multi-touch touchpad, asserts that arrays of touch sensors alone will register nothing if the point of contact is between the touch sensors (not true for capacitive sensors in general). [LuckieD02a] Luckie, D. "Newton Hall of Fame", www.msu.edu/~luckie/halloffame.htm Biographical information of major personalities in Apple Newton development team: Steve Capps (Finder), Walter Smith (NewtonScript, a.k.a. WallyScript), John Sculley, Michael Tchao ("Apple had been working for years on slate computers that read handwriting"), Stepan Pachkov, Larry Yaeger (Cursive Recognizer) See also 2010 version [Luckman01] Luckman, Coling G; Hecht, David L; Petrie, Glen W; and Kaplan, Ronald M. "Method and apparatus for embedding translation information in text-based imaged data", European Patent Application 1217537, publication date 200-06-26 Xerox Data Glyphs -- optical tablet/location pattern: see also Guimbretiere [LudwigLF03a] Ludwig, Lester Frank " Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting (High parameter-count touchpad controller)", United States Patent 6,570,078, May 27, 2003 Electronic music keyboard with touchpad keys for multi-dimensional control, instrument synthesis: detection of angle yaw/pitch/roll/rotate by analyzing changes in blob shape of a fingertip. Compare with Rekimoto touchscreen, piezoresistive touchpads. Related patents in file, e.g. 8,717,303, 8,743,076, 8,866,785, 8,878,810. [LuethiM02a] Luethi, Marco and Semini, Claudio "FingerMouse 2", Semesterarbeit Institut für Elektronik, ETH, Switzerland, July 5, 2002 (in German) Improvements to FingerMouse -- video-based real-time hand/finger tracking system for wearable computers. Difference images to detect moving hand (or other objects) [LuiCE03a] Lui, Charlton E.; Chew, Chee, and Altman, Dan W. "Adaptive auto-scrolling merge for hand written input", United States Patent 6,559,869, May 6, 2003 Automatic scrolling/positioning of electronic ink in a small display to simulate longer paper [LuiCE03b] Lui, Charlton E., Parker, Kathryn L. and Altman, Dan W. "System and method for automatically switching between writing and text input modes", United States Patent 6,552,719 April 22, 2003 SIP Soft input panel for handwriting input switches input to text / handwriting recognition mode, versus writing/drawing or ink-editing/gesture mode: says automatic, but user must switch modes by clicking/tapping on SIP input mode icon. [LuigiL03a] Luigi, Lira "Controlling content Display", WIPO Patent Publication WO 03/081458 A1, October 2, 2003 Automatically reformat Web page layout for small display (including portable touchscreen device): convert single column to multiple columns, or vice versa. Also highlight multiple rectangular portions/windows that user can click on for zoomed viec. Compare to AppLens? Scrolling may be multiplied or animated (smooth scrolling) in X and in Y. [LusseyD01a] Lussey, David "Polymer Composition", United States Patent 6,291,568, September 18, 2001



Elastomeric conductive rubber: resistance decreases under pressure to that of conductor bridges of filler material. Composition has accelerated elastic return to original state. Sensitivity to match human touch on touchscreen. [MacLeanKE02a] MacLean, Karon E.; Shaver, Michale J.; Pai, Dinesh K. "Handheld Haptics: A USB Media Controller with Force Sensing", Proc. HAPTICS 02, 10th Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems Hand-held haptic "display", accelerometer and sideways thumb-wheel input, single actuator for output, with USB interface. Initial experimental systems include video editing, visual browsing. [MacLovers02] MacLovers.com "Apple Antiquariato", www.maclovers.com/antiquariato/hello29.htm (in Italian) Apple Knowledge Navigator: cited by Ard02 as part of pen-computing history. Apple Newton 1993, Apple Message Pad 1996, Apple eMate (no tablet/handwriting recognition?) [MakeBarcode03a] makebarcode.com "Barcode Basics: Barcode symbologies and Label Standards", www.makebarcode.com, fetched 2003 Standards list for bar codes: Numeric-only, Alphanumeric, 2-Dimensional barcodes, industry and government standards. [MartinD02] Martin, David A. "Projection Display System with Pressure Sensing at Screen, and Computer Assisted Alignment Implemented by Applying Pressure at Displayed Calibration Marks", United States Patent 6,337,681 Whiteboard system using a projection display with a touch-screen digitizer. Multiple pens/stylus, eraser. Describes four-point quadrilateral calibration for projected image with digitizer to compensate for projection distortion (keystone), same method used by PenCept and CIC for aligning paper to digitizer Correct for misalignment of digitizer and image [MathewsCF03a] Mathews, Craig Forrest "Absolute Beginner's Guide to Tablet PCs", Que Publishing, 2003 Extended guide to using Microsoft Tablet PC: mentions barrel button / side switch and eraser buttons on stylus, and controls to disable. press and hold behavior can be disabled if awkward for user, or user can select side switch or press and hold. Refers to cross-functional pen styli that can also leave ink: compare with Summagraphics in 1970s. SIP Start Input Panel gesture is "shake" in proximity. Backspace (left), space (right), enter (down right), tab (up right), scratch-out gestures. Denim sketch recognition for designing web sites from Berkeley Institute for Design. Gesture shortcuts (compare with PenCept recognition macros) via Sensiva Symbol Commander -- special gesture set reminiscent of Jot/Unistroke. (Note: Leszynski.com did Tablet PC Power Toys in 2002) and other third-party products, e.g. sheet music editing with gestures, journal. [MathewsMV01a] Mathews, Max V. "Baton and X, Y, Z position sensor", Unite States patent 6,222,522, April 24, 2001 3D digitizer, transmitters in ends of baton (stylus), X/Y position determined from capacitive coupling to tablet surface with charge-ratio electrodes (triangular vs. width varying). Z position determined from total (absolute) coupling to sensors. "Baton" for use in musical instrument application: multiple batons/styli (multi-touch). Compare with charge-ratio tablets? [MatoSA01a] Mato, Stephan A. Jr. "Capacitive Sensing Keyboard and Pointing Device", United States Patent 6,204,839, March 20, 2001 Capacitive membrane keyboard combined with capacitive touch-screen / position sensor with proximity sensing. In typing mode, force or stroke velocity of pressing key uses as a gesture to indicate bold, shift/capitalization, italics, other functions with key input (e.g. shift "&" to different character "7"). Proximity sensors are individually wired. In pointing mode senses proximity gestures that can be based on multiple fingers or shape of hand (e.g. held vertical across column) [MayerM03a] Mayer, Matthias and Bederson, Benjamin B. "Browsing Icons: A Task-based Approach for a Visual Web History", University of Maryland Dept. of Comp. Sci, Report CS-TR-4308, 2009 Show web browsing history as directed graph ("animation") of all pages visited, not just a single path. [McGeeD01a] McGee, David R. and Cohen, Philip R. "Creating Tangible Interfaces by Augmenting Physical Objects with Multimodal Language", Proc. IUI '01, January 14-17, 2001, Sante Fe, new Mexico Military command office with expensive electronic whiteboards and information display systems: in actual crisis, staff preferred to use large wall map with post-it notes, because the interfaces were tangible. One factor is that the tangible/physical tools are robust: continued to work when computer systems were not operational. [McGeeD02a] McGee, David R.; Cohen, Philip R.; Wesson, R. Matthews; and Horman, Sheilah "Comparing paper and tangible, multimodal tools", SIGCHI, Minneapolis Mn, pp. 407-414 (abstract only) In military command posts, officers prefer paper maps, post-it notes, and hand-written annotations to electronic systems. Rasa adds electronic support to existing paper: paper is robust. [Measurand02a] Measurand Inc. "Measurand Shape Advantage: Shapetape Manual, Hardware and Software", www.measurand.com, June 17, 2002 Flexible tape sensor using optical fibers to detect bend and twist configuration of the tape. Integrates bend



position along tape. One application is body position (real-time). [MetzgerR02a] Metzger, Ram "Pointing Method", United States Patent Application 2002/0190946, December 19, 2002 Point to a location on a screen and enter data there, by typing, voice or other means. Pointing may be by speech input, describing objects on the screen that are then located. Fine positioning by "bouncing" to a kind of object nearly based on input. Coordination of pointing and text cursors. [Meunier03] Meunier, Jean-Luc, Dymetman, Marc, and Ferrnstrom, Christer "Encoded Sheet Material and System for Processing", United States Patent 6,585,163, July 1, 2003 Printing bar codes or other encoded information on the edges of stacks of paper: cites Dataglyphs as related art [Meunier03b] Meunier, Jean-Luc "Coding Scheme for Encoded Sheet Material", United States Patent 6,637,666, October 28, 2003 Printing bar codes or other encoded information on the edges of stacks of paper: cites Dataglyphs as related art [Microsoft01a] Microsoft Corporation "Remarks by Bill Gates", www.microsoft.com, March 26 2001 Announcements by Bill Gates at WinHEC 2001: tablet PC is large potential hardware market, tablet PC is breakthrough large market. [Microsoft01b] Microsoft Corporation "Microsoft Windows User Experience -- Official Guidelines for User Interface Developers and Designers", www.microsoft.com, May 3, 2001 User interface guidelines for Windows UI. Mentions pen-and-paper mock-ups for UI design. References both press and also hold as mouse actions: pressing is positioning over an object, and then pressing mouse button. TTM_GETDELAYTIME for pop-up context-sensitive windows. [Microsoft02a] Microsoft Corporation "Building Ink Chat", msdn.microsoft.com, MSDN, April 2002 Programming example for TabletPC: instant-messaging communications application, appears to be a TabletPC software version of the Telautograph. Compare to whiteboard systems. Electronic ink renders to GIF if electronic ink support not available on the receiving system. [Microsoft02b] Microsoft Corporation "MSDN: SDK Documentation for Windows Mobile-Based Pocket PCs: Mobile Device Hardware Review", msdn.microsoft.com, MSDN, April 2002 Hardware description for Pocket PC: resistive touch-panel digitizer. States that touch-panel input is mouse messages, 100 Pts/sec digitizer rate [Microsoft02c] Microsoft Corporation "Die Geschichte des Pen Computing", Microsoft Corporation press release, November 11, 2002, available at https://www.microsoft.com/germany/presseservice Mentions RAND-tablet (Grafacon) and Ivan Sutherland "Sketchpad" as first pen computers Hewette Crane 1964 input styluses (which see: without tablet, used accelerometer). Mentions Dynabook, but does not assert handwriting recognition for it. Also Grid Computing, Momenta. Attributes pen-centric computing to Hew Crane. Cites GO and PenPoint as the first operating system developed specifically for pen computing. [Microsoft02d] Microsoft Corporation "Start Tablet PC Input Panel with a Gesture", www.microsoft.com/windowsxp/using/tabletpc/learnmore/inputpanel.mspxa, November 7, 2002 Shake gesture to start TIP/SIP Tablet PC Input Panel on Tablet PC [Microsoft02e] Microsoft Corporation "Windows XP Tablet PC Edition: Tablet PC: An Overview", msdn.microsoft.com, June, 2002 Electronic ink / gesture integration with Microsoft Office: specifically states that Tablet PCs use an electromagnetic digitizer (with proximity) and not a resistive-touch screen: mentions palm-rest problem of resistive touch screens Two versions (?) of this, 2002 and 2004. [Microsoft02f] Microsoft Corporation "Pointer Ballistics for Windows XP", msdn.microsoft.com, Update October 31, 2002 Precision pointer / acceleration algorithms for mouse input, states that it achieves smoother pointer movements. Does not discuss digitizer/touchscreen input with absolute position input, only relativemovement mouse. [Microsoft02g] Weeks, Kevin D. "Microsoft Office: Alternative IO: Using Speech and Handwriting Recognition for Data Entry", MSOffice Magazine (msofficemag.com), April 2, 2002 Review article on handwriting recognition (cites to Stylator), 1940's and 1952 on speech recognition: Microsoft Tablet PC, states most important in practice are lexicon/dictionary, user training, hardware performance. [Microsoft03a] Microsoft Corporation "Using Gestures", msdn.microsoft.com, MSDN, January 15, 2003 3:17 p.m. Tablet PC SDK documentation from MSDN, sections include Application Gestures, CollectionMode, InkCollector; InkAndGesture and GestureOnly modes. Has pictures of gesture shapes. Section "Application Gestures and Semantic Behavior" : defined gestures include scratch-out/erase, Triangle/insert, Check-off, Curlicue/fixed (compare with proofreader's marks), Double-Curlicue/copy, double-circle/paste, leftsemicircle/undo, right-semicircle/redo, Caret gesture/paste, Inverted-caret/insert, flick left / backspace,



flick right / space (compare with GO flick gestures), left/right/up/down corners with directional gestures. See also 2008 version of same reference. [Microsoft03b] Microsoft Corporation "ApplicationGesture Enumeration -- SystemGesture Enumeration", www.microsoft.com List of Application gestures and system gestures for Microsoft Tablet PC. Lists Tap and DoubleTap as gestures that work for both, cites modal confusion with exclamation point gesture. Notable that exclamation is used both as a gesture and as a handwriting character. Gestures include curlicue similar to proofreaders cut/delete mark, double-curlicue. Second copy has pictures of all gestures (from Longhorn Beta of Windows 7) Dated Jan 15 2003. Cites additional gestures to be supported: see "Unimplemented Glyphs". [Microsoft03c] Microsoft Corporation "Microsoft Tablet PC - Windows XP Tablet PC Edition", msdn.microsoft.com, January 15, 2003 Hand obscures view of display, targets should be larger. Default alignment of menus is switched to left, because predominantly right-handed users of stylus/tablet versus mouse. Windows Mouse GUI is modified for stylus input. Targeting more difficult with stylus than with mouse. Double-tap has motion with stylus. Notes differences between double-tap gesture with a stylus/digitizer tablet and double-click with mouse, tool-tips different for mouse and tablet gestures because of difficulty targetting a single pixel with accuracy with a stylus, and double-tap hard to do on the same pixel (as with a mouse). Right-handed/left-handed tooltips and menus (different sides) [Microsoft03d] Microsoft Corporation "Microsoft Knowledge Base: Cannot See Writing with Pen Pointing Device and Super VGA, KB111569", msdn.microsoft.com, 10/14/2003 Reference to modified display drivers to show ink data in conjunction with tablet/pen driver in Windows for Pen computing 1.0 (from 1992) [Microsoft03e] Microsoft Corporation "Microsoft Tablet PC - System Events and Mouse Messages", msdn.microsoft.com, January 15, 2003 Tablet PC generates both mouse events and system events (gesture messages), regardless. Mentions difference between ISG_TAP tap gesture sent when pen/stylus is lifted, and WM_LBUTTON_DOWN message sent on mouse click. No mouse GUI equivalent to HOVER gesture of tablet GUI. Mouse messages useful for pointing, pen events for real-time ink. [Microsoft03f] Microsoft Corporation "Microsoft Tablet PC - Timeline of Mouse Messages and System Events", msdn.microsoft.com, January 15, 2003 Mentions separation of system events / Gestures from mouse input: ISG_Tap events are processed instantaneously, whereas mouse WM_ messages and mouse clicks are delayed by windows messaging service queue. DoubleTap gesture not accurate, longer time between click-like events compared to mouse. Sequence of ISG_DBLTAP and WM_DBLLBUTTON messages indeterminate. [Microsoft03g] Microsoft Corporation "Microsoft Tablet PC - Unimplemented Glyphs", msdn.microsoft.com, January 15, 2003 Tablet PC gestures appear to be nearly identical list of gestures to Hullender patent. Gestures include Circle-Circle, DiagonalLeftUp, DoubleDown, all alphanumerics, TripleDown, QuadrupleTap, etc. Infinity gesture to switch in and out of gesture mode. [Microsoft03h] Microsoft Corporation "Microsoft Tablet PC - General Threading Considerations", msdn.microsoft.com, January 15, 2003 MouseDown and CursorDown (mouse vs. tablet/stylus) events occur on different threads, for reasons of rendering electronic ink [Microsoft03i] Microsoft Corporation "Developing Applications Using your Tablet PC", msdn.microsoft.com, Microsoft TechNet, May 13, 2003 Arin Goldberg, Cory Linton presentation on Tablet PC. Soft Input Panel and also Tablet PC Input Panel: can be bound to input fields in dialogs as part of application programming. Discussion of differences in electronic ink APIs and functionality between Windows Mobile / Pocket PC and Tablet PC, touch screen and proximity digitizer. Newsgroup microsoft.public.tabletpc.developernewsgroup on msnews.microsoft.com [Microsoft03j] Microsoft Corporation "Microsoft Tablet PC - Design Recommendations", msdn.microsoft.com Pen as Pointing and Input Device; Designing for the Use of Handwriting Recognition; Interoperability of Ink Data; Readability; external monitors on Tablet PC; Accessibility. See also 2006 version. [Microsoft03k] Microsoft Corporation "DOC: Pen Documentation in Windows Resource Kit Incorrect", support.microsoft.com, Article Q86031 Re-publication of 1992(?) Notice about Windows for Pen computing 1.0, on sample tablet drivers for WACOM and Scriptel tablets being included with Pen OAK separately from version 3.1 of Windows SDK. [Microsoft03l] Microsoft Corporation "Microsoft Announces the Next Version of the Tablet PC Operating System", www.microsoft.com, November 17, 2003 Windows XP Tablet PC Edition 2004 with OneNote and Journal note taker applications with editing of electronic ink, electronic ink annotation in ExCel spreadsheets



[MihaelescuP03a] Mihaelescu, Patrik "MAE: A Mobile Agent Environment for Resource Limited Devices", Ph. D. Thesis, Monash University, January 2003 Mobile agent: software that moves itself to a host computer to continue executing on behalf of user when user turns off a portable computer /PDA. Original concept was General Magic with Telescript language, current (2003) technology is mostly Java. Thesis pre-dates Apple iPad/iPhone. Mentions Newton. Development platform for project was PalmOS. [MilekicS02a] Milekic, Slavko "Towards Tangible Virtualities: Tangialities", Proc. Museums and the Web, 2002 Essay relating to tools that enhance cognitive function. Explanation of direct manipulation, tangible user interfaces. Vibro-tactile (haptic) feedback on mouse to show boundaries of user interface objects (iFeel mouse from Immersion TouchSense). Milekic Author of KiddyFace touchscreen interface for museum exhibits. [MilekicS02b] Milekic, Slavko "Making Playful Interfaces for Serious Content", Grindstone Island Seminars, Summer 2002 User-interaction seminar: Touchscreen displays. Pseudo-3D layouts, use of pressure (force) force sensing with fingter, gesture and motion capture on flat touchscreens, haptic interfaces. Proximity sensing and proximity gestures. File contains additional materials. [MinS02a] Min, Saehee Sarah "Caressing Sound and Image", EECE 496 Final Project Report, Univ. of British Columbia, Canada, Project Code SSF3, April 5, 2002 MTC Express Multi-Touch controller with multipoint pressures (multitouch force-sensitive) using image processing interpolation to recover actual shape. Mostly about image interpolation algorithms (Pythagorean and pre-calculated sine interpolation) for multi-touch sensor: compare with Rekimoto? [MitchellGD03a] Mitchell, G. Daryn "Orientation on Tabletop Displays", Master's Thesis, Simon Fraser University, October 2003 Review of touchscreen tabletop displays and the approaches taken to rotating what is displayed for the users at different perspectives around the tabletop. Cites to MetaDesk. Different approaches includes rotating individual objects, and rotating the entire display UI. [MoranTP01a] Moran, Thomas P.; van Melle, William J.; Kurtenbach, Gordon P.; Chiu, Patrick "Computer controlled display system using a graphical replay device to control playback of temporal data representing collaborative activities", United States Patent 6,332,147, November 3, 1995 Electronic whiteboard: Record and play back synchronized electronic ink and audio: compare with Wang Freestyle? [MoravecKLC02a] Moravec, K.L.C. "A Grayscale Reader for Camera Images of Xerox Dataglyphs", Proc. BMVC 2002, pp. 698..707 Method of decoding dataglyphs (watermark/steganographic information hidden in images) not affected by image degration, angle of view, etc. Compare with image corrections in Anoto. Requires enough resolution to determine angles of image dots/oblongs. [MortonJK03a] Morton, J. Kevin "Disability Law Blog: MyIE2: Super Drag and Drop and Mouse Gestures", www.jkevinmorgon.com Mouse Gestures with web browser [Motif01a] Fountain, Antony and Fergusun, Paula "Motif Reference Manual for Motif 2.1 Volume 6b The Definitive Guide to the X Windows System", O'Reilly and Associates, 2001. Copyright 1998, 2000, 2001 moveThreshold of class MoveThreshold: minimum distance for drag. Also XmNselectThreshold, minimum distance to move to select next character in text/drag selection. [Motorola03a] Motorola Corporation "Creating Device Configurations: White Paper", www.motorola.com IDE development emulator for Motorola smartphones, using Java and CodeWarrior debugger. Simulation image of virtual Motorola smartphone (similar to T720) in J2ME SDK. Mouse emulation / mouse events in smartphone. [MotwaniM03a] Motwani, Mukesh "Colocated Dataglyphs: Master's Thesis Proposal", University of Nevada, Reno, Computer Science Department, 2003 Dataglyph: extension of error-correcting and tamper-evident properties to multiple pages [MoyleM03a] Moyle, Michael and Cockburn, Andy "A Flick in the Right Direction: An Evaluation of Simple Gesture Based Controls", http://www.cosc.canterbury.ac.nz/andrew.cockburn/papers/moyle-cockburn.pdf, also presented (in part) at Fourth Australasian User Interface Conference AUIC03, Adelaide Australia, and in Behaviour and Information Technology. 24(4): 275--288. Taylor and Francis. 2005 Also published under similar title (?): A Flick in the Right Direction: a case study of gestural input: Behavior and Information Technology, Vol 24, Number 4, July 2005, pp 275-288. Comparison of flick gestures (with a stylus) with dragging to scroll web pages, also comparison with mouse input, in PDAs. Flick gestures win. Cites study of Graffiti Unistroke alphabet (not Graffiti2/JOT) Contains references to Unistroke gestures from various authors [MoyleM03b] Moyle, Michael and Cockburn, Andy "The Design and Evaluation of a Flick Gesture for 'Back' and



'Forward" in Web Browsers", Fourth Australasian User Interface Conference AUIC03, Adelaide Australia Flick forward and flick back gestures based on marking menus. [Mueller-Prove02] Müller-Prove, Matthias "Vision and Reality of Hypertext and Graphical User Interfaces", Report 237, Fachbereich Informatik, Universität Hamburg, Germany, February 2002, (Master's Thesis) Has review of Memex by Vannevar Bush, Sketchpad by Sutherland, Flex machine and Dynabook. Shows Flex Machine with separate digitizing tablet (proposal) in 1969. Shows Parallel Textface with multi-touch (? ) touchscreen in Xanadu system, 1972. [MunsonJC01a] Munson, John C. and Wimer, Scott "Watcher: The Missing Piece of the Security Puzzle", ACSAC 2001 Conference Proceedings, New Orleans 11-14 December 2001, pp 230-239 Disallowed Operational Anomaly system: dynamic modelling of program behavior using execution path patterns, deny operation when program behavior diverges from learned patterns. Technical reference for Cylant.com, intrusion detection contrasted to signatures for files or SNORT network patterns [MunsonJC02a] Munson, John C. "Method and System for Simplifying the Structure of Dynamic Execution Profiles", United States Patent Application 2002/0138753, September 26, 2002 Real-time detection of injected unauthorized code on a computer system by pattern matching to execution profiles for usual paths of code-path execution. Refers to instrumentation points, arbitrary metric points (usually branches) in code execution [MunsonJC03a] Munson, John C. "Method and System for Establishing Normal Software System Behavior and Departures from Normal Behavior", United States Patent Application 2003/0200462, October 23, 2003 Real-time detection of injected unauthorized code on a computer system by pattern matching to execution profiles for usual paths of code-path execution. Suggests mapping programs with highly correlated behavior (?) into one model. [MyersRL02a] Myers, Robert L. "Display Interfaces - Fundamentals and Standards", Wiley-SID Series in Display Technology, Society for Information Display, 2002 Reference on displays, image standards and formats, display output interface standards, types of displays (analog/digital etc.) Very little on input (touchscreens, etc.) [NashelA03a] Nashel, Andre and Razzaque, Sharif "Tactile Virtual Buttons for Mobile Devices", ProcCHI 2003, Short Talk: Haptic Interfaces, pp 854-855 Haptic/tactile feedback on touchscreen buttons by vibrating display when user's finger is over button image. Click on edge of button, mild vibration while on button. As user presses harder, vibrate at higher frequency. If touchscreen is not pressure sensitive, use dwell time. Cites to Immersion Technologies TouchSense tactile mouse. [NationalFederationOfTheBlind02a] National Federation of the Blind "Blindness Related Computer Files", http://www.nfbnet.org/download/blind.htm Download an installation files for screen reader ASAP Automatic Screen Access Program, Access Technologies: main index links are not valid, must to go directory page. ASAP versions back to 1992. [Neonode03a] Neonode.com "Neonode N1 released", http://www.neonode.com Cellphone with optical touch screen. Sweep/flick gestures: slide-to-unlock? [NewYorktimes02a] New York Times "TECHNOLOGY: Silicon Valley's Dream Tablet, from Microsoft", http://select.nytimes.com (abstract only) Announcement of Tablet PC on Windows XP by Bill Gates, [NewmanMW03a] Newman, Mark W.; Lin, James; Hong, Jason I.; and Landay, James A. "DENIM: An Informal Web Site Design Tool Inspired by Observations of Practice", Human-Computer Interaction, 2003, Vol 18, pp. 259324 DENIM: Design Environment for Navigation and Information Models. Prototyping tool for web pages. Includes sketch recognition (square, rectangle, etc.) with editing with gestures, includes barrel button of stylus, handwriting recognition, gestures, pie-menus (tap with barrel button on stylus depressed). Gestures include Cut, Copy, Paste, Undo, Redo, Insert Text, several Pan/flick gestures. Hold (press-and-hold) selects object, tap draws dot. Cites to Rubine 91 gesture recognizer and gdt/SATIN. Press-and-hold: defines gesture as a stroke created by the pen that activates a command. Counterexample for press-and-hold as invocation of context menu, press-and-hold doing something else. [NielsenM03a] Nielsen, Michael; Stoerring, Moritz; Moeslund, thomas B; and Granum, Erik "A Procedure for Development Intuitive and Ergonomic Gesture Interfaces for Man-Machine Interaction", Technical Report CVMT03-01, ISSN 1601-3646, Aalborg University, March 2003 hand-gesture user interfaces: refers to static/positional gestures of the hands, dynamic moving gestures, and spatio-temporal gestures. menu gesture with flat hand. [NorrieMC02a] Norrie, Moira C. and Signer, Beat "Web-Based Integration of Printed and Digital Information", Proceedings of DIWeb'02, Toronto Canada May 2002 Digitally encoded paper with (bar code? dataglyph? electronic paper?) information for web links. Compare to Hyperlink. "digitally augmented paper", references to Digital Desk, Anoto.



[NorrieMC02b] Norrie, M. C. "The Myth of the Paperless Office", Monthly Meeting of ETH Emeritus Professors, Winterthur, Switzerland, April 29, 200 See also other paper same title, different author. Paper advantages over electronic ink on tablets: low power, flexible, annotation, physical human interaction. Mentions Anoto optical pattern paper, LeapFrog education books with stylus touch to pictures, Paper++ project, but outstanding problems on interactions with paper/electronic data. [NorrieMC03a] Norrie, Moira C. and Signer, Beat "Switching over to Paper: A New Web Channel", Proceedings of 7th Internal Conference on Electronic Paper, Saint Malo, April 1998. Reprint available at www.globis.ethz.ch Digitally encoded paper with (bar code? dataglyph? electronic paper?) information for web links. Compare to Hyperlink [NoyesJ02a] Noyes, Jan "Talking and writing - How natural in human-machine interaction", International Journal of Human-Computer Studies, Vol 55 No 4, October 2001 Short essay on use of handwriting and speech recognition: refers to speech recognition system in 1952, "Automatic recognition of spoken digits" [Oberquelle02a] Oberquelle, Horst; Obendorf, Hartmut; and Mueller-Prove, Matthias "Eine kurze Geschichte der Software-Ergonomie: Pioniertaten der Mensch-Computer-Interaktion", Presentation at Mench-Computer conference, 2002 Historical overview of human/computer interaction: cites to Vannevar Bush 1945, Sketchpad by Sutherland, Hypertext by Theodor Holm Helson; Mouse by Douglas Engerbart; Xerox PARC; Adele Goldberg/Dan Ingalls personal computer; etc. Note: most projects up until 1980's were US government/DARPA funded. [OberteufferJA02] Obersteuffer, John A.; Wilbanks, John; Loken-Kim, Kyung-Ho; and Kania, William "Processing Handwritten and Hand-Drawn Input and Speech Input", United States Patent 6,438,523, August 20, 2002 Combined multi-modal user interface for pen-computing gestures and voice commands. Illustration shows gesture interface as a touch interface (finger). Notes that with speech input / speech/voice recognition, screen or graphical navigation is awkward, hence use of a stylus/digitizer. [ObradovichML03a] Obradovich, Michael L. "Positional Camera and GPS Data Interface Device", United States Patent 6,525,768, Feb 25, 2003 Cellphone (handheld personal communication device PCD) with GPS receiver and camera and touchscreen, user requests maps and geo-tagged data for the map to be displayed. [ObradovichML03b] Obradovich, Michael L.; Dinkel, John; Kent, Michael "Personal Communication System for Communicating Voice Data Positioning Information", United States Patent 6,529,824, March 4, 2003 Server responds to GPS / map data from cellphone with GPS and touchscreen by sending map and geotagged data [ObradovichML03c] Obradovich, Michael L.; Pirtle, John D.; and Schebesch, Steven W. "Technique for Effective Navigation based on User Preferences", United States Patent 6,542,812, April 1, 2003 Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back route and favorites, based on preferences in a user profile. [OhJC01a] Oh, Jae-Choeul, and Park, Chong-Yok "Portable Computer having Touch Pad Input Control Function", United States Patent 6,266,050, July 24, 2001 portable computer with touchscreen does not turn on until a particular pattern (gesture) is traced on the touchscreen or touchpad. Compare with Apple slide to unlock? [OkaK02a] Oka, Kenji; Sato , Yopichi; and Koike, Hideki "Real-Time Fingertip Tracking and Gesture Recognition", IEEE CGA, November/December 2002, pp 64-71 Optical tracking of multi-touch fingertips with one camera, top view. Single and double-finger gestures, hand-position (e.g. closed thumb) gestures: circle, triangle, rectangle; forward and reverse direction, not just shape. Compare with Wellner, Eyesight. [OkaK02b] Oka, Kenji; Sato , Yopichi; and Koike, Hideki "Real-Time Fingertip Tracking of Multiple Fingertips and Gesture Recognition for Augmented Desk Interface Systems", Proc. FGR '02 Fifth IEEE Int'l. Conf. on Automatic Face and Gesture Recognition, p. 429-ff EnhancedDesk: Optical tracking of multi-touch fingertips with one camera, top view. Uses both infrared camera and color CCD camera. Single and double-finger gestures, hand-position (e.g. closed thumb) gestures: circle, triangle, rectangle; forward and reverse direction, not just shape. Compare with Wellner, Eyesight. [Olsson03a] Olsson, Andreas "Processing of Digital Images", WIPO International Patent WO 03/001450 A1, 3 January 2003 Describes first-cut optical processing for Anoto technology, optical digitizer pattern [Olsson03b] Olsson, Andreas "Method and Device for Data Decoding", WIPO International Patent WO 03/001441 A1, 3 January 2003 Describes coding pattern for Anoto technology of displaced dots [Olsson03c] Olsson, Andreas "An Optical Sensor Device and a Method Of Controlling Its Exposure Time", WIPO



International Patent WO 03/030082 A1, 10 April 2003 Describes optical sensor for Anoto [Olsson03d] Olsson, Andreas "Method and a Hand-held Device for Identifying Objects in a Sequence of Digital Images by Creating Binarized Images based on a Adaptive Threshold Value", WIPO International Patent WO 03/044740 A1, 30 May 2003 Image processing for Anoto optical digitizer/tablet using dot pattern on paper [OverholtD01a] Overholt, Dan "The MATRIX: A New Musical Instrument for Interactive Performance", Proc. ICMC Int'l. Computer Music Conference '01, Havana, Cuba, September 17-22, 2001 3D digitizer of sorts, mechanical rod displaced downward by hand form reverse image of hand. Shows mechanical design of rods. No citations to similar mechanical devices for carpentry, etc. [OverholtD01b] Overholt, Dan "The MATRIX: a novel controller for musical expression", New Interfaces for Musical Expression at CHI '01, Seattle, WA, April 1-2, 2001 3D digitizer of sorts, mechanical rod displaced downward by hand form reverse image of hand. Demonstration in remote live musical performance between Cambridge MA and Ireland. Shows mechanical design of rods. No citations to similar mechanical devices for carpentry, etc. [OverholtD02a] Overholt, Dan "New Musical Mappings for the MATRIX Interface", Proc. ICMC Int'l. Computer Music Conference '02, Goteburg, Sweden, September 16-21, 2002 3D digitizer (see other references), position of rods (scanned in rows, zig-sag) mapped to wave shape for musical performance [OverholtD03a] Overholt, Dan "Control of Signal Processing Algorithms using the MATRIX Interface", Convention Paper, Audio Engineering Society 114th Convention, March 22..25, 2002, Amsterdam 3D digitizer (see other references), position of rods (scanned in rows, zig-sag) mapped to wave shape for musical performance. Examples: each rod controls distinct display echo. Compared to Korg Kaoss Pad touch pad for musical control. [PCMagazine02] PC Magazine "Tablet Timeline", www.PCMag.com Historical timeline of certain pen-computing devices: mentions AT&T EO, but no mention of GO/PenPoint/Lombard. Focus on portable product. 1956 Rand tablet digitizer Grafacon (photograph), 1983 Tandy Model 100 stylus PDA, 1975 Apple Knowledge Navigator, 1989 GRiD GRiDPad, 1991 NCR System 3125 with Windows for Pen Computing / Pen Windows, 1992 Momenta, 1993 Apple Newton Messagepad, 1995 Zenith CruisePAD, 1996 Palm Pilot 1000, 1998 Vadem Clio, 1998 Sharp Mobilon, 1999 Aqcess Qbe, 2001 ThinkPad TransNote, 2001 Microsoft Tablet PC, 2001 AirSpeak Flair, 2002 Microsoft Mira Technologies, 2001 Sony SlipTop, Amstrad PenPad. No mention of Pencept or CIC/Handwriter. [PaceMR02a] Pace, Mark Raymond and Talley, Brooks Cash "Distributed Content Identification System", United States Patent 6,460,050 Spam and virus filtering by sending an identifier or characteristic (e.g. hash) of the file to a central server for comparison with similar identifiers or characteristics. compare with Cylant technology. [Palm02a] Palm Pilot "Graffiti documentation, character set", 2002 Palm Pilot personal organizer product information Palm Unistroke character set, except for cap lock gesture. Single tap gesture, showing punctuation shift, write in separate area to get capitals. [Palm02b] Palm, Inc. "Palm Completes Formation of Palm OS Subsidiary as Palm Powered Devices Hit 20 Million Sold", Palm, Inc. press release, January 21, 2002 Announcement of Palm OS business spin-off from hardware. Cites sales figures to date, three hardware vendors. [PalmInfocenter03a] PalmInfocenter.com "Howto: Replace Graffiti 2 with Original Graffiti", http://PalmInfocenter.com Mentions Graffiti 2 as CIC JOT recognizer [PalmSource03a] PalmSource, Inc. "(Misc technical documentation relating to PalmSource OS)", PalmSource (in bulk files) [ParadisoJA02a] Paradiso, Joseph A. "Several Sensor Approaches that Retrofit Large Surfaces for Interactivity", ACM UbiComp 2002 Workshop on Collaboration with Interactive Walls and Tables, Gothenburg, Sweden, September 29, 2002 Touchscreen/touch sensors for retrofitting to large surface, including touch tables. Gesture Wall capacitive hand tracking using AC signal in human body, laser rangefinder to track multiple hands (multi-touch), acoustic knock and tap detection using differential arrival times. Jeff Han FTIR Frustrated Total Internal Reflection (in 2002), but says operates by shadow of optical sensors on side/perimeter of glass. [Parc03a] Xerox Palo Alto Research Center "DataGlyphs: Embedding Digital Data", http://www.parc.com/research/asd/projects/dataglyphs: available also 2015 www.microglyphs.com Barcode-like pattern is printed very small as the pixels in a printed image: optical stylus can read the Glyphtones, and thus read what part of the image, or what functional "hot-spot" you are over in the



image. Compare to Anoto and Sekendur technology. Says double the data density of Barcode standard PDF417. [Parc03b] Xerox Palo Alto Research Center "DataGlyphs Frequently Asked Questions", http://www.parc.com/research/asd/projects/dataglyphs/faq.html Barcode-like pattern is printed very small as the pixels in a printed image: optical stylus can read the Glyphtones, and thus read what part of the image, or what functional "hot-spot" you are over in the image. Compare to Anoto and Sekendur technology Business and technical discussion, including print resolution [PartridgeK02a] Partridge, Kurt; Chatterjee, Sauray; and Want, Roy "TiltType: Accelerometer-Supported Text Entry for Very Small Devices", Proc. UIST '02, October 27-30, 2002, Paris. pp 201-204 TiltTyping, similar to EdgeWriting/QuikWriting and Unigesture, but using tilt. References to other wearable keyboards, devices for text entry. [PasqueroJ03a] Pasquero, Jerome and Hayward, Vincent "STReSS: A Practical Tactile Display System with One Millimeter Spatial Resolution and 700 Hz Refresh Rate", Proc. Eurohaptics 2003, July 2003, Dublin High resolution tactile display using piezoelectric actuators mounted on the skin to produce shear strain on fingertip. [PateSD03a] Pate, Seve D. "UNIX Filesystems: Evolution, Design, and Implementation", Wiley, 2003 Includes historical section on filesystem development. Multiple named data stream in same file included in NFS Version 4. [PattenJ01a] Patten, James; Ishii, Hiroshi; Hines, Jim; and Pangaro, Gian "Sensetable: A Wireless Object Tracking Platform for Tangible User Interfaces", Proc. CHI 2001, March 31 .. April 5, Seattle, pp. 253..260 Tabletop system with electromagnetic position, identity, and orientation sensors for instrumented physical objects (pucks). Capacitive touch sensor for when object is being moved/adjusted. Objects can be positioned physically, dials and buttons for adjusting properties. Prototype used Zowie multi-object (multitouch) tracking tablet (did not sense orientation). Experimental system for user interface alternatives to WIMP. Projection display from above: project (changeable) labels or "binding" onto pucks. Semantic zooming based on proximity of two pucks. [PenAndInternet03] Pen and Internet LLC "Quick Start Guide - ritePen for Windows", www.penandinternet.com/piweb/ritePenwin-qug/index.asp, Copyright 2003 ritePen and riteScript handwriting recognition application for Tablet PC or Windows XP. Uses basic gestures with input, appears to make use of screen reader accessibility technology to detect input fields in applications. Dual modes of operation in user interface for handwriting and pointing/mouse input. [PenComputing01a] Walker, Geoff "The Microsoft Tablet PC: A detailed look at Microsoft's Proposed Tablet PC Project", July 2001 issue, available at www.pencomputing.com Microsoft announced their own tablet "for computing, communicating and reading electronics books" in 1999, prototype demonstrated June 2000. Points out that Microsoft's presentation omitted mentioning 10 previous years of pen computing industry. Refers to 1991, Pen Windows (Pen Services for Windows), Pen Windows 2.0. [PenComputing02a] Walker, Geoff "Microsoft Mira at a Glance", Pen Computing Magazine, July 2002, available at www.pencomputing.com Microsoft remote tablet/display for remote (100 feet) access to a Windows XP PC, using Terminal Server software over 082.11b WiFi. Not compatible with Tablet PC OS. Similar devices are AirSpeak Flair, DOT Design Technologies Xtend, Sony Airboard, Panasonic Toughbook 07. [PenComputing03] Pen Computing Magazine "Pen Computing Buyer's Guide 2003", August (?) 2003, available at www.pencomputing.com Product review of Tablet PC laptops/slatetops for 2003: Acer Travelmate, Electrovaya Scribbler, FIC (First International computer Inc.) SlateVision, Fujitsu Stylistic, HP Compaq PC TC1000, Intermec CT60 Rugged Tablet PC, Panasonic GoBook, Motion Computing, Panasonic Toughbook, NEC Versa Litepad, PaceBlade Technologies Pacebook, Tatung Tablet PC, Toshiba Portege, ViewSonic Tablet PC V1100, Xplore Technologies [PetterssonMP03a] Petterson, Mats Petter "Method and Device for Decoding a Position-Coding Pattern", WIPO International Patent WO 03/038741 A1, 8 May 2003 Anoto position calculation: optical digitizer based on pattern of displaced dots [PetterssonMP03b] Petterson, Mats Petter "Reconstruction of Virtual Raster", WIPO International Patent WO 03/049023 A1, 12 June 2003 Anoto rasterization pattern and encoding [PetterssonMP03c] Petterson, Mats Petter and Bjoerklund, Andreas "Position Code", United States Patent 6,667,695, December 23, 2003 Anoto optical digitizer technology for "smart paper", refers specifically to not needing much pattern memory to realize the encoding of information that is larger than the writing surface. Specifically refers to



determining rotation of stylus/barrel [PetterssonMP03d] Petterson, Mats Petter and Ericson, Petter "Product with a Coding Pattern and Method, Device and Computer Program for Coding and Decoding the Pattern", International Patent Application Publication WO 03/042912 A1, May 22, 2003 Anoto position calculation: optical digitizer based on pattern of displaced dots. Second example uses both displacement, and varying size of dots. Pattern not visible to user. [PetzoldC02a] Petzold, Charles "Programming Microsoft Windows with C#", Microsoft Press, 2003 Microsoft Windows programming guide and reference, C#/C-sharp. Interfaces instead of multiple inheritance. Nothing on Tablet PC, PEN events, graphics bitmap masks. Compare with Visual Basic, PenApps. [PhilippH01a] Philipp, Harald "Capacitive Position Sensor", United States Patent 6,288,707, September 11, 2001 Capacitive proximity digitizer combined with physical keyboard, user can move cursor by moving hand over the keyboard. Sensors are triangular pads with pointy ends toward center (or left and right for singledimension input), measure amount of coupling to hand or finger, along with which pad had the coupling. [PhilippH02a] Philipp, Harald "Capacitive closure obstruction sensor", United States Patent 6,377,009, April 23, 2002 Capacitive proximity sensor to detect user's hand or finger where it might be / is caught in a hinge or door. Automotive applications? [PhilippH02b] Philipp, Harald "Capacitive sensor and array", United States Patent 6,452,514, September 17, 2002 Use array or line or matrix of capacitive touch sensors to form a keypad, keyboard, linear slider, liquid level sensor, or other sensing array. AC signal to capacitive electrodes. Slider can be curved. [PhilippH02c] Philipp, Harald "Level sensing", United States Patent 6,457,355, October 1, 2002 Capacitive disk electrode sensor determines proximity of liquid water level. When water rises to cover electrode, another electrode higher up on a rod senses further. [PhilippH02d] Philipp, Harald "Charge transfer capacitance measurement circuit", United States Patent 6,466,036 October 1,5 2002 IC integrated circuit component for capacitive sensing, uses pulsed signals (not AC or DC) to measure capacitance change. Sensor can be multiplexed for multiple sensors. [Picasoft02a] PicaSoft SA "PicaSoft User manual: Handy Scan", PicaSoft SA, France, 2002 Desktop 3D three-dimensional electromechanical digitizer. Has multiple kinds of stylus tips (round, pointed, flat, etc.) for "scanning" different types of object in 3D, depending on the detail and minutiae of the surface. Windows drivers. [Picsel02a] Picsel Inc. "Picsel Viewer for CLIE", picsel.com, 2002 Picsel CLIE Photo and image viewer for Palm OS, has flick/scroll-page gestures. Tap+Press-hold-drag for zoom/unzoom (up and down), internal scroll "throw" gesture. Thumbnails of images age out and go away after a period of time to save memory resources. [Pinard03] Pinar, Debbie and Oddy, Douglas "Telephone with Handwriting Recognition", United States Patent 6,522,729, February 18, 2003 Signature verification for cell phone access using a writing touchpad: uses a separate touchpad, not a stylus or pen built into the phone, as some other technologies do [PocketComputing02] Pocket Computing Museum "Welcome to the Pocket Computing Museum", http://cdecas.free.fr/computers/pocket/museum.htm product: Epson RC-20 writewatch with 2000 Bytes (!) storage, handwriting recognition similar to unistroke. product: Kycoera Refalo 1991 double ring binder portable computer, handwriting recognition product: Casio IF-8000, showing user writing "meeting" on the screen, handwriting calculator, 1980's product: Sony Magic Link running General Magic MAGIC-CAP OS, office-level metaphor product: EO Personal communicator, 1994, with phone, fax, etc. product: IBM Simon concept in 1994, personal computer and phone product: Sharp Power Sarus multimedia hand-held computer with digital camera product: Sharp MCG-1 communicator (date late 1990s?), includes sketch input, PIM. product: Nokia 9000 communicator: organizer, web, email, GEOS pen-computing touchscreen operating system. product: Casio VDB-100 touchscreen wrist watch, Casio NX-6000 / Planeo pen computer, Z7000 Zoomer GEOS product: INDEX portable keyboard-less (touchscreen) computer, with FM-Radio receiver : camera? [PookS01a] Pook, Stuart "Interaction and Context in Zoomable User Interfaces", Doctoral Thesis, Ecole Nationale Superieure des Telecommunications, Paris, France, 15 June 2001 Zoomit project. Pop-up/radial context menus. Mentions command compass menus of Momenta. [PorrasPA02a] Porras, Phillip Andrew and Valdes, Alfonso "Hierarchical Event Monitoring and Analysis", United States Patent 6,484,203, November 19, 2002 Network intrusion detection by analyzing network traffic, including network errors and packet inspection. Refers to statistical analysis: Compare with invariant condition analysis of Cylant? [PoupyrevI03a] Poupyrev, Ivan; and Maruyama, Shigeaki "Tactile Interface for Small Touch Screens", Proc UIST



'03, Vancouver, BC, Canada, pp. 217-220 Tactile/haptic feedback to button presses on touchscreen: actuator is only on screen itself, not on whole device, so that actions on simulated buttons feel more like actual buttons, etc. Click, touch, takes into account "lift-off" outside of button as a form of Undo gesture. [QinS01a] Qin, Sheng-feng; Wright, David K.; and Jordanov, Ivan N. "On-Line Segmentation of Freehand Sketches by Knowledge-Based Nonlinear Thresholding Operations", Jnl. Pattern Recognition, Vol 34 No 10, pp. 1885-1893, 2001 Segmentation of free-hand sketches (from tablet) based on corner points and eight-directional chain code. [QualiLife02] QualiLife SA "QualiEye User's Manual", www.qualilife.com Pointing device using head motion: assistive technology device to replace mouse/tablet. Compare with Don Drumm "Orb" device which used head tilt and simple speaking tube for mouse clicks, rather than head rotation for both position and clicks [Quantum02a] Quantum Research "Quantum Research Application Note: QMatrix Panel Design Guidelines", Quantrum Research Application Not AN-KD01, 10 October 2002 Matrix scanning touch panel / touchscreen for touch keyboards: keys need not correspond to matrix crossings, can be anywhere, varying shape/size. Electrode pairs, touch/proximity changes capacitive coupling between pairs. Gives options on electrode design: pairs of electrodes in parallel, series, spirals. Grounding traces nearby worsen performance due to additional coupling to ground (e.g. with moisture). [RabaeyJM03a] Rabaey, Jan M. "Digital Integrated Circuits, 2nd Edition", Prentice Hall, 2003 Undergraduate textbook on digital circuit design. Description of half-adders and adders from logic/boolean elements. [ReadJC03a] Read, Janet C.; MacFarlane, Stuart; and Casey, Chris ""Good enough for what?" Acceptance of Handwriting Recognition Errors by Child Users", Proc. 2003 Conf. on Interactive Design for Children, poster paper, page 155 Wizard-of-Oz study (simulated recognition): brighter children were consistently less satisfied with the same recognition performance than less-bright children. Acceptable performance seemed to be around 91%. Contrast with LaLomia study for adults. [RekimotoJ01a] Rekimoto, Jun "GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices", Proc. ISWC '01, 5th Int. Symp. on Wearable Computers, Page 21 Capacitive sensors on wrist and elbow to recognize hand gestures, sensor wearable on clothes for sensing positions of fingers for one-handed gestures and typing. See also SmartSkin, and Smith paper "Electric Field Sensing For Graphical Interfaces< [RekimotoJ02a] Rekimoto, Jun "SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces", Proc. CHI 2002, April 20-25, 2002, Minneapolis Minnesota Multi-touch touchpad capacitive digitizer in a table surface, uses both position and distance of hand to surface to simulate manipulation of objects projected visually from overhead (e.g. playing cards). Second prototype is tablet/pad size. Hand-gesture recognition, bimanual (haptic) interface. Videos of the two prototypes in use, showing UI uses for multiple finger touches, gestures, proximity. Potential field interaction: objects repelled by proximity of hands or arms, sweeping gesture without contact (compare with Wilson optical flowfield?) [RekimotoJ02b] Rekimoto, Jun "Smartskin Demo: tablet-sized multi-touch capable device (2002-2003)", Sony Laboratories, 2002. Available on youtube.com, fetched 2014 Brief demo of Rekimoto multi-touch touchpad capacitive digitizer tablet, showing multi-touch/multi-finger move/rotate of multiple objects. [RekimotoJ03a] Rekimoto, Jun "ThumbSense: Automatic Input Mode Sensing for Touchpad-based Interactions", Proc. CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida, pp. 852-853 On computers with a touchpad, ThumbSense: Two keyboard keys (F and J) used as mouse buttons when user's thumb touches touchpad. User need not remove hands from typing position. Touching touchpad can also work with further keyboard keys: e.g. ALT-TAB becomes radial pop-up menu. Can also use symmetric matches of keys for left and right hand both to have mouse buttons. In essence, turns touchpad into virtual 36-button mouse or digitizer puck. [RekimotoJ03b] Rekimoto, Jun: Ishizawa, Takaaki; Schwesig, Carsten; and Oba, Haruo "PreSense: Interaction Techniques for Finger Sensing Input Devices", Proc. UIST '03, Vancouver Canada 2003, pp. 203-212 Combined proximity touchscreen/ keypad/small keyboard input device: user can do gestures over/on keypad e.g. swipe, or gesture in conjunction with key e.g. press key then swipe off, or swipe with other finger. Two-finger (two-thumb) typing on same multitouch keypad. For limited size devices. Also preview commands when user places finger near/on key, actuate command when key pressed. tap-and-hold versus press-and-hold (2 touches vs. 1). Compare with Casio 8000 touch-pad/key-pad calculator, 1986? [RekimotoJ03c] Rekimoto, Jun: Ishizawa, Takaaki; and Oba, Haruo "SmartPad: A Finger-Sensing Keypad for Mobile Interaction", Proc. CHI 2003, April 5-10, 2003, Ft. Lauderdale, Florida



Combined proximity touchscreen/keypad/small keyboard input device. For limited size devices. Touch gestures over/on keypad/keyboard e.g. swipe, virtual job (circular gesture). Preview commands when user places finger near/on key, actuate command when key pressed : useful when keypad is modal (e.g. TV remote control OR telephone OR text input), user can see command before comitting. Compare button gestures with Casio 8000 touch-pad/key-pad calculator with handwriting recognition on keypad, 1986? [ResmanMF02a] Resman, Mark F. "Touch-Sensitive Input Screen having Regional Sensitivity and Resolution Properties", United States Patent 6,459,424, October 1, 2002 Resistive (?) touch screen that is less sensitive at edges, to achieve palm rejection. Or less sensitive at edges, used as scrolling area [RhoadsGB02a] Rhoads, Geoffrey B. "Methods and systems for watermark processing of line art images", United States Patent 6,449,377, September 10, 2002 Steganographic (invisible) watermark/data-encoding in line drawings (bank notes) by encoding data in variance of line width, offset/position, luminance, etc. of lines. Digimarc, see also 5,850,481. Compare with Dataglyphs? [RobbinsSJ01a] Robbins, Steven John "Display Method for selectively rotating windows on a computer display", United States Patent 6,326,978, December 4, 2001 Rotate a GUI window that has a frame [RobothamJS02a] Robotham, John S. and Johnson, Charles Lee "Gesture-Based User Interface to Multi-Level and Multi-Modal Sets of Bit-Maps", United States Patent Application 2002/0015064, Published February 7, 2002 Mouse emulation using stylus gestures dependent on time: press-and-hold, includes animated cursors as feedback. [RosenbergLB01a] Rosenberg, Louis B. and Brave, Scott B. "Method for providing force feedback to a User of an Interface Device based on Interactions of a Controlled Cursor with Graphical Elements in a Graphical User Interface", United States Patent 6,219,032, issued April 17, 2001 Active haptic feedback in a touchpad. Very extensive prior art listing, including tele-operation and time delays back to 1980's. [RosenbergLB02a] Rosenberg, Louis B. and Riegel, R. "Haptic Feedback for Touchpads and Other Touch Controls", United States Patent 6,429,846 B2, issued August 6, 2002 Active haptic feedback in a touchpad. Vibration and pulse feedback [RowleyH02] Rowley, Henry; Bennett, John; Goyal, Manish "The Effect of Large Training Set Sizes on Online Japanese Kanji and English Cursive Recognizers", Eighth Int'l Workshop on Frontiers in Handwriting Recognition IWFHR '02, June 13, 2002, pp 26-ff Larger training sets results in better recognition. Training set was hand-tuned, so that infrequently-used letters were not undertrained. [RubioJM02a] Rubio, Jose Moreno and Janecek, Paul "Floating Pie Menus: Enhancing the functionality of Contextual Tools", Proc. UIST '02, 2002, Demonstrations Companion, pp. 39-40 Pie menus (marking menus) made transparent and floating/movable, so that user can see tools underneath. Also menus can be "locked" open, so that user can make additional selections/adjustments without having to go through entire menu heirarchy again. Compare with tear-off menus? [SalasP01a] Salas, Pito; Beir, Jeffrey; Leffler, Melissa; Glenn, McDonald; Kleppner, Paul; Morrissey, Craig; and Tonra, James E. Jr. "Method and apparatus for controlling access to a product", United States Patent 6,314,408 November 6, 2001 Project management and organization system [SandbachDL02a] Sandbach, David L. "Detector constructed from fabric having non-uniform conductivity", United States Patent 6,369,804, April 9, 2002 Touch detectors (touchpad) composed of wires woven into fabric: resistive sheet determines X/Y position on a sensor by voltage gradient, and mechanical force , alternative uses multiple sensors for detecting multi-touch input. [SandbachDL02b] Sandbach, David L. "Detector constructed from fabric", United States Patent 6,452,479, September 17, 2002 Touch detectors (touchpad) two resistive layer composed of wires woven into fabric, separated by conductive layer. [Sazawal02] Sazawal, Vibha, Want, Roy, and Borriello, Gaetano "The Unigesture Approach: One-Handed Text Entry for Small Devices", Proceedings of the 4th International Symposium on Mobile Human-Computer Interaction Tilt-to-write: not writing with a stylus, but tilt hand-held unit (e.g. PDA, phone) to select character from circular menus in zones: compare to Quikwriting, etc. Intended for use in one-hand hand-held devices, not for writing. [ScanlanT01a] Scanlan, Timothy "Background Paper on Pen-Based Computing", Available at www.ring-writer.com, published 2001 Background to ring-writer : Expectations for pen-computing starting with PenPoint and Pen Windows not



realized. Mentions EO, Active Book, Newton PDA. Apple iBook (clamshell PC). PenWindows gave pencomputing a bad reputation, after elegant PenPoint. Proposes that mobile (hand-held) computers fail because neither handwriting recognition nor voice (speech) recognition will work well enough: Linear-X writing system of ring-writer: pie menu where user draws sequentially though three-letter segments (like phone keypad), software recognizes words from dictionary. [SchaerliN01] Schaerli, Nathaniel "Genie - An Introduction", www.iam.unibe.ch/~schaerli/smalltalk/genie/genieIntroduction.htm Gesture and handwriting character recognition in Smalltalk, for the Squeak project coordinated by Alan Kay. Does not give samples of gesture shapes or symbols. Description of symbol dictionaries, but no graphics. 2005 Author moves to Google R&D Lab in Zuerich. [ScherotterMS01a] Scherotter, Michael S. "The Charette Project", http://home/attbi.com/~mscherotter/RightPane.html "Is There a Need for a Sketch Tablet?" proposal for sketch tablet for architecture applications [SchnarelCB02a] Schnarel, Charles B.; Lomelin, Mauricio; Chin, Peter G.; Graham, Derek S.; Welker, Kate; Murdock, Robert C.; Cooper, Justin C.; Golding, Laurecne; and Walsh, Suzanne "Common visual and functional architecture for presenting and controlling arbitrary telephone line features", United States Patent 6,389,124, May 14, 2002 On smartphone (with touchscreen) control windows with UI (call slips) pop up for incoming calls, active calls. Has CIDCO iPhone Screen Phone circa 1999 (touchscreen desktop telephone with web access). [SchneidermanR02a] Schneiderman, Ron "The Mobile Technology Question and Answer book: A Survival Guide for Business Managers", American Management Association, 2002 (hardcopy book) Business tutorial on wireless, cellular, and other technologies: includes location-based services, Softwaredefined radio, Smart cards, 3G, wireless PKI [SchneidermanR03a] Schneiderman, Ron "Technology Lost: Hype and Reality in the Digital Age -- Your guide to making better personal and high-tech investment decisions", Prentice-Hall, 2003 (hardcopy book) Business tutorial on investment "busts": HDTV, LEO (Low Earth Orbit) telecommunications, home networks, DSL, Voice Recognition, e-books (before Kindle and iPad), smart cards. [SchomakerL01a] Schomaker, L. "From handwriting analysis to pen-computer application", Electronics And Communications Engineering Journal, June 1998, pp. 93..102 Good Overview article on the problems of pen-computing development, though not specific on applications development. Pen-and-paper not a good metaphor for pen-computing electronic ink applications. Electronic ink as a (neglected) datatype in its own right: does not mention JOT standard of 1991. Refers to early research efforts (but not commercial efforts) in 1980s as focusing on booklet/portable units, pen gestures. States that Alan Kay's Dynabook had electronic paper in 1968. Claims pen-computing and communications in same unit first demonstrated by EO in 1993. Notes that training to 5,000,000 characters in the Unipen database is not enough to achieve good character recognition. Mentions pen gestures. [SchomakerL01b] Schomaker, L. "The UNIPEN Project History", Internet posting: http://unipen.nici.kun.nl/unipenhistory.html Description of the UNIPEN format (similar to JOT) for a large data collection of handwriting samples for research into recognition, handwriting dynamics, etc. [SchomakerL02a] Schomaker, L. "Pen & Mobile Computing: resources", Internet posting: http://hwr.nici.kun.nl/pen-computing/pen-computing-container.html User Interface in Pen Computing; handwriting history; InfoPad; Pen Linux; Paragraph [SchomakerL02b] Schomaker, L. "UNIPEN Frequently Asked Questions (FAQ)", Internet posting: unipen.nici.kun.nl/unipen-faq.html Collection of handwriting examples for discrete and cursive on-line character character recognition on tablet [SchomakerL02c] Schomaker, L. "UNIPEN hierarchy levels viewed with upview and with upworks", Internet posting: unipen.nici.kun.nl/unipen-faq.html Collection of handwriting examples for discrete and cursive on-line character character recognition on tablet: format definition [SchonfeldE02a] Schonfeld, Eric "Are we ready for digital paper?", ZDNet, March 11, 2002 Skeptical new article on application utility of Anoto patterned paper digitizer: does not mention sound recording for notes, or indexing. Mentions also OTM Technologies, E-Pen inMotion, Digital Ink -- similar patterned technology? [Schumer02] Schumer, Alfred J. "Schumer vs. LCS/Telegraphics and Wacom", Appellate decision in Schumer vs. LCS/Telegraphics (a.k.a. pointing.com) and WACOM WinTab tablet driver with rotation and transformation of virtual tablets, and demultiplexing of input. Compare with Foley 1990 on Windows Input Event Handling. (Additional references also in file.) [ScottSD03a] Scott, Stacey D.; Grant, Karen D.; and Mandryk, Regan L. " System guidelines for co-located



collaborative work on a tabletop display", Proc. ECSCW'03, European Conference on Computer-Supported Cooperative Work 2003, Helsinki, September 14-18, 2003, 20 pages. Tabletop/whiteboard systems. On tablettop touchscreens, users rotate/reorient objects (windows) for shared viewing, to highlight. Mentions non-oriented user interfaces (no menus or command windows). Importance of synchronous user interaction versus turn-taking. Many citations to tabletop systems. [Sedaker01a] Sedaker, Steve "WACOM FID Presentation: Tablet PC: Pen input is back!", Flat Information Displays Conference, Stanford Resources, Monterey, California December 12, 2001 Techology/history review of resistive-film, electrostatic/capacitive, ultrasonic, and electromagnetic digitizers integrated with displays. Asserts that inductive (electromagnetic) digitizer required for hover/proximity (although Scriptel electrostatic did it, perhaps references is restricted to COTS components). Special section on integration problems of electromagnetic digitizer (rear-mounted) with display: inverter noise/EMI shielding, power supply coil on motherboard, edge effects from metal frame. Special laboratory to re-calibrate digitizers incorporated with each display design. Mentions that Tablet PC spec requires proximity. Paper copy is preferable. [Sekendur02] Sekendur, Oral F. "Optical Position Determination on Plain Paper", United States Patent Application Publication US2002/0158848 A1, October 31, 2002 Similar to Anoto / Bennett / Sekendur, except does not use pre-printed pattern. Instead a CCD camera captures images of the surface as-it-is, and builds up a map of the image of the surface to use instead of the pre-printed pattern. CCD camera for optical digitizer is mounted on a gymbal / universal joint (sic), so as to be able to view the entire surface [Sekendur02a] Sekendur, Oral F. "Optical Position Determination on Any Surface Paper", United States Patent Application Publication US2002/0163511 A1, November 7, 2002 Similar to Anoto / Bennett / Sekendur, except does not use pre-printed pattern. Instead a CCD camera captures images of the surface as-it-is, and builds up a map of the image of the surface to use instead of the pre-printed pattern. Specifically refers to the surface being the screen on a cell phone. CCD camera for optical digitizer is mounted on a gymbal / universal joint (sic), so as to be able to view the entire surface [SellenA03a] Sellen, Abigail J. and Harper, Richard H.R. "The Myth of the Paperless Office", M.I.T. Press, 2003 (Summary of full book) electronic paper not eliminating physical paper: physical paper has many affordances: portability, note-taking, physical/muscle memory of location in text, physical delivery to a person raises importance of communication over electronic communication, etc. Compare with Anoto references. [SensorsMag02a] Hoffmann, Franz-Josef: Segmann Inc. "A New dimension in Encoder Technology", Sensors Magazine, May 2002 Review of rotary encoders incremental/absolute, circular barcode for absolute rotary position, sin/cos cycle pattern. [ShakedD03a] Shaked, Doron; Baharav, Zachi; Levi, Avi; Yen, Jonathan; and Saw, Chit Wei (Nick) "Graphical Indicia", Proc. Intl. Conf. on Image Processing ICIP 2003, Sep. 2003, Vol 1 pp 485-488 two-dimensional bar code pattern in images by modulating halftone patterns, includes digital signature for authentication and integrity. Compare with Anoto pattern, Digimark authentification of printed documents, Xerox data glyphs (cited). [ShanisJ03a] Shanis, Jenna and Hedge, Alan, J "An Exploration of Multi-Touch Technology: Comparison of a splitangle Keyboard with Multitouch and Current Input Technologies", Proc. Human Factors and Ergonomics Society 47th meeting, Oct 13-17, Denver CO Presentation to accompany "Comparison of mouse, touchpad, and multitouch input technologies". Describes multiTouch as using a touch-matrix surface, versus a touch grid. Mostly review of ergonomic of a multi-touch keyboard. [ShawSJ03a] Shaw, Scott J.; Day, Shawn P.; Trent, Robert A. Jr.; Gillespie, David W.; and Errington, Andrew M. "Capacitive Mouse", United States Patent 6,587,093, July 1, 2003 Mouse with capacitive/touch sensors instead of mechanical buttons. Notes that touchpad can emulate mouse, but user experience is not same (ergonomics). Ball rotation / rotary encoder also detected via capacitive sensor vs. optical. Coasting/momentum behavior based on velocity of finger motion as finger lifts, simulated friction. [ShellSR02a] Shell, Scott R.; Shields, Kevin Timothy; and Kitowitz, Anthony "Loading Status in a Hypermedia Browser Having a Limited Available Display Area", United States Patent 6,339,780, January 15, 2002 Showing a busy/loading animation while web-browser content is loading [ShneidermanB03a] Shneiderman, Ben "Promoting Universal Usability with Multi-Layer interface Design", Proc. CUU '03, November 10-11, 2003, Vancouver, Canada Design user interfaces with "layers" of menu/gesture option: simple features, intermediate, advanced. Useful to include show-me animations. Example on US Census map data. [ShwartsSL01a] Shwarts, Scott L. and Dunham, David R. "Tool Set for Navigating Through and Electronic Book",



United States Patent 6,243,071, June 5, 2001 Electronic book GUI: on-screen navigation (page turn, etc.), electronic ink annotation / mark-up, text search, bookmarks, TOC/table of contents browser links, scroll operations. Scroll arrows are fixed (Newtonstyle), not on-screen. [SiegelSH02a] Siegel, Steven H. "Method for the auditory navigation of text", United States Patent 6,442,523, August 27, 2002 Speak user interface elements with auditory (voice) output when user does mouse-over or hover on element or text. Cites to outSpoken, compare with other screen readers e.g JAWS, or TI speak-and-spell for teaching reading to children, Multi-touch display from 1970's educational tool. [Siemens01a] Siemens AG "KidPad - das kindgerecht Online-Multimediagerät", Innovations-Report, June 21 2001 (In German) Slate / WebPad touchscreen device for children -- not the same as KidPad drawing software. Includes steerable camera for videochats, authentification of user (child) using fingerprint scanner. [SignerB03a] Signer, Beat "Beat Signer: Paper++, Digitally Augmented Paper, XIMA, iServer, Digital Paper", Web posting, www.inf.ethz.ch/~signer, 2003 Research summary posting: Integration server (iServer) with cross-media linking between paper and other electronic media types: compare with Anoto. Automatic generation of links. [SignerB03b] Signer, Beat "Paper++ Disappearing Computer Programme", Web posting, www.inf.ethz.ch/~signer, 2003 Review of Paper++, linking of paper documents to other systems, appears to use Anoto optical stylus/digitizer? [SilverbrookK03a] Silberbrook, Kia and King, Tobin Allice "Hand Held Mobile Phone with Integral Internal Pointer", United States Patent 6,628,430, September 30, 2003 Touchscreen cellphone/smartphone with built-in printer and optional video camera and display for videoconferencing. [SinasohnR02a] Sinasohn, Roger "Device information GRIDCase 3 ruggedized / GRID Compass 1101 Clamshell-GRIDPad 1910 ruggedized", Uncle Rodger's Computers, www.sinasohn.com, copyright 2002 GRiD Touchscreen computers: also AST PenExect pen-based slate/convertible [SingerM02a] Singer, Michael "Text Display with User-Defined Appearance an Automatic Scrolling", United States Patent Application 20020130891, September 19, 2002 Automatic scrolling of text so that user does not have to scroll the text manually through a document, in a special window [SinghM02a] Singh, Mona and Lands, Robert M. "Display Control for Hand-held Data Processing Devices", United States Patent 6,400,376, June 4, 2002 Scroll/pan a virtual page using a simulated button (area) on a touchscreen digitizer -- also with position sensor of device. Compare with Freestyle "Guide"? [SinghM02b] Singh, Mona "Movable Magnification Icons for Electronic Display Screens", United States Patent 6,359,615, March 19, 2002 Movable magnifier (rectangular example) in touchscreen user interface to magnify small text to tmake it more readable. Compare with AppLens. [SklarewR01a] Sklarew, Ralph "Handwritten Keyboardless Entry Computer System", United States Patent 6,212,297, April 3, 2001 Abstract refers to transparent sensing surface over display, electronik ink, character recognition, and learning mode Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer" Long list of prior art [SmailagicA02a] Smailagic, Asim and Siewiorek, Daniel "Application Design for Wearable and Context-Aware Computers", IEEE Pervasive Computing, Vol 1 No 4, Oct-Dec 2002, pp. 20-29 Wearable Computers, cites ParcTab with touchscreen interface, Folio foldable display pen computer, Digital Ink pen computer. Cites to wearable computers with video for master/apprentice training. [SmartTech03a] SMART Technologies Inc. "SMART Board 3000 interactive whiteboard", www.smarttech.com SMART board interactive whiteboard digitizer: has separate physical button on Pen Tray for right mouse button. DViT (Digital Vision Touch) -- optical sensor for touchscreen? [SmartTech03b] SMART Technologies Inc. "SmartTech Smart board software for Windows Whiteboard", www.smarttech.com SMART board interactive whiteboard digitizer: has separate physical button on Pen Tray for right mouse button. Projected Mode on physical screen: calibration for pointing with finger. Smart AWARE software injects electronic ink data (and handwriting recognition?) into compatible Windows applications. Floating Tools toolbar. Smart Keyboard on-screen virtual keyboard with handwriting recognition, similar to SIP in W4PC. DViT (Digital Vision Touch) -- optical sensor for touchscreen? [SmartTech03c] SMART Technologies "Bridgit Conferencing Software User's Guide", www.smarttech.com Whiteboard conferencing software for the Smart Whiteboard product. WebCam image shown on the



desktop via the overlaying whiteboard/conferencing software [SmithBA03a] Smith, Barton Allen and Zimmerman, Thomas Guthrie "Digital pen using interferometry for relative and absolute pen position", United States Patent 6,592,039, July 15, 2003 Laser/optical digital pen for tablet on any surface: if surface has bar coded pattern, determine absolute position, of no pattern, uses speckle (?) pattern to track relative position. 3D force sensor also in stylus/pen: used for rotation correction (?) and for touch/contact sensing. Compare with Anoto, Brouhon 07? [SmithiesS01a] Smithies, Steve; Novins, Kevin; and Arvo, James "Equation Entry and Editing via Handwriting and Gesture Recognition", Behavior and Information Technology, Vol 20 No 1, January 2001, pp. 53-67 handwriting/gesture user interface for inputting and editing two-dimensional mathematical expressions. Cites to bibliography of mathematical and algebraic systems and user interfaces [SnyderC01a] Snyder, Carolyn "Paper prototyping", Web posting, 1 November 2001, snyderconsulting.net Paper prototyping: sketch out user interface and interactions with users using literal paper, scissors, and sticky notes. Contains references to papers on usability testing, low- and high-fidelity prototyping systems, and accessibility (handicapped) resource centers at IBM. [SnyderC03a] Snyder, Carolyn "Paper prototyping: The Fast and Easy Way to Design and Refine User Interfaces", Excerpt from book of this title, Morgan Kaufmann, pubs. 2003 Paper prototyping: sketch out user interface and interactions with users using literal paper, scissors, and sticky notes. Traces history to physical mock-ups on paper back to early 1970s, for usability studies on mail sorting devices, etc. [StifelmanL01a] Stifelman, Lisa; Arons, Barry; and Schmandt, Chris "The Audio Notebook: Paper and Pen Interaction with Structured Speech", Proc. CHI 2001, March 2001, CHI Letters Vol 3 No 1, pp. 182-189 Combine digitizing tablet with regular notebook paper on top, microphone and speakers for audio: User can play back their notes to see them drawn in real time, with the matching audio from a lecture. Cites to Digital Desk in describing opposite tack to computer user interfaces. Compare with Anoto audio/digitizer pen on regular paper, Wang Freestyle (not cited) [StorkA02a] Stork, A.; Santos, P.; Fleisch, T.; and Araujo, B. "SmartSketches: A Multimodal Approach to Improve Usability in the Early States of Product -- Recommendations for Base Hardware and System Software", Technical Report D5, Contral IST-2000-28169, Information Society Technologies Programme (Spain) November 20, 2002 Prototype sketching/drawing system for pen-based/tablet system using tilt, pressure (tip force): Gimp only application to use tilt/pressure. XInput on X-windows supports touchscreens, tablets, other devices in addition to mouse and keyboard. Uses WINTAB driversw (see) and Wacom tablets, OpenGL with alpha blending. [StrachanJP01a] Strachan, John Paul "Instrumentation and Algorithms for Electrostatic Inverse Problems", Masters Thesis, EECS, M.I.T., September 2001 Thesis on general use of electrostatic/capacitive sensing for determining position. Example application is 3D mouse. Trade off in resistivity and leakage currents between resistive/conductive sheet and electrode sensors. duPont carbon film on cardboard (not ITO on glass). Multiple methods for measuring capacitance e.g. RC time constant in response to voltage step. [StrickonJA03a] Strickon, Joshua Andrew "Smoke and Mirrors to Modern Computers: Rethinking the Design and Implementation of Interactive, Location-Based Entertainment Experiences", Ph.D. Thesis, Program of Media Arts and Sciences, M.I.T., February 2003 Thesis on experimental applications and user interfaces: includes Laser Rangefinder multi-touch touchscreen tracking multiple hands in front of large rear-projected images. Tapper (by Joe Paradiso) sensing touches/taps on transparent/ordinary glass, determines both position and intensity/amplitude. [SuGM03a] Su, Guan-Ming; Wu, Min; Liu, K.J. Ray "TabletPC Programming: ENEE408G Multimedia Signal Processing, Pen Computing and Tablet PC Programming Manual, Using Microsoft Visual Studio.NET C# and Tablet PC Platform SDK", Course Notes October 7, 2003, Dept. Elec. and Comp. Eng., U. of Maryland at College Park Show Tablet PC ink objects and ink fields: compare with PenPoint and with PenApps of Slate. [SuhB02a] Suh, Bongwon and Bederson, Benjamin B. "OZONE: A Zoomable Interface for Navigating Ontology Information", Proc. AVI 2002, Trento, Itally, pp. 139-143 zooming in/out: double-click gesture zooms object/document to original size and moves to center of screen. Animation of zooming provides context. Semantic zooming: when object is zoomed small, be smart about showing less text but font still at readable size. [SynapseAdaptive01a] Synapse Adaptive / Alva Technology "Active Areas in outSPOKEN 3.0 / Outspoken Solo 3.0 Screen Reader Software for Windows 95 / Windows NT", www.synapseadaptive.com, fetched 2014 outSpoken screen reader: active areas defined in particular application windows using trigger and action: e.g. trigger on change in text in field (Word document showing text name and size), action to read/speak text. Defined relative to window or relative to entire screen [SynapseAdaptive01b] Synapse Adaptive / Alva Technology "Feature highlights of outSPOKEN Ensemble 3.0 /



outSPOKEN software for Windows 3.0", www.synapseadaptive.com, fetched 2014 outSpoken screen reader: active areas definitions, automatic detection/reading of message boxes, tool tips, warning dialogs, double-tap/double-touch gesture for right mouse button. [Synaptics01a] Synaptics Incorporated "New Touch Screens Improve Handheld Human Interface", Synaptics White Paper WP-12, P/N 507-000003, Rev A. Synaptics Incorporated, San Jose California, 2001 Resistive touch-screens use PET polyethylene terephthalate film with ITO Indium Tin Oxide resistive film. Reference to spacer dots/bumps, similar to Elographics. Capacitive sensing can be front-mounted, but fewer layers than resistive touchscreen tablet digitizer. ClearPad (capacitive) and Spiral (inductive) digitizer tablets. Vague reference to proximity/hover sensing with ClearPad capacitive digitizer. States that capacitive film has proximity sensing. Defines touchscreen, ITO, PET Polyethylene terephthalate [Synaptics01b] Synaptics Incorporated "Synaptics TouchPad Interfacing Guide, Second Edition", Synaptics Incorporated, Document 510-00080-A, January 22, 2001 Transparent capacitive grid touchscreen, for PS/2, serial protocol RS-232, and Apple Macintosh ADB protocols. States supports mouse input plus Synaptics extensions (gestures). Proximity sensing, force/pressure (actually contact area) of finger or hand. Capacitance/Z values affected by environment. "W" mode reports multi-touch contact: 1, 2, 3 fingers (capMultiFinger)or full palm (capPalmDetect), or pen (capPen). Principles of operation: includes some internal noise filtering, recommends additional filtering for fine positioning. Touchscreen firmware supports tap, drag, edge-motion, scrolling, tap-and-a-half (tapslide/tap-glide) gestures: further documentation in driver pack. Code sequences on software to read serial interface. IsDragEnabled, IsTapEnabled APIs. [Synaptics03a] Synaptics Incorporated "Transparent Capacitive Position Sensing / Transparent Capacitive Force Sensing / Inductive Position Sensing", Synaptics Incorporated, San Jose California, 2003 Descriptions of Capacitive, Inductive, and resistive tablet technologies from Synaptics. ClearPad is grid of two layers of ITO, determine accurate location of finger by signal strength at multiple intersection nodes. Describes combining capacitive (finger-touch) and inductive (stylus) to work with both, or to work with resistive: proximity sensing capacitive digitizer for pen computing [Synaptics03b] Synaptics Incorporated "Comparison of Resistive and Capacitive Technologies (for touchscreens)", Synaptics Incorporated, San Jose California, 2003 Synaptics comparison of resistive (resistive/conductive sheet with air gab) with electrostatic grid. Does not discuss grid decoding, or multiple touches or palm rejection [TabletPC01a] Microsoft "Bill Gates announcement of Tablet PC as keynote speech at Comdex 2001", Press Reports TabletPC first announced in 2001, for availability in 2002 [TabletPC01b] Microsoft "Windows XP Tablet PC Edition FAQ: Tablet PC Frequently Asked Questions: What is a Tablet PC exactly?", www.microsoft.com, November 9, 2001 Description of Tablet PC as PC running Microsoft Windows XP Tablet PC Edition, with pen and speech input. Link to Comdex 2001 demonstration by Bill Gates. [TabletPC02a] Microsoft "Pressemitteilungen: Microsoft TabletPC beim CeBIT, WINHEC, usw. / Press clipping file", c't, 2002, others Press clippings on announcement of Tablet PC at CeBIT and other conferences. Electromagnetic stylus for improved display quality; Acer TravelMate small-format convertible. "TabletPC: Gates Lieblingskind". [TacTex01a] TacTex Inc. "Express multi-touch pressure-sensitive touchpad", www.tactex.com, 2001 TacTex Kinotex Smart Fabric: MTC-Express mouse-pad sized multi-touch pressure (force) sensing controller. File contains multiple references. [TandlerP01a] Tandler, Peter and Prante, Thorsten "Using Incremental Gesture Recognition to Provide Immediate Feedback while Drawing Pen Gestures", Proc. UIST 2001 Recognition of pen gestures is error-prone: recognize gestures as they are made, if gesture can no longer fit any recognition template, stop recognizing and treat as electronic ink (scribble gesture): mentions audio feedback on recognition of pen gestures [TappertCC01] Tappert, Charles C. and Mosley, Pauline H. "Recent Advances in Pen Computing", Technical Report Number 166, August 2001, School of Computer Science and Information Systems, Pace University, While Plains, New York Virtual Reality (?) via Whiteboard project "Flatland" at Xerox PARC, Kumon Machine project for teaching mathematics via pen computing (tablet computer for children). List of pen computer / tablet PC / PDA products as of 2000: Handspring, PalmTop, Pen*KEY from Intermec, Sharp ZR-3500X, Casio Cassiopeia. Pen-computing games [TayYH01a] Tay, Yong Haur; Khalid, Marzuki; Lallican, Pierre Michel; Knerr, Stefan; Viard-Gaudin, Christian "An Analytical Handwritten Word Recognition System with Word-level Discriminant Training", Proceedings of ICDAR 2001, pp 726-730 Hidden Markov Model HMM for optical recognition of handwriting; points out need to include "junk" data



(things that can be mistaken for characters) as well as "good" data in training set [Tealpoint01] Tealpoint, Inc. "TealScript User's Manual Version 2.30", http://www.tealpoint.com, 2001 Replacement for Graffiti unistroke character/stroke recognizer: trainable recognition, for the Palm Pilot PDA Refers to matching system, using a match-percentage and squelch values for character recognition; mentions recognition/stroke/character macros, up to seven characters [TedescoDE02] Tedesco, Daniel E.; Walker, Jay S.; and Kessman, Marc D. "Method and Apparatus for Performing Supplemental Searches over a Network", United States Patent 6,349,295 B1, February 19, 2002 Asynchronous search of remote (network) database on another thread from a word-processing program. [Telepen03a] Telepen Barcode Systems "Telepen Barcode Systems: Barcode Reader - Scanners", www.telepen.demon.co.uk, fetched 2003 Product information on barcode scanners and labels. Original project pen-shaped hand-held barcode reader [TenGO03a] The Edge, Singapore "Doing the TenGO", The Edge, Singapore, May 19, 2003, Page E small-format touchscreen virtual keyboard for PDAs/smartphone. Combination of grouped keys (e.g. 26 keys of alphabetic divided into six groups). User types/taps on keys, dictionary match to words that could be spelled with letters of those keys. Use with other pointing device for accessibility: compare with other accessibility heirarchical keyboards. Compare with touch-tone address look-up? [TenGO03b] Xrgomics "TenGO User Manual Version 1.0 (Palm Version)", Xrgomics, Inc. September 26, 2003 Small-format touchscreen virtual keyboard for PDAs/smartphone. Combination of grouped keys (e.g. 26 keys of alphabetic divided into six groups). User types/taps on keys, dictionary match to words that could be spelled with letters of those keys. [Tenon03a] Tenon Intersystems "Xtools X Windows Server User's Guide", Tenon Intersystems, 2003 Xtools windowing overview and installation guide: X11, multiple Window managers: twm, Motif, mwm[ThomasEA03a] Thomas, Eric Anthony "Methods apparatus and computer program products for providing user input to an application using a contact-sensitive surface", United States Patent 6,624,832 Touchscreen/digitizer with double-ended stylus, determined by number of contact points on end of the stylus: compare with Freestyle double-ended stylus. Uses SAW digitizer that can determine multiple points of contact. Cited to Agulnick patent. multi-touch: Carroll Touch SAW digitizer/touchscreen [ThornerC02a] Thorner, Craig and Heilman, Paul M. III "Universal tactile feedback system for computer video games and simulations", United States Patent 6,422,941, July 23, 2002 Haptic feedback vibrator seat or vest. (Compare "butt-tickler). Can be retrofit (host-independent) to an existing game by responding to the sounds output by the electronic game, or can be controlled by commands (host-dependent) from the computer. Effects include crashes. [TillyC01a] Tilly, Chris "The History of Windows CE", HPC: Factor, February 18, 2001. Fetched 2014, 2003 History of Windows CE development, originated with WinPad/Pegasus/Pulsar projects circa 1996 with handwriting recognition in handheld/palmtop device. Mentions RISC processor support. Stylus as mouse equivalent. ActiveSync. [TokiokaM01a] Tokioka, Masaki; Tanaka, Atsuhi; Yoshimura, Yuichiro; Yanagisawa, Ryozo; Kobayashi, Katsuyuki; and Sato, Hajime "Coordinate Detecting Device for Outputting Coordinate Data when Two Points are Simultaneously Depressed", United States Patent 6,255,604, July 3, 2001 multi-touch (two points) on resistive film digitizer: Additional current from two points of contact indicates two points, rather than one, firmware uses most recent single point and second point. [Tomitsch03] Tomitsch, Martin "Trends and Evolution of Window Interfaces: Discussion on Interfaces and Interaction Techniques for Window Management", Master's Thesis, University of Technology, Vienna, December 2003 Review of Windows interfaces, mentions post-WIMP (see van Dam) gesture UI user-interfaces: pie-menu "marks" (marking menus) or quick gestures with mouse -- direction is easier to do with mouse than distance or position for a pop-up menu. Zomit navigation menus for mouse. Bruce Tognazzini on gesture input (from Tog's Blog, 2003), including hand gestures and other input. Gestures on whiteboard systems. Task Gallery project included audible (musical?) feedback on correctly-recognized gestures. [TomsHardware03] TomsHardware.com "The Digital Writing Pad: Tablet PCs", http://www.tomshardware.com Review of Tablet PCs, mentions Paceblade Pacebook Tablet PC. Uses term digitizer display to refer to a digitizer integrated with a flat display. [Toshiba02a] Toshiba Singapore Pte Ltd "Touch Technology for Today and Tomorrow", Tech Brief/Emerging Technologies: Toshiba Singapore, 16 September 2002 Synaptics cPad touchscreen/digitizer in Toshiba laptops: touchpad for cursor control, signature capture. X/Y grid of ITO conductors forming capacitors at cross-over points. Pressure (equivalent) determined from amount of capacitance change from finger, location is cross-over with peak change, comparing adjacent electrodes (interpolation) for more precise location. Refers to including haptic feedback on digitizers. [Toshiba02b] Gateway Computers "Microsoft Windows/XP Tablet PC Edition convertable laptop circa 2002",



Toshiba (physical device) Tablet PC convertible: obtained as working device, may need stylus. [Toshiba03a] Toshiba America Electronic Components "Toshiba America Electronic Components demonstrates first system on glass (SOG) input display with built-in image capture", Toshiba America, May 12, 2003 touchscreen/"input display" integrated LCD display and 2D matrix of optical pixel sensors together. One application is laying business card on display to scan it in. [TsukadaY02a] Tsukada, Yujin and Hoshino, Takeshi "Layered Touch Panel: the Input Device with Two Touch Panel Layers", Proc. CHI 2002, April 20-25, 2002, Minneapolis, pp. 584-585 IR infra-red optical touch sensor over contact-screen touch digitizer: essentially giving you proximity sensing when finger interrupts IR optical beams before touch. gestures include pick and drop, rollover gesture. [TuovinenAP02a] Tuovinen, Antti-Pekka "Object-Oriented Engineering of Visual Languages", PhD Thesis, Series of Publications A, Report A-2002-1, U. Helsinki, Finland, February 2002 VILPERT (Visual Language Expert) system for implementing Visual languages (e.g. LabView): parsing and grammatical models, with context (rather than context-free implementation). Includes drag-and-drop (visual) language editor with nesting/containment. [UDelaware02] University of Delaware "UD researchers develop revolutionary computer interface technology", University of Delaware Press Release, September 27, 2002 Fingerworks multi-touch technology by Wayne Westerman and John Elias, company later acquired by Apple for the iPhone. Mentions simulated devices on a touch-screen -- compare with Kaplow? [UedaT03a] Ueda, Tsutomu and Ooue, Toshikazu "Pressure-sensitive conductive ink composition", United States Patent 6,555,024, April 29, 2003 Pressure-sensitive ink, like conductive rubber: curable ink material with conductive filler (small particles?). Example of matrix/grid pressure/touch sensor [UllmerB01] Ullmer, Brygg and Ishii, Hiroshi "Emerging Frameworks for Tangible User Interfaces", "Humancomputer Interaction in the New Millenium", Addison-Wesley, August 2001, pp. 579-601 Review article on tangible user interfaces, and augmented physical/tangible objects: cites to Buxton and Shneiderman among others. [UlrichRR01a] Uncapher, Kieth W. "Pattern and Color Abstraction in a Graphical User Interface", United States Patent 6,239,795, May 29, 2001 Color/style schemes in windows-based UI (Apple MacIntosh): theme style also can change shape of windows (rounded corners, etc.) not just colors, by changing the presentation layers for user-interface objects. [Uncapher02] Uncapher, Kieth W. "Biographical information for Kieth W. Uncapher, Senior Vice President, CNRI", http://www.cnri.reston.va.us/bios/Uncapher.html: CNRI, 1895 Preston White Drive, Suite 100, Reston, Virginia 20191-5434 He managed ... the RAND Tablet project which included real time recognition of hand printed characters using a tablet and stylus, in 1961 [VaeaenaenenJ02a] Vaeaenaenen, J. "Method and device for browsing information on a display", United States Patent Application 2002/0175896, November 28, 2002 Scroll/browse by tilt of handheld device: scroll to position of tilt (absolute), not in direction of tilt (relative) [ValdesR01] Valdes, Ray "DDJ Handprinting Recognition Context Wrap-up", Dr. Dobb's Journal, July 22, 2001. Available at www.ddj.com Review of handprint recognition systems provided by readers. Most recognizers were "ad-hoc" (rulebased), less than 2000 lines of code. [VanBrocklinAL03a] Van Brocklin, Andrew L.; Oross, Glen A.; Bausch, James F.; and May, Gregory J. "Optical Pointing Devices", United States Patent 6,552,713, April 22, 2003 Optical trackball with no moving parts: top surface is curved lens, optical sensors below detect motion of finger across top. [VanDerVlistE02a] van der Vlist, Eric "XML Schema, First Edition", O'Reilly ISBN 0-596-0252-1, 2002 (hardcopy book) XML schema reference [VanWestJ02a] Van West, Jeff "Tablet PC Quick Reference", Microsoft Publishing, November 6 2002 (abstract only) TOC only: Input Panel options, Send as Text vs. Send as Ink, Journal and Notes note-taking applications for electronic ink: Virtual Keyboard in Tablet PC, Tablet PC / Windows gestures. [VernierF02a] Vernier, Frederic; Lesh, Neal; and Shen, Chia "Visualization Techniques for Circular Tabletop Interfaces", Mitsubishi Electric Research Laboratories Technical Report TR2002-01, March 2002. www.merl.com: to appear in Advanced Visual Interface conference, May 200, Trento Italy Layout and visualization for multiple users at multi-person touchscreen tabletop browsing



documents/photos together: tree heirarchies oriented towards user. Prototype uses overhead projection an Mimio ultrasonic digitizer stylus. [VietsRR02a] Viets, Richard R.; Motes, David G.; Greve, Paula Budig; and Herberg, Wayne W. "System and method for controlling access to documents stored on an internal network", United States Patent 6,357,010, March 12, 2002 Project management and organization system [VijayaKittuM03] Vijaya Kittu, M. "Keep a Tab on the Tablet PC / What makes the Tablet PC Tick / The Tablet PC Arrives", Information Technology, January 2003 pp 45..50, 59..61, December 2002 pp 14..16 Refers to Tablet PC as a "Slate Computer", but does not mention Slate Corporation. Overview of Input Panel for handwriting recognition input, gestures, Windows Journal, Sticky Notes. Mentions ISF Ink Serialized Format (similar to JOT format standard for electronic ink?). Text Services framework for handwriting recognition adds dictionary alternatives, using "factoids" for type of input: field type EMAIL, DIGIT, TELEPHONE, POSTCODE. [Vitis02] Vitis, Pavel "Gestures: mouse stroke controlling system", lsg.euweb.cz/show.php3?f=gestures/README, fetched 2002 Description of Gestures, simple mouse stroke controlling system as used in Opera browser, but works on whole system: stroke gesture recognition for mouse input [VonHardenbergC01a] von Hardenberg, Christian and Berard, Francois "Bare-Hand Human-Computer Interaction", Proc. ACM PUI 2001, Orlando Florida, pp. 1-ff. Optical/video detection of finger pointing and hand gestures. Gives examples of image differencing against reference image of background to detect additional object (hand) and motion of hand, finger-finding algorithm using pixel width of fingers and relative widths of fingers (e.g. thumb. versus little finger), segmentation using colors as well as image subtraction. Dropped frames (could not find finger due to moving) generally less of a problem than miss-classification of final pointing position. Multiple-finger gestures on image of desktop: multi-touch. No citation to Wellner? [VongW02a] Vong, William and Schwitters, Chad "Handheld computing device with external notification system", United States Patent 6,374,276, April 16, 2002 On/Off or deactivation button for alarm with built-in LED light on hand-held PC with a touchscreen. Similar appearance to IBM Simon. [VuoriV02a] Vuori, Vuokko "Adaptive methods for on-line recognition of isolated hand-written characters", Acta Polytechnica Scandinavica, Mathematics and Computing Seris No. 119, December 14, 2002 Adaptive handwriting recognition for discrete characters, training intended to occur during normal use (not separate enrollment). Uses dynamical time warping: compare with elastic matching by Tappert? Multiple citations to Ward/Kuklinski for handwriting variability. [VuurpijlL01a] Vurrpijl, Louis and Schomaker, Lambert "dScript: an agent-based handwriting recognition demo", hrw.nici.kun.nk/recog/dscript multiple simple classifiers/recognizers for handwriting recognition, in a multiple-agent architecture [W3C03a] W3C www.w3.org "Requirements for the Ink Markup Language", W3C Note 22 January 2003, version www.w3.org/TR/2003/NOTE-inkreqs-20030122/, www.w3.org/TR/inkreqs Exchange format requirements for electronic ink data interchange: handwriting and gesture recognizers, signature verifiers, ink-aware (pen-aware?) applications on Web. Includes pen angle, pen tip force (pressure), sampling mode, sampling rate, screen context. Compare with JOT standard. [Wacom02a] Wacom Components "Pen Input is back!", Wacom Components: press release, June 10, 2002. Announcement by WACOM at IBC Mobile Devices conference concerning Wacom tablets and Microsoft Tablet PC. Wacom integrating electromagnetic resonance (cordless stylus) tablets with LCDs for mobile touchscreen computers since 1985. [WakeamJ03a] Wakeam, Jamie "Ink Analysis with the Tablet PC SDK", Microsoft MSDN, May 2003 API and structures for Divider object to add segmentation manually to electronic ink, such as for strokes the developer assigns algorithmically to characters, to words, or to sketches [WalkerG01] Walker, Geoff "A detailed look at Microsoft's Proposed Tablet PC project", Pen Computing Magazine, July 2001 General review of June 2000 and COMDEX November 2000 Bill Gate's demonstrations of Tablet PC prototypes. TabletPC announced by Microsoft without reference to previous Windows for Pen Computing or previous decade of portable pen-computing products. Microsoft Notebook ink notetaking application for Tablet PC: acquired from Aha Software: InkWriter. Describes moving ink words as though they were text: compare with van Raamsdonk. Delayed or background recognition: compare with Slate deferred recognition. Windows Pen Services 1.0 "MARS" handwriting Recognizer, Pen Services 2.0 used GRECO recognizer, Windows CE 2.0 used CIC's Jot recognizer at the last minute, 1999 purchase of Calligrapher non-exclusive handwriting recognizer, combined with in-house gesture recognizer (?). Term "reco fusion" for combining multiple recognizers. Prototype used FinePoint Innovations digitizer.



[WalkerG01b] Walker, Geoff "Microsoft Tablet PC History: A detailed look at Microsoft's proposed Tablet PC (2001)", RuggedPCReview.com, 2001 Extensive review of history of Tablet PC from Microsoft, comparison with Windows for Pen Computing / PenWindows history. Electromagnetic digitizer ("active" tablet/touchscreen). [WalkerG02a] Walker, Geoff "Tablet PC Update", Pen Computing Magazine, Spring 2002 Review of Tablet PC, announced by Alex Loeb at WinHEC2001. Questions from Geoff Walker to Alex Loeb, comparing GO / PenPoint pencentric approach to Windows for Pen Computing / Tablet PC adding pen/stylus/gesture functionality to mouse-centric OS. [WalkerG02b] Walker, Geoff "Tablet Taxonomy: Special Report: WebPads", Pen Computing Magazine, January 2002, available at www.pencomputing.com Taxonomy of pen-computing digitizer tablet pen computers: Internet Appliance (Compaq Internet Appliance, Sony eVilla, Virgin WebPlayer, Gateway TouchPad, Compaq IA-2), Thin Client (terminal: AirSpeak Flair, NCD ThinSTAR), WebPad (Windows CE, QNX, Midori Mobil Linux, CE Tablet (Fujitsu PenCentra), Pen Tablet (SONICblue ProGear, Tablet PC. Says Tablet PC hardware will not ship until 2002. [WalkerG03a] Walker, Geoff, editied by Purdy, J. Gerry "Tablet PC Report", MobileTrax Enterprise IT Service, Cupertino, CA, August 2003 Business review of Microsoft Tablet PC system. OneNote replaces Windows Journal -- describes use of ink notes an annotations in Microsoft Office: Word, Excel. "Active Digitizer" providing hover/proximity sensors (Wacom digitizer) and FinePoint innovations. Credits Fujitsu with originally creating Pen Tablet market in 1993. Detailed list of application vendors and hardware vendors. [WallaceH03a] Wallace, Hugh and Gordon, Gary B. "Optical screen pointing device with inertial properties", United States Patent 6,621,483, September 16, 2003 optical trackball with no moving parts: finger moves across lens, optical sensors below detect motion. Emulates flick motion/gesture of physical/mechanical trackball: motion stops when user touches again. User-definable acceleration/decelleration characteristics/parameters. [WanBo02] Wan, Bo and Watt, Stephen "An Interactive Mathematical Handwriting Recognizer for the Pocket PC", ML conference 2002, Presentation Papers Two-dimensional parsing of mathematical expressions: assumes symbols are written left to right; no references to work before 1998 [WangALC03a] Wang, Avery Li-Chun "An Industrial-Strenth Audio Search Algorithm", Proc. 4th Int'l Conf on Music Information Retrieval, 2003. Also White paper, Shazam Entertainment Ltd., Palo Alto CA, 2003 Rapid search algorithm for identifying music over low-quality audio system: cellphone microphone, background noise, mixing with other tracks. Describes combinatorially hashed time-frequency contellation analysis to compute signature for comparison with signatures/audio fingerprints in a database. Constellation map of feature points extracted from spectrogram. [WangJ03a] Wang, Jintao "Online Handwriting Recognition Technology -- State of the Art", Presentation notes. Group for User Interface Research, [email protected] Primarily about hand-held devices since 1997, several references to Chinese/Asian product with handwriting recognition States that classifier/recognizer is not the most influential factor on recognition accuracy, but does not state what. mentions Lexicus, Paragraph (licensed to Microsoft), DragonPen, PenPower [WardDR01] Ward, Daniel R; Wood, Robert P.; Harel, Jacob; Holtzman, Rafi "Pen Positioning System", United States Patent 6,184,873 B1, February 6, 2001 Electronics for Imaging: ultrasonic/acoustic transducers in the pen, mentions pressure-sensitive tip also using ultrasonics: cites Blesser/Philbin/Philips patent for spatial filters, apparently as prior art for tilt, height, and parallax correction [WardJR03a] Ward, Jean Renard "System and method for ad hoc management of credential, trust relationship, and and trust history in computing environments", United States Patent Patent Application Filing 20030163686, August 28, 2003 Community-based electronic credentialing using non-CA PKI. Non-heirarchical alternative to traditional PKI, making use of a management system separate from a CA Certifying Authority (a.k.a. Certification Authority) for credentials from multiple sources. [WebPad01] WebPad "Die Web-Pads kommen!", c't magazine, number 16, 2001, page 120 Review of stylus or touchscreen digitizer notebooks with wireless web access: mostly running WindowsCE, mentions TabletPC in 2001 [WestermanW01a] Westerman, Wayne and Elias, John G. "Multi-Touch: A New Tactile 2-D Gesture Interface for Human-Computer Interaction", Proc. of Human Factors and Ergonomics Society, 2001, pp. 623-636 Multi-touch surface using capacitive sensor array, used to recognize multi-finger gestures for graphical manipulation. Gesture examples detect up to four finger touches, plus pointing input like a mouse. Cites to educational studies with children. Also cites Lee, Buxton, and other earlier multi-touch devices. Chord



gestures are recognized, then user can continue with pointing or other input without a separate action (button) for mode switch between typing and pointing, etc. [WestermanW01b] Westerman, Wayne and Elias, John G. "Method and Apparatus for Integrating Manual Input", United States Patent 6,323,846, November 27, 2001 Multi-touch digitizer, electrostatic/capacitive. Uses an X/Y grid of capacitive sensor points. The touchscreen sensor technology later used in the Apple iPhone. Large family of patents. Includes ellipse fitting for multitouch surfaces to track finger touches. Virtual trackball device with accelleration. [WestermanW03a] Westerman, Wayne Carl and Elias, John Greer "Multi-touch System and Method for Emulating Modifier Keys via Fingertip Chords", United States Patent 6,570,557, May 27, 2003 multi-touch surface (MTS) touchpad used for chorded input in a touch-sensitive keyboard, up to four fingers simultaneously. Thumb input stops key input, so that hand can be left rested, thus reducing palmrejection problem. By using multi-touch gestures, need not regard absolute position of a virtual keyboard. [WienerkeM01a] Wienercke, Markus; Fink, Gernot A.; and Sagerer, Gerhard "A Handwriting Recognition System Based on Visual Input", 2nd Int'l Workshop on Computer Vision systems, Vancouver, Canada, 2001, pp. 63-72 Combination of optical/video digitizer to track pen position, and dynamic optical character recognition. Target is for PDAs, use built-in camera and regular pen and paper instead of small display touchscreen. Compare with Wellner, MetaDesk, etc.? [Wikipedia02a] Wikipedia.com "Sony Ericsson P800", Wikipedia.com, fetched 2012 Sony smartphone/PDA with touchscreen interface: Symbian operation system [Wikipedia03a] Wikipedia.com "Sony Ericsson P900", Wikipedia.com, fetched 2012 Sony smartphone/PDA with touchscreen interface: Symbian operation system [WilksBG01a] Wilks, Barry G. and Murphy, Christopher T. "Method and Apparatus for Overlaying a Window with a Multi-state Window", United States Patent 6,246,407, June 12, 2001 Window can be made transparent/translucent when inactive/disabled, user can see and manipulate window/object underneath. When active, window is opaque, and processes input itself. [WilsonGD03a] Wilson, Geoffrey D.; Borgnis, Victor E.; Kent, Joel; Lewis, Mike; Loucks, Drew; Roney, James; and Patti, Michael Bruno "Touch Confirming Touchscreen Utilizing Plural Touch Sensors", United States Patent 6,504,530, January, 2003 Combine two types of sensors in a touchscreen, and only count as a touch when both detect. E.g. a force sensor (may be a single sensor for the whole touchscreen) to discriminate real touches from spurious signals in an acoustic/SAW touchscreen due to contamination on the surface. [WobbrockJO02a] Wobbrock, Jacob O.; Forlizzi, Jodi; Hudson, Scott E.; and Myers, Brad A. "WebThumb: Interaction Techniques for Small-Screen Browsers", Proc. UIST '02, October 27-30, 2002, Paris, pp. 205-208 On a small touchscreen PDA, show thumbnail of full normal web page or application, user can select portions of thumbnail image and get just that portion zoomed up in a separate part of the display. Thumbnail as a whole can also be zoomed/panned/scrolled. [WobbrockJO03a] Wobbrock, Jacob O.; Myers, Brad A.; and Hudson, Scott E. "Exploring Edge-Based Input Techniques for Handheld Text Entry", Proc. ICDCSW 03, 23rd Int'l. Conf. on Distributed Computing Systems, pp. 280-282 Edge Keyboard / EdgeWrite unistroke alphabetic handwriting, user moves to corners and sides of small hand-held touchscreen with a frame: recognition depends only on order of corners, not on continuous contact. Intended for difficulties of writing while holding a small device. [WobbrockJO03b] Wobbrock, Jacob O.; Myers, Brad A.; and Kembel, John A. "EdgeWrite: A Stylus-Based Text Entry Method Designed for High Accuracy and Stability of Motion", Proc. UIST '03, Vancouver BC Canada, pp. 6170 Intended as accessibility aid for users with motor empairments (cerebral palsy, etc.). Edge Keyboard / EdgeWrite unistroke alphabetic handwriting, user moves to corners and sides of small hand-held touchscreen with a frame: recognition depends only on order of corners, not on continuous contact. Cites to Quikwriting, plastic templates over touchscreens. Edge Keyboards / virtual on-screen keyboards require constant visual attention, EdgeWrite does not. [WuM03a] Wu, Mike and Balakrishnan, Ravin "Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays", Proc. UIST '03, Vancouver, Canada, pp. 193-202, CHI Letters Vol 5 No 2, 2003 RoomPlanner: prototype room furniture layout application. Multi-finger/Multi-hand interaction techniques with focus on whiteboard application. Prototype with DiamndTouch multi-touch table, identifies users by coupling. Gesture include tap, double-tap, press-and-hold (?), horizontal and vertical hands, open/flat hand, two corner-shaped hands, two vertical hands. RoomPlanner: Object manipulation uses partiallytransparent objects (furniture). [YaegerL02b] Yaeger, Larry "Larry Yaeger, Personal correspondence", Personal Correspondence, April 2002 Correspondence on baseline for handwriting recognition for descenders, etc. [YagerT02] Yager, Terry "Agnes: The Pen is Mightier Than ...", www.vintage-computer.com



Review of Toshiba T200 Pen-Pc running Pen Services 2.0 on Windows 3.1. States that gestures (in version Win4Pen Windows for Pen Computing 1.0) were easier to use than the pen-enabled toolbar of Windows 95 for backspace, undo, delete, new-line, etc. [YangL03a] Yang, Li; ZhiWei, Guan; GoShong, Dai; XiandShi, Ren; and Yong, Han "A Context-Aware Infrastructure for Supporting Applications with Pen-Based Interaction", Jnl. of Computer Science and Technology, Vol 18 No 3, May 2003, pp. 343-353 WEAVER, pen-enabled/pen-aware enhancements for Windows, hooks into existing GUI code. Includes contact information for specific application (compare with JAWS?), e.g. in word-processing arrow gesture means move, lasso/closed-loop gesture is multiple selection. Chinese handwriting character recognition with pop-up editing menus. Separate gesture-based word processor. Cites to Windows for Pen Computing (PenWindows), PUI (Pen-based user interface), WinTab specification. [YanoT01a] Yano, Takashi; Tanaka, Makoto; Tsuda, Kunikazu "Write point detecting circuit to detect multiple write points", United States Patent 6,229,529, May 8, 2001 In a SAW or optical-beam touchscreen, can detect more than one point of touching. Disambiguate which one is a write point (stylus) by it being closer to where the stylus was on the previous sample. [YaoP02a] Yao, Paul "Email correspondence, "Re: Publications on handwriting/cursive baseline"", Email note to [email protected], April 2, 2002 Assertion that estimation of handwriting baseline prior to recognition, instead of making it part of handwriting recognition, can *hurt* recognition success [YocumT02a] Yocum, Todd and Peurach, Thomas M. "Haptic device attribute control", United States Patent 6,433,771, August 13, 2002 Haptic Force feedback on mechanical manipulators, adjustable for amplitude, phase, frequency, and spring gain constant [ZanibbiR01a] Zanibbi, Richard, Novins, Kevin, Zannibi, Katherine, and Arvo, James "Aiding Manipulation of Handwritten Mathematical Expressions through Style-Preserving Morphs", Graphics Interface 2001, June 2001, pp 127-134 Cleans up / pretty-fies handwritten mathematical electronic ink by applying some normalization of the electronic ink, making use of some heuristics of mathematical syntax, but not full character/symbol recognition, including use of writing baseline [ZeleznikR01a] Zeleznik, Robert; Miller, Timothy; and Forsberg, Andrew "Pop Through Mouse Buttons: A Simple Hardware Change and its Software UI Impact", Proc. UIST '01, ACM 2001, pp. 195-199 Add force sensor to mouse button to allow user to press and hold harder for pop-through menus: unlike double-click does not require user to hold mouse without moving. [Zizzi01] Zizzi, Stephen "Method of Transparent Encryption and Decryption for an Electronic Document Management System", United States Patent 6,185,691, February 6, 2001 Encrypt automatically when saving a file, decrypt when opening: claims all cryptographic functions, which would probably include digital signatures. Re-issued with new claims. Subject of press reports on patents needing re-review. [Zoebisch03] Zoebisch, F. and Vielhauer, C. "A Test Tool to support Brute-Force Online and Offline Signature Forgery Tests on Mobile Devices", IEEE International Conference on Multimedia and Expo 2003 (ICME), 2003 Signature verification/ handwriting verification / biometrics on Palm PDAs, and on Windows using Wacom tablet. Primarily about training users to produce forgeries, based on giving users only the clear-text, lowforce where users see static image of writing only, and brute-force where users see dynamic display of writing. [Zongker02] Zongker, Doug "Chicken Chicken Chicken: Chicken Chicken", PoCSi 02: Potentially Computer Science Conference May 2002, U. Washington. Available at http://isotropic.com Chicken chicken chicken chicken chicken chicken chicken chicken chicken: chicken chicken chicken. (Geek humor In English) [Zytronic02a] Zytronic PLC "Zytronic Zytouch Projected Capacitive Touch Screens", Zytronic.com.uk, 2002 and later Capacitive grid touchscreens using X/Y grid of microfine (.01mm) wires, approximately 16x16 wires for a small display. Additional documents (through 2012: same product line) in electronic file. Support drivers include Windows 95 and Windows 98. Mentions select-on-dwell driver support: press-and-hold. biblio00 (Prev page)

Main Page


"You can win all the battles, and still lose the war." -- Samurai saying "Die Zeit ist nur das, was man an der Uhr abliest."



-- Albert Einstein, bekannter Physiker und Geigenspieler





Main Page


[AgrawalaM05a] Agrawala, Maneesh and Shilman, Michael "DIZI: A Digital Ink Zooming Interface for Document Annotation", Proc. INTERACT '05, IFIP Int'l Conf. on Human-Computer Interaction, 2005, pp. 69-79 Electronic ink annotation of electronic documents (PDFs). Electronic ink sloppier than real ink (no explanation given), so hard to read. Magnify a window portion of document, so annotation can be written larger. Input window automatically scrolls/expands to create more space. Flick and rectangle gestures to initiate zoom annotation window. Compare with zoomable electronic ink annotation in Slate PenDayTimer? Demonstration video in electronic file. [AlvaradoC04a] Alvarado, Christine J. "Multi-domain Sketch Understanding", Ph.D. Thesis, M.I.T, Dept. of EECS, August 19, 2004 domain-flexible recognition systems for sketch recognition: recognize components such as line, vertex (high curvature, low speed motion), ellipse, arc, polyline via a primitive recognizer/fragmenter (parse), match to shape descriptions for shapes of domain (Electronics, architecture, etc.) using constraint-based pattern matching. [AlvaradoC04b] Alvarado, Christine and Davis, Randall "SketchREAD: a multi-domain Sketch Recognition Engine", Proc. UIST 2004, pp 23-32 Pen-based UI: examples of bottom-up and top-down semantic context aiding sketch recognition, discovery of gestures and symbols: family tree diagrams, circuit diagrams. [Anoto04a] Anoto AB "Press Release: Anoto won US patent case", Anoto AB Anoto/Sekendur patent litigation [Anoto05a] Google Answers: about.com "About Anoto.com", answers.google.com, fetched 2005 General information on Anoto and Anoto technology for optical pattern paper pen/digitizer. Business model is to license portions of Anoto identifying pattern to companies providing business forms (FiloFax, DayTimer, etc.), for automatic data capture and tracking. Also cites to OMTech VPen, accelerometer pen/digitizers, sonic tablets, [Apple05a] Apple Computer, Inc. "Apple Human Interface Guidelines", Apple Computer, 2005 User interface guidelines for Apple MacIntosh computers: use of standard components (address book,



etc.). Mouse: can substitute other direct-manipulation devices such as trackballs and stylus pens (tablet). Press-and-hold behaviors: repeated mouse clicks (scroll bar), display a (pop-up) menu: generaly same as click. Universal accessibility: applications should not override built-in accessibility features, such as all features accessible via keyboard without mouse. Low-vision graphics (dialog layout). [AriService04a] ari-service.com "AST Computer Information Web: ARI Service", www.ari-service.com, fetched March 2010 Support site with downloadable files for AST products, such as GRiDPad pen computer with touchscreen. Includes: PenCell Working Modek, PenCell Help file; On-Screen Keyboard for GridPAD, Pen-based mouse Driver for DOS Apps, Unsupported PenRight! development tools, GRID 4025 BIOS, Win4PEN Drivers, PenRight! Driver for GRiDPAD. CD with some of the files in physical files: Web site says it would be taken down in 2009 [AscensionTechnology04a] ascension-tech.com "Ascension Technology: The Flock of Birds - Position and Orientation Measurement System, Installation and Operation Guide for Standalone and Multiple Transmitter/Multiple Sensor Configurations", Ascension Technology Corporation, 2004 3D/6DOF "tablet" position sensors, used in a number of hand-gesture projects, virtual reality inputs (i.e. sense position and orientation of toy "gun". Multiple small electromagnetic sensors with cables to match one or more multiple base transmitters. Original version of manual 1991. [AvitzurR04a] Avitzur, Ron "The Graphing Calculator Story", www.pacifict.com, last updated 2004, fetched 2014 Story of graphing calculator project for Apple PowerPC in 1993/1994, continued by developers after project cancelled and contractors laid off by sneaking back into the building, with informal collusion by engineering staff at Apple. [BaarDJP04a] Baar, David J.p.; Carpendale, M. Sheelagh T.; Cowperthwaite, David J.; Tigges, Mark; Komar, Robert; and Bauer, Jerome F. "Elastic Presentation Space", United States Patent 6,768.497, July 27, 2004 User-interface, view a document at a zoomed-back scale, zoom in / magnify one portion selectively. Adjacent parts of document blend/zoom gradually to the selected portion. Compare with citations to/from AppLens. [BalakrishnanR04a] Balakrishnan, Ravin ""Beating" Fitt's law: virtual enhancements for pointing facilitation", Int'l. Journal of Human-Computer Studies, Vols 61, pp 857-874, 2004 Fitts' law on pointing with a stylus/pointer can be "beat" by making target area larger, snapping-to the target, area cursors, etc. But this cannot be applied when there are lots of targets. File contains additional citations by Balakrishnan with other authors. [BaldwinDR04a] Baldwin, David Robert "Doubly-virtualized texture memory", United States Patent 6,683,615, January 27, 2004 Graphics buffers taken from paged main memory, locked down when needed for display. Citations are to caching of texture maps: compare to other real-time caching? [BaudischP04a] Baudisch, Patrick; Pruitt, John; Ball, Steve "Flat Volume Control: Improving Usability by Hiding the Volume Control Hierarchy in the User Interface", Proc. CHI 2004, April 24-29, Vienna, Austria slider/toggle controls in graphical user interface (touchscreen) [BaudischP04b] Baudisch, Patrick and Gutwin, Carl "Multiblending: displaying overlapping windows simultaneously without the drawbacks of alpha blending", Proc. CHI 2004, April 24-29, Vienna, Austria, pp. 367-374 Transparent windows: instead of alpha blending, show overlay window as completely transparent except for GUI element lines and labels, which are shown with edge-enhancement contrast, etc. Less obscuring of window / background below. [Bechtel04] Bechtel, J. Scott; Blanc, Arthur; Black, Gerald R. and Gustin, Jeffery L. "Developing The World's First Biometric Writing Instrument", Purdue University Innovation Realization Laboratory White Paper, January 30, 2004 See also Pen-One, Incorporated, www.pen-one.com. Several patent references for Gerald R. Black: United States Patent 6,307,956, etc. Fingerprint Sensor Pen: Writing pen with fingerprint sensor built-in: appears not to be biometric signature verification. [BedersonB04a] Bederson, Benjamin B.; Clamage, Aaron; Czerwinski, Mary P.; and Robertson, George G. "DateLens: A Fisheye Calendar Interface for PDAs", ACM Trans. on Computer-Human Interaction, Vol 11 No 1, March 2004, pp. 90-119 Windows Pocket PC: Zoom user interface into region of interest / degree of interest or information of interest in a small display: e.g. tap on a date in a calendar month view, get the day information with the rest of the calendar month pushed out to the sides. Mentions work on DataLens and fish-eye menu techniques back to 1982. [BedersonB04b] Bederson, Benjamin B.; Grosjean, Jesse; and Meyer, Jon "Toolkit Design for Interactive Structured Graphics", IEEE Trans. on Softw. Eng, Vol 30 No 8, August 2004 GUI toolkit using run-time composition of objects, rather than compile-time class heirarchy. Example is photomesa using Zoomable User Interface and Jazz. Zoomable user interface zooms by showing more



semantic information and reducing other semantic information, rather than simply magnifying. [BeezerJL05a] Beezer, John L. et al "Method and Apparatus for Capturing and Rendering Annotations for NonModifiable Electronic Content", United States Patent 6,957,233, October 18, 2005 Storing annotation separate from document with link back to location in document: numerous citations, but compare with Anoto, Slate and InkWriter [BellG04] Bell, Gordon "A Time and Place for Standards", VoiP Magazine, Vol 2, No 6, September 2004 Essay on positive/negative role of technical standards in promoting or impeding innovation, quotes "Better is the Enemy of Good Enough". Quote from Stewart Alsop regarding PenPoint vs. Pen-Windows: "If GO had ever been given the chance, they would have succeeded.\". [BiagiottiL05a] Biagiotti, Luigi; Borghesan, Gianni; and Melchiorri, Claudio "A Multimodal Haptic Mouse for Visually Impaired People", Proc. ENACTIVE05, 2nd Int'l. Conf. on Enactive Interfaces, Genoa Italy November 17-18, 2005, 7 pages Force-feedback haptic mouse for the blind: used pantographic tablet with force transducers. User can "feel" graphical objects (square, circle) by simulated potential-energy force. [BishopCM04a] Bishop, Christopher M.; Svensen, Markus; and Hinton, Geoffrey "Distinguishing Text from Graphics in On-line Handwritten Ink", Paper for verbal presentation, IWFHR, Tokyo, October 26..29, 2004 Classifying/recognizing electronic ink on tablet computer as text or as sketch: based on length, other characteristics of strokes. Segmentation based on time (time-stamps for cut-and-paste ink data, deferred recognition). Mentions time information in applying context rules (Viterbi algorithm). [BlaskoG04a] Blasko, Gabor and Feiner, Steven "Single-Handed Interaction Techniques for Multiple PressureSensitive Strips", Proc. CHI 2004, pp. 1461-1464 Using Synaptics TM41P-200 capacitance-based TouchPad, reliably detect multiple (up to fourteen?) simultaneous finger touch inputs (multi-touch) in simulated sliders, in user interface to control multiple audio or other parameters at the same time. Pressing harder (pressure-sensing) gives access to secondary pop-through virtual slider. Synaptics touchpad has output bit to indicate two-finger (wide) input. No electronic modifications to Synaptics touchpad. Compare with Buxton sliders? [BodnarEO04a] Bodnar, Eric O. "Graphical password methodology for a microprocessor device accepting nonalphanumeric user input", United States Patent 6,686,931, February 3, 2004 On small device, navigation by small keypad/keyboard by single-click operation or taps on touchscreen, drill-down for more detailed information (e.g. select name to get full record). For no full alphanumeric keyboard/keypad, recognize path through a set of icons as password for user authentication. [Boesen04] Boesen, Peter V. and Mann, Thomas J. "Method and Medium for Computer Readable Keyboard Display Incapable of User Termination", United States Patent 6,784,783, August 31, 2004 Touchscreen simulated/virtual keyboard in a portable device (shown using stylus with cord). Compare with Kaplow? [BookAndComputer05] Book and the Computer, The "The Dynabook Revisited: A Conversation with Alan Kay", http://www.honco.net/os/kay.html Interview with Alan Kay on history of Dynabook as educational tool: cited elsewhere as mentioning pen computing [BovesL04a] Boves, Lou; Neumann, Andre; Vuurpijl, Louis; ten Bosch, Louis; Rossignol, Stephane; Engel, Ralf; and Pfleger, Norbert "Multimodal Interaction in Architectural Design Applications", Proc. 8th ERCIM Workshop on User Interfaces for All (UI4All), Vienna, 2004, pp 384..390 Experimental Tablet pen-computing UI with voice/speech commands for drawing blueprints/layouts (for a bathroom), synthesized voice output for computer response. User satisfaction depends highly on handwriting recognition and speech recognition. No mention of gestures. [BrewsterSA04a] Brewster, Stephen and Brown, Lorna M. "Tactons: Structured Tactile Messages for Non-Visual Information Display", Proc. AUIC2004 5ht Australasian User Interface Conference, Dunedin, AUS. pp. 15-23 Tactile/haptic feedback for the blind/visual-impaired: frequency, amplitued, duration, pattern, location of tactile pulse. Used Audiological Engineering VBW32 transducers. Comparison with Braille display / Braillezeiler, VirTouch tactile mouse (braille-like transducers). Graphs and displays for the blind. [BrewsterSA04b] Brewster, Stephen and Dunlop, Mark (eds.) "Proc. Mobile Human-Computer Interaction MobileHCI 2004", Proc. Mobile Human-Computer Interaction MobileHCI 2004, Glasgow UK, September 13-16, 2004 Conference proceedings: Gesture interfaces on Tilt, Touch, and Text Entry; semantic scrolling [BrownL05a] Brown, Lorna "An Investigation into the Design of Structured Vibrotactile Messages for Non Visual Information Display", University of Glasgow, 2005: www.dcs.gla.ac.uk/~eve/tactonsWebsite/vibro.html Vibrating Transducers for haptic vibrotactile/electrotactile touch input with human computer interaction: www.eaiinfo.com Enginering Acoustic C2 Tactor. [BrownL05b] Brown, Lorna M.; Brewster, Stephen A.; and Purchase, Helen C. "A First Investigation into the Effectiveness of Tactons", WHC '05 First Joint Eurohaptics Conf. and Symp. on Haptic Interfaces for Virtual



Environment and Teleoperator Systems, pp. 167Vibrating Transducers for haptic vibrotactile/electrotactile touch input with human computer interaction: simulate roughness with sinusoidally modulated sinusoidal vibration. Square waves give roughly sinusoudal vibration due to mechanical response. Below 20/50 Hz users perceive individual vibrations. [ButlerCG04a] Butler, Coling G. and St. Amant, Robert "HabilisDraw DT: a bimanual tool-based direct manipulation drawing environment", Proc. CHI 2004, pp 1301-1304 Multi-hand drawing program using DiamondTouch multi-user tablet, with addition of gloves so that a single user appears as four users for up to four multi-touch points. [Buxton04a] Buxton, W. "Resource Page on Early HCI Research of the Lincoln Lab TX-2 Group (beta)", http://www.billbuxton.com/Lincoln.html Mentions sources for videos for Sketchpad, Graphical Programming, Genesys, interviews with early researchers before 1966. [Buxton04b] Buxton, W. "Marking Interfaces: Chapter 13 (draft)", http://www.billbuxton.com/input13.markup.pdf Marking interfaces: Buxton's term for for paper-like and pen-based interfaces, gestural / gesture-based user interfaces; line recognition as synonym for sketch recognition Describes character recognition as a "black hole" for pen-computing interfaces Early 1969 animation system Genesys for gesture input Review of numerous paper-like application products from 1980's and 1990's: Newton MessagePad 2000, Liveworks Liveboard, Wang Freestyle, CompuThink Paperless Office, Winfax PRO, CrossPad, etc. Alto User's Handbook (1976) InkWriter by Aha!: recognition of handwritten paragraphs, not of characters QuickWriting unistroke / single-stroke characters Papyrus Allegro unistroke / single-stroke characters Casio AT-550 character recognition watch with finger tracing 1980, Casio DBC-150-1 Databank watch, PF-8000 calculator First shorthand / unistroke: 63 BC, Marcus Tullius, "notae Tironianae" File also contains 2010 draft. [Buxton05] Buxton, William (organizer); Henderson, Austin (Discussant); Baecker, Ron; Clark, Wesley; Richardson, Fontaine;, Sutherland, Ivan;, Sutherland W.R "Interaction at Lincoln Laboratory in the 1960's: Looking Forward - Looking Back", Panel Discussion, CHI 2005, April 2-7, 2005, Portland, Oregon Video and presentation of Sketchpad; very early work on whiteboard "chalk talk" systems by Baecker; early gesture-recognition system by Richardson (?) and Curry (?) at Applicon and Lincoln Labs [CIC04a] CIC Communications Intelligence Corporation "Jot for Palm OS Version 2.1: Help", www.cic.com Documentation on JOT (a.k.a. Graffiti2 for Palm OS) handwriting recognition and neography by CIC. FAQ states that JOT is not a Unistroke character set. Includes reference chart for non-English European characters, Scandinavian, German, special gestures for newline/enter, etc. [CIC04c] CIC Communications Intelligence Corporation "Sign-It: Move into the Age of eCommerce with secure electronic signatures", www.cic.com Technical information, version 4.1, dynamic signature verification as biometric token authentication ("secure electronic signatures"). Signature "bound" to document by including with secure hash (TripleDES) of document and signature. Authentication done in server, plug-ins for document signing fields for Adobe, Word, etc. Note: says template (used for authentication) can also be used as "rubber stamp" to apply signatures to documents? Actual signing act can be live signature, voice recording, stamp, or password(?) "Ceremony" software for capturing timestamp for signature, etc. [CIC04d] CIC Communications Intelligence Corporation "PenX: Pen enablement for modern computing", www.cic.com Technical brochure, CIC PenX Pen-enablement with software virtual keyboard, electronic ink collection, writing-in-place: inking overlay for whole screen with handwriting recognition, input directed towards active field/input. (For small touchscreens). [CMC05] CMC Electronics "CT-1000G Electronic Flight-Deck Organizer with JeppView electronic charts", CMC Electronics, Ville Saint-Laurent, Quebec Canada H4M 2S9 Windows 98 Pen computer for private aircraft, with GPS, mountable on steering controls. Copyright 1999? [Caldwell04] Caldwell, Alex "Handwriting Word Recognizer", Posted to http://wiki.tcl.tk/9094 Describes modification of "Mouse-stroke character recognizer" by Mike All, which in turn borrows code from other sources. Recognizes cursive words written as a single stroke: crossing of T's, etc. not supported. [CampbellV05] Campbell, Virginia "Why Johnniac Can Read: The first big step in handwriting recognition came 40 years ago", American Heritage of Invention and Technology, Summer 2005 Historical article on RAND digital tablet in 1964, "not the first, but the first that was practical". Dimond, GRAIL, Groner Recognition as soon as stylus was lifted, slashed Z and 0 required versus O and 2. Ink was not quite real time, displayed only after stroke lift. Quotes Leonard (Leonid) Kitainik of Paragraph, Apple Newton, penandinternet.com, www.evernote.com [CaoX04a] Cao, Xiang "An Exploration of Gesture-Based Interaction", Master's Thesis, Dept of computer Science, University of Toronto, 2004



Optical (tracking camera) recognition of gestures made with handl-held want: pointing, tilt, tap, paralleltap, push, pull, rotate, flip gestures. Multiple-user input (or two handed input?) with more than one Wang, six-segment pie menus. Adaptive recognition of gestures mostly based on unistroke character shapes, plus pig-tail. [CarveyH05a] Carvey, Harlan "Windows Forensics and Incident Recovery", Addison-Wesley, 2005 (hardcopy book) Forensic examination of Windows (through Windows 95/NT/2003): focuses on using standard event logs, logging, other monitoring tools. Historical section on Robert Morris worm, Clifford Stoll, logic bombs, Slammer worm, brute-force DDOS attack, etc. [Casio05] Casio "Casio VDB-1000 wristwatch", Collection from Pocket Computing Museum, 200, available at http://www.integra.fr/dreamteam/caz/computers/pocket/collection.htm Wristwatch with touchscreen input. Aegis Tele-Pad wristwatch is same device? Same reference includes Casio NX-6000 / Planeo keyboard-less organizer; Sharp IQ-9200 touchscreen organizer; Casio Z7000 Zoomer PDA, like Apple Newton; Apple Newton Message Pad 110 2005: New URL appears to be http://cdecas.free.fr/computers/pocket/if8000.php [Caz04] Caz Pocket Computers Collection "Caz Pocket Computers Collection", http://cdecas.free.fr/computers/pocket/if8000.php Images of various pocket and pen-tablet computers: Ricoh RCD i-700, Sharp MI-10, Casio IF-8000, Sharp IQ-9200, Casio VDB-1000 wristwatch with touch screen (but not handwriting recognition), Kyocera Refalo. [ChangKHP04a] Chang, Kenneth H.P. "Encoding and decoding a message within an image", United States Patent 6,819,774, November 16, 2004 Glyph pattern (compare DataGlyphs) for embedding message (e.g. URLs/address) in an image. Glyph pattern example is individual pixels turned on/off in a monotone black/white image. Includes multiple citations to two-dimensional symbology. [ChristieGN04a] Christie, Gregory N. and King, Nick "Depicting and setting scroll amount", United States Patent 6,714,221, March 30, 2004 Coasting action on scroll/slider after user input stops, scrolling parameters user-adjustable, scrolling stops when user taps scrollpad/touchpad. Scrolling coasting speed decays organically (friction?). [ChrysochoosM05a] Chrysochoos, Michael and Kosinski II, John T. "Radio system with Touch Pad Interface", United States Patent Application 2005/0003851, January 6, 2005 Using gestures on a touchpad to control a mobile radio. Gestures are letters and digits: T for tune, etc. "U" Gesture to unlock, "L" gesture to lock user interface from other input. [ClellandJPR04a] Clelland, John P.R. "Multiple Target Representation using a Haptically Enhanced Visual Display", Master's Thesis, Univ. of Glasgow, September 2004 Prototype air traffic control radar display, haptic/tactile feedback on haptic trackball/mouse used to improve targeting. Prototype uses Immersion Touchware and Studio. Cites to Digistrips for ATC information display, Engel for trackball with contextual force feedback. Immersion Studio as graphical GUI IDE. [CohenPR04] Cohen, Philip R. and McGee, David R. "Tangible Multimodal Interface for Safety-Critical Applications", CACM, Vol 47 No 1, January 2004, pp 41-46 tablet and PC user interfaces distract too much by focusing user attention on the desktop/computer, not on the task at hand Review of "Smart paper"/"Digital pen" in actual use (Rasa: military battle operations) using Anoto technology and other technologies: one key is maintaining operation using the physical ink when the electronic system is non-operational / deliberately crashed Natural Interaction Systems LLC: compare with VIPDATA? Use of combined voice and electronic ink annotation for "mutual disambiguation": compare with Freestyle system at Wang, 1992 (not cited?) Digital Desk by Wellner, 1992, tangible paper system using computer vision [ColoradoBar04] Colorado Bar Association "JurisNotes: Patent containing mere germ of an idea invalid for lack of enablement", Colorado Bar Association Intellectual Property Section Newsletter, Vol. 8 No. 8, November 2004, p. 3 Anoto AB v. Sekendur, patent dispute on optical pattern digitizer. Turn of phrase: plaintiff "miserably failed" [Computerwoche05a] Computerwoche "Bill Gates glaubt immer noch an den Tablet PC", Computerwoche.de, June 28, 2005 Bill Gates still supporting Tablet PC 3 1/2 years after announcement at Comdex 2001 (actually, earlier announcement in 2000). Tablet PC support on Longhorn / Windows 7. [ConferWJ05a] Confer, William James "The Activity Metric for Low Resource, On-Line Character Recognition", Ph.D. Thesis, Auburn University, Alabama, 16 December 2005 Trainable handwriting/gesture recognition using histograms (?) / activity parameters, intended as algorithm for use on very low power processors. Describes Graffiti (CIC) recognition as a variant of Unistroke (Xerox) recognition algorithms.Discussion of T-Cube pie-menu unistroke writing (sic) from Apple. Testing done with Kassel collection of handwriting samples.



[CrawfordGP04a] Crawford, Gregory P.; Cairns, Darran R.; Bowley, Chirstopher C.; Danworaphong, Sorasak; Fontecchio, Adam K.; Faris, Sadeg M.; and Li, Le "Reflective liquid crystal strain gauge with aspected particles and polarization-sensitive devices", United States Patent 6,778,236, August 17, 2004 Optical strain-guauge (not force sensor) that can be applied to a surface, stretching of liquid crystals changes reflectivity. [CrooksCE04] Crooks, Clayton E. II "Developing Tablet PC Applications", Charles River Media, 2004. ISBM 158450-252-5 (hardcopy book) Includes example of using stylus angle for implementing a Joystick UI user interface. Chapter 18 cites to Graffiti for UI for a media player for Video and Audio, using gestures on Tablet PC including arrow gestures. Annotation (handwriting ink) on videos: compare with Freestyle at Wang. [DRSTacticalSystems05] DRS Tactical Systems "DRS Technologies Acquires Walkabout Computers", Press release, June 27, 2005, available at www.drs-ts.com Ruggedized tablet PCs with integrated tablet and display: product line originally from Walkabout, acquired by DRS Tactical Systems [DSInternational05] DSInternational "DigiMemo Digital Writing Pad", Fentek Industries, Cottonwood AZ: www.fentek-ind.com/digimemo.htm Clipboard-styled digitizer with internal storage for electronic ink -- no built-in display [DeHerreraC04a] de Herrera, Chris "Tablet PC Hardware Comparison 2004", www.tabletpctalk.com, 2004 Hardware feature comparisions for Microsoft Tablet PC vendors: all (?) use Wacom electromagnetic touchscreen/digitizer [DeHerreraC05a] de Herrera, Chris "Tablet PC Hardware Comparison - 2004", www.tabletpctalk.com, revised January 2, 2005 Hardware comparison of Tablet PCs' announced since September 1, 2003: essentially all vendors using electromagnetic digitizers by Wacom, a couple of resistive-film touchscreens [Denoue05] Denoue, Laurent and Foote, Jonathan T. "Force-Feedback Stylus and Applications to Freeform Ink", United States Patent Application US 2005/0243072 A1, November 3, 2005 Haptic force-generating stylus which allows the user to feel a force generated electromagneticly; contrasted with other haptic devices, which are able to generate vibration but not actual force feed-back to the user. Compare with force-feedback devices developed at M.I.T. A.I. lab by Margaret Minsky (?) et al around 1980, which used mechanical linkages [DesturaG04] Destura, Galileo; van Berkel, Cees; Cirkel, Peter A. "Touch Sensitive Display Device", European Patent Publication WO/2004/066136 Touchscreen digitizer using a touch sensitive layer between two optical layers. Uses QTC quantum tunnelling composit or piezo-resistive material as the pressure/force-sensitive sensor, to be embedded between the pixels on an LCD display. [DiazMarinoR04a] Diaz-Marino, Roberto Arturo; Tse, Edward; and Greenberg, Saul "The GroupLab DiamondTouch Toolkit - The Video", Video Proceedings of ACM CSCW Conf. on Computer Supported Cooperative Work, Nov 2004 Video Demonstration of UI Toolkit applications using DiamondTouch multi-touch (multi-user) touch tablet: multi-user/multi-touch drawing application, multi-user memory game, sound-manipulation art [DietzPH05a] Dietz, Paul H. Leigh, Darren L.; Yerazunis, William S.; and Lee, Johnny Chung "Hand-held Haptic Stylus", United States Patent Application 2005/0248549, Nov. 10, 2005 Haptic stylus with solenoid actuator in top of pen stylus, feedback effect is along axis of stylus [Digibarn04] Digibarn Computer Museum "PenPoint tablet pen computer by GO Corporation", Available at http://www.digibarn.com/collections/systems/go "Lombard" hardware prototype from GO Corporation, manuals, documentation, photographs. Several manual pages shown in readable photographs. Recorded interviews with donor Gary Downing . [DigitalInk04] Digital Ink, Inc. "Company contact information: Digital Ink, Inc.", Digital Ink, Inc., 2113 Wells Branch Parkway, Austin, TX 78750 Gary L. Barrett Pointers, touch-pads, touchscreens, resistive film (?), integrated with LCD display [Dodge05] Dodge, Steve "Reflowable Ink: Simple Reflow", Microsoft Corporation, MSDN documentation, January 2005, Tablet PC Technical Articles Word-wrapping functionality with Tablet PC for editing ink: compare with van Raamsdonk [Dresevic05a] Dresevic, Bodin "Natural Input on Mobile PC Systems: Full client Usability for the Highly mobile Market", Powerpoint presentation at WinHEC 2005 on Tablet PC "sneak peek" Microsoft WinHEC presentation on Tablet PC additions in Windows Vista/Longhorn: flick gestures, visual feedback on system gestures/shell gestures, change in checkbox behavior for stylus/gesture input. Short mention of n-trig touch+stylus electrostatic digitizer, but says "for best recognition accuracy use a highresolution EM (electromagnetic) digitizer" with coil in pen. Finger-Touch input. Discussion of pen-enabled and voice-enabled system for Tablet PC. [E-Ink05a] E-Ink Corporation "World's First Electronic Paper Watch Demonstrated", Press Release: www.eink.com



E-Ink electronic ink product: digital wristwatch with Seiko Watch Corporation [EatonJ05a] Eaton, John and Moog, Robert "Multiple-Touch-Sensitive Keyboard", Proc. 2005 Int'l. Conf. on New Interfaces for Musical Expression (NIME05), Vancouver, BC, Canada, May 26-28, 2005, pp. 258-259 Single touchpad sensor on each key of music keyboard, sensing X/Y location, area of contact (multi-touch? ), depth of key travel, and force when at limit of travel. Additional multi-touch control touchsurface for global parameters. [EsentherA05a] Esenther, Alan and Wittenberg, Kent "Multi-User Multi-Touch Games on DiamondTouch with the DTFlash Toolkit", Mitsubishi Electric Research Laboratories report MERL TR2005-105, December 2005 Prototype multi-touch, multi-user games on DiamondTouch touchscreen surface. Documents/Windows may be flicked, rotated to varying orientations: two finger (one finger per user) rotation gesture, one finger/user gives center of rotation. Multi-finger input by one user for scraping gestures. DiamondTouch SDK based on Flash (cross-platform). [EslambolchilarP04a] Eslambolchilar, Parisa and Murray-Smith, Roderick "Tilt-Based Automatic Zooming and Scaling in Mobile Devices: A State-Space Implementation", Proc. 6th Int'l Symp. on Mobile Human-Computer Interactions -- MobileHCI 2004, September 13-16, 2004, Glasgow, pp. 120-131 Human-factors study: PocketPC and acceleromiter to use tilt gestures to move zoom window [EverNote04] EverNote.com "EverNote - ritePen product information", www.evernote.com Successor to Paragraph handwriting character recognition for Pen-Windows: Full-screen writing, floating toolbar, editing gestures for capitalization, any Windows application. Refers to forms recognition (with predefined and known fields?) rather than handwriting recognition. ritePen and riteScript handwriting recognition technology for unconstrained handwriting recognition riteForm forms input with contextsensitive recognition with templates and dictionaries : compare with field types and subsets in PenCept, CIC, GridPAD. gestures for text editing, word menu, pop-up punctuation pane, cut/paste, on-line dictionary on-line demo of gesture recognition using mouse / mouse gestures: JavaScript? Java? OCX? [EveryMac04] EveryMac.com "Newton MessagePad Series", www.everymac.com Technical specifications on Apple Newton MessagePad 130: textured display to give a more paper-like writing surface Newton OS attempted to recognize natural handwriting. Artificial intelligence to tie / link relevant information together [EverythingPen05] EverythingPen.com "Service Companies", www.everythingpen.com List of service companies working with Anoto technology: includes www.digipen.de, www.changingtides.co.za, NetPen UK netb2b2.com, www.penvision.de [FASgroup05] FAS Group "Forms Automation System", www.fasgroup.net Consortium of companies developing applications using "smart paper" or "digital paper" (see Anoto), no separate tablet, data capture occurs in real time on patterned paper Datasheet, Forms Automation System 1.2, Digital Pen and Paper [FabritiusH05a] Fabritius, Henna "Changing an Orientation of a User Interface via a Course of Motion", United States Patent Application 2005/0114788, May 26, 2005 Diagonal stroke gesture rotates display 180%, stroke along edged of display rotates 90% in the direction of the stroke. Compare to slide-to-unlock Later US patent 7,814,419 [FeichtingerA04a] Feichtinger, A. "Dokumentation/Beschreibung Usability Tests Paper Prototyping", Fachhochschule Joanneum Informationsmanagement, d.4. Dezember 2004 Outline of project for paper prototyping of interactive mapping user interface. Paper user interface designs for touchscreen PDA, Wizard of Oz simulation of computer interaction. Has additional references. [Fingerworks05a] Fingerworks Inc. "Fingerworks multi-touch keyboard, multi-touch touchscreen, MyGesture Editor", www.fingerworks.com File of materials from Fingerworks web site before it went off-line, 2002-2005. Documentation on types of gestures, gesture editor, hardware products. Multi-touch Touchpad digitizer tablet with built-in gesture recognition. Fingerworks includes: Mouse Emulation - Gesture Guide; Typing With Gestures on the TouchStream ST _ ExtremeTech; MyGesture Editor; XWinder -- The Future of Window Management; XWinder for Mac OS X and Windows XP; RightHand_GestureMap; TouchStream_QuickGuides; TouchStream Booklet; iGesture withKeys Booklet; MyGesture Editor; The Only Touchpads with Ergonomic Full-Hand Resting and Relaxation; Frequently Asked Questions; TouchStreamLPQWERTYSupplement; TouchStreamLPQWERTYManual; Tips_Tricks; iGesture_QuickGuides; iGesturePad_Installation_Operation; iGesturewithKeysBooklet; TouchStreamBooklet; ThirdParty Cypress_sensor_an2233a, TouchStreamLongCable Yechiam_Erev_Yehene_Gopher, multi-touch-CHI-DC, multicursors-gi2006, JTSdefense1; Advantages of MultiTouch Technology -- The First Seamless Integration of Pointing, Typing, and Hand Gestures [Fingerworks05b] Fingerworks Inc. "Fingerworks Summary as of April 26, 2005, Updated September 2, 2005", fingerfans.dreamhosters.com User commentary on Fingerworks ceasing operations: purchase by Apple, information on gesture-based



tablet pc from Apple, Apple files for patent to Elias. [FleckDC04a] Fleck, David C. "Digitizer system with cursor shape changing as function of pointer location on a menu strip and/or physical structure on tablet surface indicating menu strip location", United States Patent 6,689,965, February 10, 2004 Change cursor to indicate where a stylus is in "hover" over a fixed menu on a hardware tablet surface (versus displayed menu) [FleischmannM04a] Fleischmann, Monika; Straus, Wolfgang; Li, Yinlin; Groenegress, Christoph "Gesture-based input device for a user interface of a computer", European Patent EP1457863, September 15, 2004 Hardware patent (but is labeled "Software Patent") on electrostatic/capacitive non-contact digitizer, similar in principle to a theremin, to sense in-air gestures. Hardware for iPoint device from Fraunhofer Institut? [ForlinesC05a] Forlines, Clifton; Shen, Chia; and Buston, Bill "GLIMPSE: A novel input model for multi-level devices", Mitsubishi Electric Research Laboratories Report TR2005-024, May, 2005 Use three-level force/pressure sensor to "preview" the effects of a change, as better alternative to Undo operation in a user interface. Similar to bounce at end of scroll? [Fraunhofer04a] Fraunhofer-Institut fürRechnerarchitektur und Softwaretechnik "Fraunhofer-Institut: Touchglobe 3-D-Eingabegerät auf Polymer-Basis", Fraunhofer-Institut, Berlin Touchglobe: multi-touch input device using sensor skin of piezo-electric polymers: pressure-sensitive and multi-touch capable. [Fraunhofer05] Fraunhofer-Gesellschaft "Projekte und Themen - GestureID", www.melodiesuche.de/de/projekte_themen/gestureid.htm In-air hand gestures based on visual processing to remove a known background and detect handgesture visually. [Fujitsu05a] CADigital.com "Fujitsu Stylistic 1000 Pen base computing", www.cadigital.com/1000.htm Technical information and photographs of Fujitsu Stylistic 1000, Windows for Pen Services 1.0, Windows for Pen Services 2.0 (a.k.a. PenWindows) [GTCOCalcomp04a] GTCO/Calcomp "WinTab no longer supports scaling and rotation", www.gtco.com, October 8 2004 Scaling and rotation transformations removed from industry-standard Wintab devices drivers. Resolution part of patent lawsuit between Schumer and LCS/pointing.com. [GeaghanB05a] Geaghan, Bernard O. and Robrecht, Miachel J. "Light Emitting Stylus and User Input Device Using Same", United States Patent Application 2005/0110781 A1, May 26, 2005 Optical digitizer, stylus emits collimated light (e.g. laster pointer), optical photo-detectors detect beam. Interpolation to higher resolution when multiple sensors illuminated. Determine angle (and height?) of stylus by cross-section of light cast on touchscreen, infer hand position. Same photo array can detect shadows of hand, etc. Information (switches, buttons, pressure/force) can be encoded by modulating light beam. Stylus as touchscreen input. [GibetS05a] Gibet, Sylvie; Courty, Nicolas; Kamp, Jean-Francois eds. "Gesture in Human Computer Interaction and Simulation", Published proc. of Gesture Workshop 2005 Hand and body gestures, including music user interface for performance [GibsonR04] Gibson, Rich "Introduction to the Gumstix tiny linux computer", www.oreilly.com Tiny computer with Linux and XScale processor, on a USB drive. Can be used to re-boot Windows computers? Similar concept to "Stunt Box" of approximately 1977. [Go05a] Fried, Ina "Go files antitrust suit against Microsoft", Cnet news.com.com, July 1, 2005 Jerry Kaplan, US District Court, Anti-trust suit vs. Microsoft [GocinskiF04a] Gocinski, Frank "Tablet 101 Column 3: Strokes and Recognition", Microsoft Corporation MSDN documentation, December 2004, TabletPC Edition Development Kit 1.7 Term: passive/resistive digitizer versus active/electromagnetic digitizer: 40 points/sec versus 100 points sec, .25 mm versus better "Building ink chat" instant messaging using electronic ink: compare with Telautograph Specific reference to arrow-left, circle, and check gestures [GoertzM04a] Goertz, Magus "User Interface", United States Patent Application 2004/0109013 A1, June 10, 2004 PDA touchscreen user interface: separate menu and display areas, slide/swipe motions (not to scale) in menu area as gestures to select functions in lists, etc. Compare with slide to unlock? [GoldbergA04] Goldberg, Arin "Developing Tablet PC Software by Using the Windows XP Tablet PC Edition 2005 Recognition Pack", Microsoft MSDN, October 2004 Microsoft documentation, describes how to install Tablet PC SDK 1.7 recognizers on non-Tablet PC versions of Windows for development purposes only, external/separate tablet is still not supported. Tablet data rate is much higher than mouse, mouse not for recognition accuracy. [Golem05] Golem.de "Wacom stellt elektronischen Stift für PDAs vor", www.golem.de/0507/39388.html Wacom introduces electronic stylus for PDAs. Super Slim Pen from Wacom, intended for PDA market. Mentions screen / electromagnetic digitizer of 4.3 inches size or smaller. Mentions use of proximity sense,



rear-mounting of digitizer to display, asserts better visual clarity and physical robustness (no scratches?) [GorbunovA05a] Gorbunov, Artem and Voronezh, Roma "Infoskroller, or Scroll bar in the submission of content in the form of graphic symbols", artgorbunov.ru/bb/ul/infoscroller, fetched 2008 Scrollbar with small images in the scroll bar for what you are scrolling through: eg. scrollbar has thumbnails of document pages. Russian patent. Includes video demonstration. [GouzmanR04a] Gouzman, Roman and Karasin, Igor "Tactile interface system for electronic data display system", United States Patent 6,762,749, July 13, 2004 Mouse with tactile feedback, multiple braille-like pins under fingers. One is fiduciary point indicating position of cursor on display, others display surrounding parts of image. Pins can be partially activated for gray scale. [Grid05] GSCS Inc., formerly GRiD "GRiDCASE Data Sheets and Product Information", www.pd.com, www.grid.com Technical specifications on 2005 models of GRiD Computers: ruggedized portables, pen input optional [Groklaw05] Groklaw "A New Antitrust Lawsuit - GO Corp. v. Microsoft", www.groklaw.com, July 4, 2005 News article on filing by Jerold Kaplan, former GO CEO, against Microsoft and certain vendors: numerous reader comments Photocopy of NDA between GO Corporation and Microsoft, Signed by Bill Gates 7/8/88. See also "GO/Microsoft" file. [GrossmanT04a] Grossman, Tovi; Wigdor, Daniel; and Balakrishnan, Ravin "Multi-Finger Gestural Interaction with 3D Volumetric Displays", Proc. UIST '04, October 24-27, 2004, Santa Fe NM, pp. Rotating volumetric display inside a clear globe: multi-finger/multi-point input on surface using commercial (www.vicon.com) video motion tracking system and colored dots on fingers.. Tap and tap-and-hold (pressand-hold) gestures. Mentions resistive touchscreen surface could also be used, but would not permit tracking of multiple fingers. Menus displayed dynamically at correct position and orientation (rotated) for user: in prototype, menus physically taped onto display surface. Three-dimensional 3D ray cursors: projected line from 3D pointing finger into 3D display. [GuhaA05] Guha, Angshuman "Feature Extraction for Real-Time Pattern Recognition Using Single Curve per Pattern Analysis", United States Patent 6,898,315, May 24, 2005 Feature extraction in handwriting/gesture recognition, Chebyshev polynomials, also aspect ratio and center of gravity. Mentions using tip force (a.k.a. pressure) from stylus as a Z value in gesture/handwriting recognition. [HakenL04a] Haken, Lippold "Continuous Music Keyboard", United States Patent 6,703,552, March 9, 2004 (published 2003) Multi-touch musical keyboard surface, up to 10 fingers. Hall effect sensor and magnet detect deflection of key rod for Z/pressure, resistance sensing for Y along rod, X for which rod. To detect sliding of finger across "keys", computes X/Y/Z peak using surrounding sensors, effectively giving a continuos-valued input in X/Y/Z. [HammondT04a] Hammond, Tracy and Davis, Randall "SHADY: A Shape Description Debugger for Use in Sketch Recognition", AAAI 2004 Fall Symposium on Making Pen-Based Interaction Intelligent and Natural, pp.71-77 Debugging techniques for LADDER: sketch recognition language defining sketch recognition constraints. Constraint definitions automatically transformed into shape recognizers, editing recognizers, and shape displayers. Works by user sketching a shape the definition should recognized, and debugger displays what constraint(s) were not matched. [HammondT05a] Hammond, Tracy and Davis, Randall "LADDER, a sketching language for user interface developers", Computers and Graphics, Vol 29, pp. 518-532. Elsevier.com, 2005 LADDER: sketch recognition language defining sketch recognition constraints. Sketch recognition techniques for multiple problem domains: class diagrams, GUI design, web page design, stick figures, etc. Sketch interfaces becoming common as pen-based input systems become common. [HanJ05a] Han, Jeff "Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection", Proceedings of 18th Annual ACM Symposium on User Interface Software and Techology Fingertip Touching on internal illuminated glass causes partial internal reflection, detected by optical sensors. Cited elsewhere (iPhone) as multi-touch reference. Mentions "Minority Report" gesture user interface in three dimensions, bi-manual two hands. See also Fingerworks reference, FTIR (Frustrated Total internal Reflection) Paper files mention LED Touch display, touching on a grid of LED diodes, the LEDs also act as photodiodes and detect contact reflection, as digitizer / touchscreen [Handtops04] Handtops.Com Handtop Computers "OQO's Wacom Penabled screen", www.handtops.com, September 28, 2004 OQO using Wacom electromagnetic digitizer: refers to Sony U50/ U70 using a "passive" digitizer, as contrasted to a Wacom "active" digitizer -- reference to proximity sensing, higher data rate, and higher resolution than a PDA touchscreen at 40 points/sec instead of 130 points/sec [HaskellB04a] Haskell, Bert "Portable Electronics Product Design and Development", McGraw-Hill publishers, 2004



(hardcopy book) Palm-top PID and other devices Fig. 1.11, Contains references to 1991 Kyocera Refalo and 1987 Casio IF8000, Fig 10.1, IBM Simon 1994. 1997 Nokia 9000 Communicator. HP Journada Smart Phone / PDA with touch screen input. Section 4.5 pen input / stylus input via ITO resistive film sheets. Display may be integrated by making top surface of LCD glass be lower later of two ITO sheets. [Hawkins05] Hawkins, Jeffrey C.; Rees, William B.; Chyi, Debbie O.; Haitani, Robert Y. "Interface for Processing of an Alternate Symbol in a Computer Device", United States Patent 6,975,304, December 13, 2005 User interface pop-up display of alternate characters for a main character: e.g. Accented versions of "E" for the letter "E", alternate currency symbols (Pound, Yen) for currency symbol. Mainly to reduce size of keyboard or simulated keyboard in a palm/PDA device, or to simplify handwriting recognition. Has extensive bibliography, numerous references. [HechtDL05a] Hecht, David L.; Flores, L. Noah; and Gorbet, Matthew G. "Operations on Images having Glyph Carpets", United States Patent 6,935,562, August 30, 2005 Xerox dataglyphs dot pattern for encoding information in pictures and printed documents. Encoded information can be can be for tactile, audible, visual or other information for user. Compare with Anoto optical pattern digitizer pen. [Heisei05] Heisei Electronics Col. Ltd. "Come-Book Palmtop/Handheld PC", www.computex.biz/heisei small Tablet-PC like product, resistive touch-screen digitizer, OS is Windows CE / Windows Mobile (?), might be considered a large-format Windows Mobile / Pocket PC device [Heller05] Heller, Martin "Programming for the Tablet PC", byte.com, January 10, 2005 Mentions addition of dictionary/context/linguistic information to specific dialog windows of non-Pen application using Context Tagging tool: compare with JAWS scripts for accessibility aids for screen readers for the blind [HinckleyK04a] Hinckley, Ken; Ramos, Gonzalo; Guimbretiere, Frencois; Baudisch, Patrick; Smith, Marc "Stitching: pen gestures that span multiple displays", Proceedings of AVI '04, working conference on Advanced Visual Interfaces interpolate drag gesture from one tablet/screen to a second one; opening document in one tablet/screen that are selected in another [HinckleyK04b] Hinckley, Kenneth P. and Bathiche, Steven N. "Method and system for accelerated data navigation", United States Patent 6,738,045, May 18, 2004 Accelerate scrolling based on user input using look-up acceleration table, scrolling inhibited if user scrolls in opposite direction (to fix overshoot). [HinckleyK05a] Hinckley, Ken; Baudisch, Patrick; Ramos, Gonzalo; Guimbretiere, Frencois "Design and Analysis of Delimiters for Selection-Action Pen Gesture Phrases in Scriboli", Proceedings of CHI 2005, April 2-7, Portland, Oregon, pp. 451..460 Selection actions with pen gestures: lasso combined with pig-tail gesture, timeout without motion and without lifting pen resulting in pie/flick gesture menu, clicking on separate button. [HolmanD05A] Holman, David; Vertegall, Roel; Altosaar, Mark; Troje, Nikolaus; and Johns, Derek "PaperWindows: Interaction Techniques for Digital Paper", Proc. CHI 2005, pp 591-599 Optical/visual motion capturing to recognize gestures on paper ("augmented paper"), such as rubbing to transfer image from paper to electronic touchscreen, two-handed pointing, single-hand pointing. [HopkinsM05] Hopinks, Mark "Mobile Ink Jots 6: Using Gestures in Tablet PC Applications", msdn.microsoft.com Tablet PC SystemGestures: gestures DoubleTap, RightTap, Tap, Drag, RightDrag, HoldEnter, HoldLeave, HoverEnter, etc. gestures. Defines "gesture" as an ink stroke or pen movement that matches the set of glyphs defined by a recognizer. See also "Integrating Application Gestures": "a glyph defines the shape traced by the gesture". [HortonTE04a] Horton, Thomas E. "HabilisDraw: A Tool-based Direct Manipulation Software Environment", Master's Thesis, Dept. of Comp. Sci, NCSU, Raleigh NC, 2004 Direct-manipulation on touchscreen using graphical tools: compass, straight edge (to align objects), inkwells (select drawing color), straight pins (to fix position of an object), glue bottles (to group objects). Graphical tools can be combined/composited: e.g. compass and stretch ruler to draw a spiral. Second prototype implemented on DiamondTouch multi-touch tabletop. [HoyleMD04a] Hoyle, Martin David "Computer interface method and apparatus with portable network organization system and targeted advertising", United States Patent 6,771,290, August 3, 2004 Automatically-upgrading application over internet, showing advertisements. CIP of 6,141,010 [Huang04] Huang, Gregory T. "Microsoft's Magic Pen", MIT Technology Review magazine, May 2004 Article on a optical digitizer technology at Microsoft's research lab in China, using a printed pattern on paper for regular documents Compare with Anoto, Sekendur, and Silverbrook [HudsonSE04a] Hudson, Scott E. "Using Light Emitting Diode Arrays as Touch-Sensitive Input and Output Devices", Proc. UIST '04, October 24-27, 2004, Santa Fe, USA pp 287-290



Array of bidirectional LEDs used as photo sensors as well as display light sources: one LED used as light source, adjacent LED used as sensor for light passing through partially-translucent human finger. Includes optical (display) press feedback. [Hullender05a] Hullender, Gregory and Gounares, Alexander "Ink Gestures", United States Patent 6,938,222, August 30, 2005 Ink gestures recognized using Chebychev Polynomials and Bayes net Refers to an ink gesture being included in a stroke: gesture is part of a larger stroke? In description, lists example gestures and their semantics: e.g. braces (discontinuous selection), circle-tap, , flick (scroll), double-right-flick for scroll to end, double-left-flick, triple-up-flick, etc. Appears to be same basic set of gestures as Microsoft "Unimplemented Glyphs". Compare with GO/PenPoint flick and other gestures [Hyperlabel05a] Hyperlabel.com "Hyperlabel product information", www.hyperlabel.com hidden optical printed pattern on labels allows for encoding of digital signatures, for tracking of product through the distribution chain. Does not refer to tracking the provenance of manufacturing, or to security documents. Mentions encoding of web hyperlinks. Silverbrook product announcement -- Australian patent house. Compare with Anoto, and with Digimark filing "Protection of Identification Documents using Open Cryptography" [Hyperlabel05b] Hyperlabel.com "Introduction to Hyperlabel Technologies", www.hyperlabel.com Application notes on Hyperlabel optical printed pattern. Tracking documents, product codes (vs RFID), batch numbers, etc. Compare with Anoto, and with Digimark filing "Protection of Identification Documents using Open Cryptography" WardJR04 [IPen4You05] IPen4You "i-Pen - Presentation Digital Pen / Optical Pen Mouse", www.ipen4you.com Optical digitizer pen, similar to Anoto / Microsoft? Talks about writing on any surface, also writing on any screen. Include RitePen handwriting recognition software, annotation applications [IRISPen04a] Data ID Online "IRISPen: Text recognition where you want it, when you want it", www.dataid.com/irispen.htm, 2004 Product information on on IRISPen: hand-held optical scanner with OCR for portable document scanning. OCR software includes capturing formatting information e.g. tables. [IRex05] IRex Technologies "Electronic Reader ER 0100 "iliad"", www.irextechnologies.com EInk display electronic book: 1024x768 display resolution. Includes stylus for adding electronic ink text and scribbling, OS not certain, not a Pen Computer: USB port, support PDF XHTML Mp3 other formats. Internal storage is 256 MB Flash. See also www.eink.com, electrophoretic display technology [IgarashiT05a] Igarashi, Takeo; Moscovich, Tomer; Hughes, John F. "As-rigid-as-possible shape manipulation", ACM Trans. Graphics, Vol 24, No 3, July 2005, pp. 1134-1141 shape editing/ direct-manipulation deformation user interface using triangle mesh for flexible stretch/compress of images. Uses multi-touch (more than two) finger touches and fits deformation to however many points are used. Clay-like editing of simulated 2D and 3D objects. Cites to Rekimoto for multi-touch touchscreen digitizer. [IgarashiT05b] Igarashi, Takeo; Moscovich, Tomer; Hughes, John F. "As-rigid-as-possible shape manipulation", www-ui.is.s.u-tokyo.ac.jp/~takeo/research/rigid, Copyright 2005. Video demonstration of as-rigid-as-possible shape manipulation, shows multi-touch direct manipulation of 2D two-dimensional cartoon figures. [IgnatCL04a] Ignat, Claudia-Lavinia and Norrie, Moira C. "Grouping in Collaborative Graphical Editors", Proc. CSCW '04, November 6-10, 2004, Chigaco Collaborative group graphical editing (shapes colors). Conflicting user actions on locally-cached copies of object (no central copy?) resolved by simple serialization and some ordering rules, suggest locking of object / sub-objects also needed. [InksterRD04a] Inkster, Robert D. "Touch pad using a non-electrical deformable pressure sensor", United States Patent 6,788,295, September 7, 2004 Pressure touchpad using optical pressure sensors. Can detect multi-touch input by independent sensing of multiple indentors (buttons). [Inovatech05] Inovatech "Toshiba T200CS als Webpad einsetzen", http://T200.html Technical information, software, support on converting Toshiba T200CS Dynapad pen computer (early 90s) as a web-pad unit for internet access with a stylus computer Refers to PenPad software from Toshiba: same name used for hardware handwriting recognition product by Pencept in 1980's Links to Dynapaedia for same topic: Dynapaedia documentation on Wacom 500 tablet in Toshiba T200 [Interlink04a] Interlink Electronics "ePad Electronic Signature Tablets", Product listings, steadlands.com, 2004 ITO resistive touchscreen for point-of-sale POS signature capture. Claims that signatures can be bound to documents, signature verified/authenticated later. [IoannidisJPA05a] Ioannidis, John P.A. "Why Most Published Research Findings Are False", PLoS Medicine, Vol 2 No 8, e 124, pp. 0696-0701



Bias in research results: small sample size, weak correlation, analyis defined after-the-fact, also financial interest. See also "Unskilled and unaware of it", NagyG83b, Gelman 1994. [IwamotoT04a] Iwamoto, Takayuki; Akaho, Daisuke; and Shinoda, Hiroyuki "High resolution tactile display using acoustic radiation pressure", Proc. SICE Annual Conf., Hokkaido Ins. of Tech., August 4-6, 2004, pp. 1239-1244 Proposal and design report for ultrasound haptic/tactile display using acoustic radiation pressure in water (not in air). X and Y orientation of two linear arrays ultrasonic transducers. [Jaimes05] Jaimes, Alejandro and Sebe, Nico "Multimodal Human Computer Interaction: A Survey", IEEE International Workshop on Human Computer Interaction, in conjunction with ICCV 2005, Beijing China October 2005: same title and authors Computer Vision and Understanding, Vol 108, pp. 116-134 (2007) handwriting recognition, gesture, facial gesture recognition, voice input [JazzMutant05a] JazzMutant.com "The Lemur Owner's Manual", www.jazzmutant.com Multi-touch controller touchscreen interface, used for multi-touch control of music mixing. Electronic file contains additional materials, including product announcements from 2004. [Kairer05] Kairer, Ryan "palmOne LifeDrive Review", www.palminfocenter.com, May 22nd 2005 Review of Palm LifeDrive / Palm Pilot PDA: same digitizer as Palm T3 [KaltenbrunnerM05a] Kaltenbrunner, Martin; Bovermann, Till; Bencina, Ross; and Costanza, Enrico "TUIO - A Protocol for Table-Top Tangible User Interfaces", Proc. GW 2005, 6th Int'l Workshop on Gesture in HumanComputer Interaction and Simulation, 2005, Vannes, France Standardize API/support for multi-touch touchscreen and tabletops. From reacTable. Includes blobs, blob tracking, direct manipulation of multiple objects, gestures, tracking of tagged object, recognition of object geometry on surface. Capacitive, optical/camera touchscreens. Electronic file contains additional information from TUIO project website. [KaplanSJ05a] Kaplan, S. Jerrold "Legal Complaint: Kaplan v. Microsoft lawsuit", //docs.justia.com/cases/federal/district-courts/california/candce/5:2007cv01389/189975/889/1.pdf - fetched 2008 Publicly available on multiple web sites: Filings in GO/Microsoft lawsuit under Cartwright Act in California, concerning PenPoint and Pen Windows pen computers. [KarlsonAK05a] Karlson, Amy K. and Bederson, Benjamin B. "AppLens and LaunchTile: Two Designs for OneHanded Thumb Use on Small Devices", Proc. CHI 2005, April 2-7, Portland, Oregon pp. 201-210 User-interface for one-handed touchscreen PDA (Palm Pilot): Informational fish-eye magnified view of data when thumb is over UI element. Notification zoom, context zoom, Full zoom levels of information at appropriate detail. Applications represented by tiles, tiles group into zones of a worldspace. User can scroll between zones, but UI snaps to zone when released. [KayeS05] Kaye, Stephen T.; Timmons, David Eric; Eckert, Brad; Doubrava, Dana; and Jacobs, Wesley "Collaborative Input System", United States Patent 6,930,673, August 16, 2005 Multiple-user whiteboard system. Using multiple untethered/wireless digitizer tablets to allow several users to annotate a drawing or document. Primarily discusses hardware. Schumer patent cited as prior art. See also eInstruction file. [KayeS05b] Kaye, Stephen T. "A Brief History of the Wireless Tablet in Classroom Instruction", Product whitepaper: www.einstruction.com Multiple-user whiteboard system using untethered electromagnetic digitizers and a central projection display. Using multiple untethered/wireless digitizer tablets to allow several users to annotate a drawing or document. Includes educational software for presenting images and annotation: can sub-divide display area into multiple virtual tablet displays for multiple student's work. Refers to dual-input digitizer whiteboard: wireless and USB/wired? [KeelyB05] Keely, Bert "Full Client Usability in the Highly Mobile Market", Powerpoint presentation at WinHEC 2005 on Tablet PC "sneak peek" Microsoft WinHEC presentation on Tablet PC additions in Windows Vista/Longhorn. Mentions N-trig transparent digitizer as thinner/lighter than electromagnetic (Wacom). Microsoft patent on "direct hinge" alternative to clamshell, license included with Windows OEM agreement. Also mentions spatial audio on microphone array with Tablet PC. Changes to display to reduce parallax by thinner protective layer. [KeelyL04] Keely, Leroy; Jones, David F; Switzer, David; Tsang, Michael Hin-cheung; and Vong, William Hong "Simulating Gestures of a Pointing Device using a Stylus and Providing Feedback Thereto", United States Patent 6,791,536, September 14, 2004 Tap, Press, press-and-hold with stylus for left-click, right-click, and left-mouse-button drag gestures. Refers to mouse click as a gesture [KeelyL05] Keely, Leroy; Lui, Charlton E; Jones, F. David; Cukierman, Ryan Edward; Cazzanti, Susanne Alysisa Clark; Iwema, Marieke; and Jarrett, Robert "High Level Active Pen Matrix", United States Patent 6,897,853, May 24, 2005 Tap, Press, press-and-hold with stylus for left-click, right-click, and left-mouse-button drag gestures. Refers to mouse click as a gesture



[KeelyL05b] Keely, Leroy B.; Iwema, Marieke; and Cazzanti, Susanne Alysisa Clark "Selection Handles in Editing Electronic Documents", United States Patent 6,891,551, May 10, 2005 Text-editing user-interface with a stylus: when text is selected, also show manipulation handles for dragging/moving/resizing text, handles are easier to target. Handle for rotating also. [KellogW05] Kellog, Wendy "A Conversation with Ray Ozzie: Cooperate, Communicate, Collaborate", ACM QUEUE, November 2005, pp 18 ff. Inventor of Lotus notes, currently CTO of Microsoft. "Cornucopia of the commons" attributed to Dan Bricklin, benefits of open-source collaboration and development. [KhellaA04a] Khella, Amir and Bederson, Benjamin B. "Pocket PhotoMesa: A Zoomable Image Browser for PDAs", Proc. MUM 2004, Oct. 27-29, 2004, College Park, Md, pp. 19-24 Browse photos not by uniform thumbnails, but by (smoothly/animated) zoomable display (semantic zoom) of photos at different scales. Stylus touchscreen interface more limited than mouse because of lack of additional buttons, and of scroll wheel. (Does not consider multi-touch, even for side-tap of 2nd finger) [KinrotU04a] Kinrot, Uri and Kinrot, Opher "Edge Detector", United States Patent 6,737,664, May 18, 2004 detect that an edge of an object has entered a field of view by detecting motion indication via OTM, speckle pattern of reflected laser light. [Klaehn05] Klaehn, Martin and Schulthes, Stefan "OCR Optical Character Recognition", Semesterbericht im Rahmen der Veranstaltung Künstliche Intelligenz bei Prof. Dr. Eduard Heindl, Fachhochschule FurtWangen, Wintersemester 2004/2005, Germany contains historical notes on optical character recognition, citing back to unidentified US patent from 1800, also to Gismo system by David Shepard in 1951: Gismo could read up to 23 different letters and speak them out loud, as well as recognizing Morse Code. Example experiment of character recognition with neural nets. [Knerr05] Knerr, Stefan "MyScript and Vision Objects handwriting recognition technology", 1eres Recontres XAange Capital, Paris, March 23, 2005 Description of Vision Objects application development for Anoto digital/smart paper: interactive educational input for children using handwriting recognition [KnippingL05a] Knipping, Lars "An Electronic Chalkboard for Classroom and Distance Teaching", Ph.D. Thesis, Free University of Berlin, 14 February 2005 Whiteboard system for distance teaching. Includes use of pie (marking) pop-up menus on whiteboard. Numerous references to uses of pen- and tablet-computers in education for whiteboarding. Further information from www.echalk.de in paper file. See also FriedlandG. [Koblentz05] Koblentz, Evan "The Evolution of the PDA, 1975-1995", Computer Collector Newsletter, www.snarc.net/pda/pda-treatise.htm Side-reference that the history of handwriting recognition is old, and thus a bit unclear, but history of PDA hand-held device is simpler. Historical reference Isaac Asimov "Pocket Terminal" and PocketSec" for pen computing Photograph: 1980 Casio PF-8000 with character recognition Section on Linus Technologies WriteTop with Ralph Sklarew First hand-held pen/stylus computer: Litton Data Systems "handheld terminal unit", approximately 1977 [KrugerR04] Kruger, Russell; Carpendale, Sheelagh; Scott, Stacey D.; and Greenberg, Saul "Roles of Orientation in Tabletop collaboration: Comprehension, coordination and Communication", Computer Supported Cooperative Work, Vol 13, No. 5-6, pp. 501-537, Dec. 2004 (abstract only) In Tabletop UIs for multiple users, orientation with respect to a given user is not best, users orient graphical objects in various ways to share. Observational study. [KrugerR05a] Kruger, Russell; Carpendale, Sheelagh; Scott, Stacey D.; and Tang, Anthony "Fluid Integration of Rotation and Translation", Proc. CHI 2005, April 2-7, Portland OR, pp. 601.-620 single-touchpoint (vs. multi-touch) direct manipulation gesture for rotating and translating: if touch in central region (not exact center), just move object. If outside central region, drag with rotation due to dragging behind. Touch at corner (of rectangular window or document) to rotate only. [KustanowitzJ05a] Kustanowitz, Jack and Shneiderman, Ben "Meaningful Presentations of Photo Libraries: Rationale and Applications of Bi-Level Radial Quantum Layouts", Proc. ACM JCDL '05, June 7-11, 2005, Denver, pp. 188-196 Tree-maps/photo-browsing: For large collections of photos/albums, rather than groups by tags in a simple tree heirarchy, show in a circular layout: main topic in center, sub-topics or related topics (groups of photos) surrounding. [LaViola04] LaViola, Joseph J. Jr. and Zeleznik, Robert C. "MathPad(2): a system for the creation and exploration of mathematical sketches", ACM Trans. Graphics, Vol. 23 No. 3, August 2004; Proceedings of the 2004 SIGGRAPH Convergence, Session: Identifying and sketching the future (abstract only) Just Abstract and references Several citations to recognition/editing of mathematical expressions, but no citations to 1970's work, anything prior to 1990s



[LaViola05] LaViola, Joseph J. Jr. "Mathematical Sketching: A New Approach to Creating and Exploring Dynamic Illustrations", Ph. D. Thesis, Brown University, May 2005 recognition/editing of mathematical expressions: mentions gestures for equation editing [LangdonP04a] Langdon, P.; Keates, S.; Clarkson, P.J.; and Robinson, P. "Using haptic feedback to enhance computer interaction for motion-impaired users", Proc. 3rd Int'l Conf. Disability, Virtual Reality & Assoc. Tech., Alghero, Italy 2000, pp. 25-32 For motion-handicapped users (cerebral palsy), vibrotactile/vibrating haptic feedback not helpful, but forcefeedback feedback is. Feedback prototypes use Immersion Technologies and SensAble Technologies devices. [LapstunP04a] Lapstun, Paul and Silverbrook, Kia "Sensing Device for Coded Electronic Ink Surface", United States Patent 6,724,374, April 20, 2004 Australian development. Pen digitizer, using printed pattern on same surface as electronic ink. Refers to detecting regions via a printed code: compare with dataglyphs. Refers to detecting motion of sensor, rather than position of sensor. Refers to Memjet and Netpage, printing of forms. Bennett and Sekendur patents as prior art. Compare also to "Protection of Identification Documents using Open Cryptography", WardJRW04. [LapstunP04b] Lapstun, Paul; Silverbrook, Kia; and Lapstun, Anne "Method and System for Note Taking using Processing Sensor", United States Patent 6,829,387, December 7, 2004 Australian development. Pen digitizer, using printed pattern on same surface as electronic ink. Refers to detecting regions via a printed code: compare with dataglyphs. Refers to detecting motion of sensor, rather than position of sensor. Refers to Memjet and Netpage, printing of forms. Bennett and Sekendur patents as prior art. Compare also to "Protection of Identification Documents using Open Cryptography", WardJRW04. [LapstunP04c] Lapstun, Paul; Silverbrook, Kia; and Lapstun, Anne "Method and System for Note Taking", United States Patent 6,681,045, January 20, 2004 Take electronic ink notes using Anoto-like paper "digitizer", and index them. Cites to Sekendur, but not to Anoto? [LapstunP05a] Lapstun, Paul and Silverbrook, Kia "Hand-Drawing Capture via Interface Surface Having Coded Marks", United States Patent 6,947,027, September 20, 2005 Optical Digitizer and intelligent paper, combined with encoded information in a particular writing/drawing field. Cites Sekendur95, Bennett91, and Ward96. Uses term netpage. [Leapfrog05] Leapfrog Enterprizes Inc. "Instruction Manual: FLY PenTop computer", www.leapfrog.com Educational toy device: Anoto pen optical paper digitizer tablet, with educational games. One example is drawing a calculator on paper, then using the calculator. Unclear what recognition technologies (if any) used. [LeeJC04a] Lee, Johnny C.; Dietz, Paul H.; Leigh, Darren; Yerazunis, William S.; and Hudson, Scott E. "Haptic Pen: A Tactile Feedback Stylus for Touch Screens", Proc. UIST '04, October 2004, Santa Fe NM, pp. 291-294 Haptic stylus for touch screens, with pressure (tip force) sensor, using solenoid in top of pen stylus, so that it can simulate button-like forces along the axis of the touch stylus. Cites to Anoto as a passive pen touchscreen technology. [Lemelson04] Lemelson, Samuel (subject) "Lemelson's Legacy: Great Inventor or Patent Hoarder?", www.wweek.com, www.cognex.com, www.techdirt.com, other sources Articles on Samuel Lemelson "submarine patents", Cognex lawsuit to invalidate, Lemelson foundation funding projects for small inventors at M.I.T., etc. [Lemelson05] Lemelson, Samuel (subject) "Press materials and reports: Lemelson v. Cognex, etc.", Cognex/Lemelson Press materials on Lemelson/Cognex patent dispute, reports on Lemelson [LeungCK05a] Leung, Christopher K. "Handling Ambiguous User Input on Touchscreen Kiosks", Master's Thesis, MIT EE/CS, June 2005 Touchscreen accuracy limited: use circular targeting (near-miss) instead, or let user select using simple gestures. Compare with ink targetting and gestures in PenPoint and Slate PenApps, or stroke parsing in Pencept. [LiJ05a] Li, Junfeng; Zhang, Xiwen; Ao, Xiang; Dai, Guozhong "Sketch Recognition with Continuous Feedback Based On Incremental Intention Extraction", Proc. IUI '05, January 2005, San Diego, pp. 145-150 Sketch recognition in real time, primarily for arcs and straight lines and connected multi-lines, not domainspecific objects or shapes. [LiY05a] Li, Yang; Hinckley, Ken; Guan, Zhiwei; Landay, James A. "Experimental Analysis of Mode Switching Techniques in Pen-based User Interfaces", Proc. CHI 2005, Paril 2-7, Portland, Oregon, USA, pp. 461-470 Modes for switching between electronic ink input and gestures: did not like pressure mode of digitizer, preferred switching with other (non-preferred) hand. Barrel-button (compare with Pencept recognition



macros: may depend on design of barrel button?), press-and-hold (says this was the worst), eraser end of pen/stylus. Note: press-and-hold pen travel threshold [LinJ05a] Lin, Jaoching "Sensing an Object with a Plurality of Conductors", United States Patent 6,891,531, May 20, 2005 Electrostatic/capacitive grid touch tablet [LinjamaJ04a] Linjama, Jukka and Kaaresoja, Topi "Novel, minimalist haptic gesture interaction for mobile devices", Proc. NoriCHI '04, October 2004, Tampere, Finland, pp. 457-458 Tactile haptic feedback in handheld device using vibrating DC motor with counterweight. Effects include bouncing of ball, centering spring, and damping. Feedback synchronous with graphic and tactile events (tap, kick), combined with 2-D acceleration sensor. Cites to Immersion in cell phones. [Logitech04] Logitech Inc. "Logitech io2 Digital Writing System, Logitech io Digital Pen", www.logitech.com Logitech version of Anoto digital pens: applications include handwritten notes, handwriting recognition. MyScript notes by Vision Objects with shape recognition. Background handwriting recognition for indexing documents. [LohbihlerA04a] Lohbihler, Andrew and Jaeger, Denny "Method and apparatus for powering detecting and locating multiple touch devices on a touch screen", United States Patent Application Publication 2004/0056849, March 25, 2004 Transparent CDMA (capacitive?) digitizer with two styli, styli are powered from an EM standing wave generated in a layer of the tablet: multi-touch [LokhorstDM04a] Lokhorst, David M. and Alexander, Sathya R. "Pressure Sensitive Surfaces", United States Patent 6,715,359, April 6, 2004 Tactex: touch digitizer with pressure sensitive sensors (in a grid), membrane or surface that distributes force across adjacent sensors, interpolate pressure/force image to determine accurate fine position. Protrusions over the sensors under the covering membrane to focus force on sensors. Resistive, optical, piezo force sensors. Membrance can have fine structure (holes, etc.) to focus force. [LongPJG05a] Long, P.J.G. "Introduction to Octave", www.octave.org, September 2005 Octave: MATLAB clone. Graphical plotting and numeric mathematical utility with keyboard input userinterface. Octave introduction based on Tutorial Guide to Matlab by Paul Smith. [LuPorter04] Lu-Porter, Li, and Helfrich, Antje "Handwriting Recognition Improvements in Microsoft Windows XP Tablet PC Edition Service Pack 2", MSDN documentation, Microsoft Corporation, March 2004 Changes to improve handwriting recognition in Tablet PC Service Pack 2: Remove unusual words from lexicon dictionary, improve recognition of umlauts and other diacritical marks, better support for delayed strokes: quotation marks, etc. [Luff04a] Luff, Paul; Heath, Christian; Norrie, Moira; Signer, Beat; and Herdman, Peter "Only touching the surface: creating affinities between digital content and paper", Proc. 2004 ACM conference on Computer Supported Co-operative Work, Chicago Illinois Co-axial pen reads pattern on (enhanced) paper with invisible position/paper-ID pattern in conductive inks, electromagnetic pen/stylus: compare with Anoto optical digitizer paper, Xerox DataGlyphs. Guimbretiere: Paper augmented digital documents. Grasso/Karsenty/Susani: Augmenting paper to enhance community information sharing. [LuiCE04a] Lui, Charlton E. and Blum, Jeffrey R. "Time based hardware button for application launch", United States Patent 6.727,830, April 27, 2004 click, press-and-hold, and double-click on buttons in a PDA. Different functions/applications depending on how long a button it held, or if tapped/clicked multiple times. Buttons may be on a touchscreen. Compare with gesture sensitive buttons? [LuiCE04b] Lui, Charlton E.; Altman, Dan; Smith, Anthony Scott; Tee, Cynthia; and Pichalah, Shenbagalakshmi "System and Method for Scaling and Repositioning Drawings", United States Patent Application 20040205624 Automatic rescaling and repositioning of drawing images when the font size is changed in a document, or text is move. Automatic adjustment if this causes two drawing images to overlap in the laid-out layout page. [Maentyjaervi04] Maentyjaervi, Jani; Kybergh, Kjell; Himberg, Johan; and Hjelt, Kari "Touch Detection System for Mobile Terminals", Proc. MobileHCI 2004, LNCS 3160, pp. 331-336, 2004 Impedance measurements (resistive?) touch-screen touch detection system for mobile phones: Recognizes/detects touch of different objects such as cases and cotton trouser pockets, not just finger touch [MalikS04] Malik, Shahzad and Laszlo, Joe "Visual Touchpad: A Two-handed Gestural Input Device", Proc. 6th International Conference on Multimodel Interfaces, State College, Pennsylvania, 2004, pp. 289-296 Optical touchpad digitizer (intended to simulate a multi-touch touchpad) using optical camera to read hand positions over a desktop surface, and finger orientation/directions. Mentions advantage over actual touchscreens, because screen is not obscured by hand. Cites Enhanced Desk, 4D Touchpad.



[MalikS05a] Malik, Shahzad; Ranja, Abhishek; and Balakrishnan, Ravin "Interacting with large Displays from a Distance with Vision-tracked Multi-Finger Gestural Input", Proc. UIST '05, October 25-37, Seattle, USA pp 43-51 Multi-user multi-hand tracking system: uses optical tracking of hands over black touch surface, able to read all fingers and entire hand, not just points of contact. Gestures by hand/fist posture: e.g. pointing, double pointing, open hand. Compare with "Put that there" [MarkoffJ04] Markoff, John "Newly Released Documents Shed Light on Microsoft Tactics", New York Times, March 24, 2004 Minnesota antitrust lawsuit v. Microsoft. Jerry Kaplan of GO/PenPoint, also cites Marlin Eller book "Barbarians led by Gates" on PenWindows was intended to stop GO from doing business with Intel. [MathewsJE05] Mathews, James E., Thacker, Charles Patrick; and Tsang, Michael Hin-cheung "In-air Gestures for Electromagnetic Coordinate Digitizers", United States Patent 6,903,730 B2, June 7, 2005. Assigned to Microsoft Corporation In-air gesture (a.k.a. "shake"), stylus must stop moving to indicate completion. Does not mention Polhemus. Spike motion is basis for gestures. Proximity position sensing. Does not refer back to Windows for Pen Computing, which may have had some of the same gestures. [MatsushitaN05a] Matsushita, Nobuyui; Ayatsuka, Yuji; and Rekimoto, Junichi "Apparatus and method for manipulating a touch-sensitive display panel", United States Patent 6,958,749, October 25, 2005 Algorithm for recognizing a two-finger gesture on a multi-touch digitizer, where one finger is fixed and the other is moving. Example is rotating an object. The digitizer technology cited is resistive sheet, recognize that there are two fingers from reduced resistance between the two sheets. Rekimoto co-inventor, filing date in 2000. [McVey05] McVey, J. "Telautography", www.jmcvey.net/cable/elements/telautograph1.htm History of the Telautograph, precursor both to the Fax/facsimile machine, and to digitizing tablets. First invention by Elisha Gray 1981, other early patents by George S. Tiffany and Foster Ritchie. See also "Dead Media Project" [Microsoft04a] Microsoft Corporation "Windows XP Tablet PC Edition: Tablet PC: An Overview", msdn.microsoft.com, August 25, 2004 Electronic ink / gesture integration with Microsoft Office: specifically states that Tablet PCs use an electromagnetic digitizer (with proximity) and not a resistive-touch screen: mentions palm-rest problem of resistive touch screens [Microsoft04b] Microsoft Corporation "Issues that may occur with mobile devices that use Outlook Mobile Access and the Windows Mobile operating system or the Pocket PC 2003 operating system", msdn.microsoft.com, KB871185, 7/21/2004 Failure to support extended (non-ASCII7) characters in Pocket PC OS: explicit reference to Windows Mobile Operating System, Pocket PC operating system. [Microsoft04c] Microsoft Corporation "WISPTIS.EXE running on Terminal Server", Google Groups: microsoft.public.windows.terminal_services, June 28, 2004 WISPTIS.EXE Windows Ink Services for Pen, Tablet Input Subsystem: installed automatically by Adobe Acrobat 6 and other application even if not originally present on Windows XP Operating System, not just Tablet PC. [Microsoft04d] Microsoft Corporation "Deploying Microsoft Windows XP TabletPC Edition 2005", www.microsoft.com/technet: updated August 5, 2004 Automated installation scripting for Tablet PC: notes that Tablet PC must have additional hardware buttons for changing orientation of the display, CTL-ALT-DEL reboot. TabTip.exe, tpa.exe, wisptis.exe services. Linearity Data in system registry for correction of position accuracy errors in the particular digitizer tablet. [Microsoft04e] Microsoft Corporation "Anatomy of a digitizer: Tablet 101 Column 3: Strokes and Recognition", msdn.microsoft.com/technet: originally fetched 2008 Shows Microsoft architecture of "active" digitizer: electromagnetic coil digitizer rear-mounted to display, with shielding layer (ground plane), active (electromagnetic) digitizer required to qualify as a Tablet PC. Software notes on InkData, InkCollector objects. [Microsoft05a] Microsoft Corporation "Die Geschichte des Pen Computing", www.microsoft.com Microsoft press note on the history of pen computing: factual errors include statement of RAND Tablet Grafacon as the first digitizing tablet. Mentions Hewlette Crane at Stanford Research Institute with 1964 patent, but not earlier work. Also omits later references to CIC, only to SRI. Alan Kay Dynabook: does not specifically repeat assertion that the Dynabook "vision" included handwriting recognition or pen/tablet computing [Microsoft05b] Microsoft Corporation "Microsoft Tablet PC - Ink Data", msdn.microsoft.com Defines stroke as pen-down/pen-move/pen-up. Stroke data consists of packets of X/Y co-ordinates and other PacketDescription data: pressure (tip-force), angle, etc. "Cusps" changed by Bezier processing (filtering/pre-processing) into loops. Describes cusps as parse points for erasing electronic ink, without



regard to handwriting recognition. [Microsoft05c] Microsoft Corporation "Microsoft Knowledge Base: Drag-and-Drop Editing not Functional with Pen Windows", msdn.microsoft.com, 5/20/2005, KB110398 Notes conflict between behaviors of mouse GUI and tablet PC GUI (mostly likely PressHold gesture?) in Windows for Pen Computing 1.0 [Microsoft05d] Microsoft Corporation "Platform Builder for Microsoft Windows CE 5.0: Debugging Resources Overview", msdn.microsoft.com Debugging/development tools for Windows Mobile 5.0: reference to Windows Mobile OS, Windows Mobile 5.0 as an OS [Microsoft05e] Microsoft Corporation "Windows Mobile Version 5.0 SDK: Code Samples", msdn.microsoft.com Explicit reference to Windows Mobile OS Dvorak Soft Input Panel / SIP / simulated/virtual keyboard on Windows Mobile [Microsoft05f] Microsoft Corporation "Microsoft Tablet PC - Ink Analysis - Ink Recognition - Ink Collection", msdn.microsoft.com Ink Analysis has Divider and DivisionResult processing to segment ink for electronic ink editing, compare to van Raamsdonk. Ink Collection InkCollector object intercepts device input (any Tablet PC device including mouse) to lay down ink. InkOverlay extends InkCollector with events for start/begin/end stroke, ink editing (cut copy paste) and annotation. Multiple recognizers and alternate results -- recognition only on collections of strokes. [Microsoft05h] Microsoft Corporation "How to Buy a Tablet PC / How to buy the right Tablet PC", www.microsoft.com, 2005 List of hardware vendors for Tablet PC [Microsoft05i] Microsoft Corporation "Visual Studio 2005 -- Internet Archive", www.microsoft.com, 2005 Internet archive of Microsoft Visual Studio 2005, includes penwin.h (Windows for Pen Computing) [Microsoft05j] Microsoft Corporation "Application Gestures and Semantic Behavior", msdn.microsoft.com, MSDN, October 2005 Vista/Tablet PC SDK documentation from MSDN: defined gestures include scratch-out/erase, Triangle/insert, Check-off, Curlicue/fixed (compare with proofreader's marks), Double-Curlicue/copy, double-circle/paste, left-semicircle/undo, right-semicircle/redo, Caret gesture/paste, Inverted-caret/insert, flick left / backspace, flick right / space (compare with GO flick gestures), left/right/up/down corners with directional gestures. See also 2003 version of same reference. [Microsoft05k] Microsoft Corporation "WinHEC 2005 List of Tracks and Sessions", msdn.microsoft.com, 2005 List of presentations/workshops at Microsoft WinHEC 2005 conference. "Natural Input on Mobile PC Systems" UMPC / ultra-mobile PC. "Full Client Usability for the Highly Mobile Market". Check for tablet calibration, coil in stylus, parallax. [Microsoft05l] Microsoft Corporation "Microsoft Windows User Experience -- Offical Guidelines for User Interface Developers and Designers", msdn.microsoft.com, 2005 User interface guidelines/rules for Windows 95, Windows 2000, Windows Vista. context menu / pop-up menu / shortcut menu: invoked by Application key more than right mouse button / secondary mouse button. Publication date approximately 2005 (Windows Vista) earlier edition 1999. No mention of touchscreens, tablets, Tablet PC, pen computing. [Microsoft05m] Microsoft Corporation "Microsoft Mobile Device Hardware Overview (Pocket PC Developer's Guide)", msdn.microsoft.com, 2005 Hardware guidelines for vendors building Windows Pocket PCs: navigation/program buttons, resistive film touchscreen / resistive touch panel (!), stylus. Input panel (SIP) with on-screen virtual keyboard or handwriting recognition. Audio input/output. last update 2005, date for earlier version pre-2000? [Microsoft05n] Microsoft Corporation "Microsoft Windows XP Tablet PC Edition 2005 -- Tablet PC hardware models", msdn.microsoft.com, 2005 Listing of Tablet PC manufacturers and models. TravelMate, convertible models, HP tc1100 small format (compare UMPC), ruggedized. [Milekic05] Milekic, Slavoljub "User Interface for Removing an Object from a Display", United States Patent 6,920,619, July 19, 2005 User interface via direct manipulation on a touchscreen: in particular a gesture-like flick (rapid motion) results in object being "thrown away" / deleted rather than being moved around: the recognition of the feature is based on the velocity only, not on shape. Appears to refer to directional gestures with different meanings, but apparently does not describe a recognizer of shapes per se. Other disclosures are about proximity sensing, using multi-finger (two-user) behaviors of touch input. Include two-user/two-finger "transfer/share" gesture for resistive-sheet touchscreen. [Mimio04] Mimio / Virtual Ink "Mimio Interactive Whiteboards, by Virtual Ink", http://www.mimio.com wireless digitizer tablet, also works with laptop computer



[Mitsubishi04a] Mitsubishi Electric Research Laboratories "Freehand Touch Gestures", Available at www.merl.com multi-touch hand gestures on an interactive tabletop. Compare with Entertaible. [Mitsubishi04b] Mitsubishi Electric Research Laboratories "DiamondTouch", Mitsubishi Electric Research Laboratories Press Release: November 25, 2004. Available at http://www.merl.com Digitizer tablet using electrostatic/capacitive sensing in a table-top display surface. Multi-touch. Compare with Entertaible. [Mitsubishi05a] Mitsubishi Electric Research Laboratories "Haptic Stylus", Mitsubishi Electric Research Laboratories Press Release: July 14, 2005. Available at http://www.merl.com Part of DiamondTouch project: solenoid vibrator in a handheld stylus to give different tactile feedback when stylus of digitizer is over a particular icon or object in proximity [MoscovichT05a] Moscovich, Tomer; Igarashi, Takeo; Rekimoto, Jun; Fukuchi, Kentaro; and Huges, John F. "A Multi-finger Interface for Performance Animation of Deformable Drawings", Paper and demonstration (video on YouTube) from UIST '05, User Interface Software and Technology, October 2005 "Dragimation": video shows using multiple fingertips (up to 5, with two hands) to animate cartoon images of flying bats, crawling snakes, etc. Includes demo of finger sensing image on SmartSkin touchscreen digitizer. [MotionComputing04a] Motion Computing "Motion Computing M1400 Tablet PC User Guide", Motion Computing, 2004 User Guide for Tablet PC. Tablet PC Input Panel TIP (see PenWindows SIP): virtual keyboard/writing pad with recognition for input sent to active application/program. Speech recognition. Menu settings for lefthanded users, "Sending as Ink" from the Input Panel, Windows Journal electronic ink note-tacking, Sticky Notes. [MotionComputing04b] Motion Computing "Tablet PC Microsoft Windows XP Tablet PC Edition 2005 User Guide", Motion Computing, 2004 User Guide for Tablet PC operating system. [MotionComputing05a] Motion Computing "Motion Tablet PC Quick Start Guide", Motion Computing, 2005 Tablet PC computer: Tablet Input Panel (TIP) soft keyboard / writing panel. Fingerprint reader may be used for vertical scroll (also horizontal?) [Motorola04a] Motorola Corporation "Motorola MPx product information", www.motorola.com Touchscreen (stylus) cell phone with dual hinge: can open as flip-phone or in portrait orientation using Windows Mobile 2003 OS. Handwriting recognition in separate areas for lowercase, uppercase, and numeric characters. File contains additional materials [Motorola05a] Motorola Corporation "Finger Writing", www.motorola.com capacitive digitizer for determining center position of finger writing on a normal push-button keypad, Motorola Shanghai Human Interaction Research; Compare to Casio 8000 hand-PDA calculator unit from 1980's, which had handwriting input on pressure-membrane keypad [Motorola05b] Motorola Corporation "MotoMING A1200 MotoManual", www.motorola.com MotoMING A1200 GSM cellphone. Built-in stylus for touchscreen. Handwriting recognition dual area for single characters, time-out or swap writing in the two areas. Compare with Wang GUIDE? Subset/mode selection alpha, numeric, shift-capitalize, special characters. Document viewer with drag gesture for scrolling. Two-stroke (tap-plus-line) for zoom gesture. tap and hold gesture for pop-up menu on item. Dwell/press and hold. [MulliganRC05a] Mulligan, Roger C.; Badaye, Massoud; Wong, Alex K.; and Lim, Brian G. "Lattice touch-sensing system", United States Patent 6,970,160, November 29, 2005 Cross-conductor capacitive grid for touchscreen digitizer, senses multiple touches for gaming application, or for rejecting multiple touches / multi-touch. [MunsonJC04a] Munson, John C. and Elbaum, Sebastian G. "Dynamic Software System Intrusion Detection", United States Patent 6,681,331, January 20, 2004 Real-time detection of injected unauthorized code on a computer system by pattern matching to execution profiles for usual paths of code-path execution. Assigned to Cylant [MunsonJC05a] Munson, John C. and Elbaum, Sebastian G. "Method of and System for Detecting an Anomalous Operation of a Computer System", United States Patent 6,963,983, November 8, 2005 Real-time detection of injected unauthorized code on a computer system by pattern matching to execution profiles for usual paths of code-path execution. Assigned to Cylant [Mutoh05] Mutoh "Mutoh EasySIGN 4.5 Power Pack for Mutoh", http://www.mutoh.be Drawing/editing program for ink and text (?). Editing electronic ink and text: compare with van Raamsdonk? [NakamuraS04a] Nakamura, Shuji; Hayama, Masanobu; and Koike, Tamotsu "Mouse", United States Patent 6,677,930, January 13, 2004 Mouse with touch-sensitive surface instead of buttons, responds to degree of force and timing of taps on



areas of mouse (instead of buttons): press-and-hold [Neonode04a] Neonode.com "Neonode N1 Quick Start Guide", http://www.neonode.com Cellphone with optical touch screen. Right/left sweep/flick gesture for open/close tabs, up and down flick/sweep gestures for other functions. May be similar to marking menus? [NeuroScript05] NeuroScript "Handwriting links", http://www.neuroscript.net Bibliography, Web links to numerous patents, applications, papers on handwriting motor measurement, character and gesture recognition, other discussion forums [Nokia05a] Nokia "Nokia 770 Internet Tablet", www.nokia.de Small-format Internet Tablet / WebPad similar to Tablet PC / Apple iPad, stylus touchscreen, base OS is Linux, no cellular phone: instead VOIP over local WIFI or bluetooth connection to local workstation . Compare also with UMPC. [NorrieMC05a] Norrie, M.C. and Signer, B. "Overlaying Paper Maps with Digital Information Services for Tourists", Proc. ENTER 2005, 12th Int'l. Conf. on Information Technology and Travel and Tourism, Innsbruck, January 2005 Printed paper maps enhanced with Anoto digitizer/optical pattern technology. Cites also to Pegatech, Mimio and LeapPad paper on electronic tablet electronic books. Prototype is map book for Edinburgh festival. [NorthwayC05a] Northway, Craig "Understand Compositing and Color extensions in SVG 1.2 in 30 minutes!", www.svgopen.org/2005/papers/abstractsvgopen, August 23, 2005 Quick introduction to mechanics of alpha compositing / alpha blending of images. [NovakC05] Novak, Chris J. "Housing for a Computing Device", United States Patent 6,933,929, August 23, 2005 Housing for hand-held touch-pad/touchscreen device with palm rest: compare with MicroPad? [OTM04a] OTM Technologies "VPen Product Information", OTM Technologies, Inc. Optical digitizer pen, somewhat similar to Anoto, using optical tracking motion sensors (OTM) based on interferometry. (See also VIPDATA) Mentions integration with a cell phone, both as drawing and as handwriting recognition input three-dimensional motion digitizer, one dimension is pressure sensor? Included mode button in side of pen, similar? to PenCept (1986) mode button for recognition mode / digitizer mode VPEN product: Virtual Input Pen: cell phone with built-in optical digitizer pen, reads differential image of surface: contrast with Anoto? Mentions use in multimodal user interface for military users: compare with Cohen reference (Natural Interaction Systems LLC) [Origami05a] Origami WinHEC Product Announcements "Small Form-Factor Devices to Take Center Stage at WinHEC", www.microsoft-watch.com April 25, 2005 WinHEC preview of Origami small tablet PC, formerly code-named Haiku [PallakoffM05a] Pallakoff, Matt "System and Method for Portable Multimedia Client", United States Patent Application 2005/012723, January 20, 2005 PDA/portable viewer with touchpad surfaces on sides, so that user can touch (and slide to scroll) without obscuring display with finger. [Palm05a] PalmOne Press Release "palmOne Wins Summary Judgement Invalidating Xerox's Unistroke Patent", pressroom.palmone.com, May 21 2004 Unistroke patent ruled anticipated, based on broad interpretation of claims by plaintiff [Palm05b] palmOne "palmOne LifeDrive User Manual", palmOne Inc. Vertical upwards stroke as mode-shift gesture to enable/select punctuation recognition [PalmInfocenter05a] PalmInfocenter.com "Top 10 PDA "Failures" named", http://PalmInfocenter.com Comment on PenPoint as a failure / non-failure? [PalmInfocenter05b] LegoDude522 (sic) "Opening Palm Screen [Zire 72 and now T|C]", www.palminfocenter.com Series of photographs showing disassembly and reassembly of a Palm handheld PDA showing LCD display, digitizer [PalmSource04a] PalmSource, Inc. "Palm Powered Products", PalmSource List of various products using Palm OS: Treo smartphone by palmOne, Sony CLIE PEG-UX50 with full keyboard, Data Wireless by AlphaSmart with full keyboard and handwriting [PalmSource05a] PalmSource, Inc. "Palm OS Developer Suite 1.2 Data Sheet", www.palmsource.com Palm OS debuggers, compilers, simulators, SDKs, and related tools: mentions web browser. Palm OS Garnet release 5.4. [PaperPlusPlus05a] Paper++ "Beyond the Classroom Computer: Innovative Technologies, Learning and Education", paperplusplus.com Applications of optical digitizer smart paper in education: augmented story-telling book, augmented maths text book, using augmented paper Compare to Anoto and Sekendur technology: illustration of pattern in materials is cross-hatch pattern [PaperPlusPlus05b] Paper++ "Paper++ Project Overview", paperplusplus.com Non-optical pattern paper similar to Anoto: unit conductive barcode encoding page number and X/Y position, inductive barcode reader. Active areas on paper correspond to web content (links). list of papers 2001-2005. Compare to Anoto and Sekendur technology: Non-optical?



[PasqueroJ04a] Pasquero, Jerome; Levesque, Vincent; Hayward, Vincent; and Legault, Maryse "Display of Virtual Braille Dots by Lateral Skin Deformation: A Pilot Study", Proc. Eurohaptics 2004, Munich, June 5..7, 2004, pp. 96..103 Prototype study for braille tactile/haptic display element in contrast to vertical pins with norizontal piezoelectric actuators. Dots move laterally/sideways slightly instead of vertically: modeled on brushing motion of fingertips by Braille users. [Pegasus05] Pegasus Inc. "PC NoteTaker digitizer / Pegasus USB PC compact notetaker", www.rapserv.com/au/prod5454.htm Sonic digitizer, 100 DPI, intended as a portable product. Joint marking in a Refalo-style notebook, no OS built in, mentions interface to PenTel character recognition [PenComputing04a] Pen Computing magazine "Fujitsu' flexible touchscreens", Pen News: pencomputing.com, April 1, 2004 Flexible plastic substrate touchscreen from Fujitsu [PenOne04] Pen-One, Incorporated "Fingerprint Sensor Pen", Pen-One, Incorporated, www.pen-one.com Writing pen with fingerprint sensor built-in: appears not to be biometric signature verification. [PenWindows05] Microsoft Corporation "PENWIN.H", Microsoft Visual Studio 2005, Microsoft Corporation PenWin.h include file: gives definitions for both Pen Windows Pen Services 2.0, and Pen Services 1.0. Last copyright date in file is 1995. [PenpointMuseum05] Penpoint Museum "The PenPoint Museum", http://www.ojisan.com/penpoint/index.shtml In Japanese: On-line museum for PenPoint. Images of documentation in Japanese, future models of Hobbit other hardware, PenMagic and other applications. Sketches from hand-written design notebooks for PenPoint? [Pentel05] Pentel Co. Ltd. "AirPen Storage Notebook product information, PC NoteTaker", www.pegatech.com AirPen: Notebook/filofax with built-in ultrasonic/acoustic digitizer and pen, storage of handwritten notes as electronic ink, transfer to PC later Descriptions in Japanese and English: Developed by Pegasus Technology of Israel, manufactured by Pentel Co. Lt. of Japan: www.airpen.jp PC NoteTaker: ultrasonic/acoustic digitizer to be mounted on normal clipboard tablet Compare with Polhemus, Science Accessories acoustic/sonar digitizers, providing 3D position information. [PereiraJP04a] Pereira, Joao P. and Ferreira, F. Nunes "SmartSketches: A Multimodal Approach to Improve Usability in the Early States of Product Design: 2D Sketch Editor Manual and Report", Final reports and documentation, SmartSketches project, smartsketches.inesc-id.pt Sketch-editing UI with gestures: Gesture definition language (list of characteristics), ambiguous gesture result in pop-up selection, press-pause-and-drag (press-and-hold) with mouse is one Zoom gesture, pressand-hold near window's limits is rotate image. Gestures for solid-geometry images: block, pyramid, duct with rectangular section, revolution object. Rectangular-box gesture zoom is intentionally ambiguous (i.e. shape is not ambiguous, but may have different meanings depending on second menu input, produces expectation list / pop-up selection if zoom in, zoom out, zoom all. [PerskiH04a] Perski, Haim and Morag, Meir "Dual Function Input Device and Method", United States Patent 6,762,752, July 13, 2004 Combine touchscreen and electromagnetic/electrostatic grid stylus digitizer in one device. Finger touch does not sense proximity: resistive grid? States that can resolve multiple touches (up to two?) by rescanning suspect matrix/grid conductors. N-trig. [PhatWare05] PhatWare Corporation "CalliGrapher 7.5 for Pocket PCs", Available at www.phatware.com Handwriting recognition software for Windows CE: multiple/separate versions for European/Roman languages, includes pop-up keyboards, other GUI features, single-character mode, PenCommander mode (like recognition macros?) [Polyvision05] Polyvision Corporation "TS Series Interactive Whiteboards", www.polyvision.com Interactive whiteboards: touch-sensitive projection screen, software to capture annotations on the projected image [PorrasPA04a] Porras, Phillip Andrew and Valdes, Alfonso "Network Surveillance", United States Patent 6,711,615, March 23, 2004 Network intrusion detection by analyzing network traffic, including network errors and packet inspection. Refers to neural network analysis: compare with Cylant? [PoupyrevI04a] Poupyrev, Ivan; Okabe, Makoto; and Maruyama, Shigeaki "Haptic Feedback for Pen Computing: Directions and Strategies", Proc CHI 2004, April 24-29, 2004, Vienna, Austria Pen (stylus) similar to touchscreens, but faster and more fluid for drawing or small PDA displays. References to work on haptic mice. Important that haptic feedback not provide too much detail. Tactile/haptic feedback very useful in improving drawing/targeting precision by highlighting position of target. Haptic feedback not useful/noticed on tap/click, may be more useful on slow careful targeting of small targets.



[Psion05a] Psion Teklogix Inc. "Workabout Pro Hand-Held Computer with Mindows Mobile 2003 SE User Manual", Psion Teklogix Inc., Missisauga, Ontario Canada Hand-held computer (format is like hand-held terminal) with Windows Mobile SE operating system, alphabetic (non-QWERTY) keyboard. Touchscreen used basically for mouse clicks. [Quantum05a] Quantum Research "Quantum Research Application Note: Secrets of a Successful QTouch Design", Quantum Research Application Note AN-KD02: Secrets of a Successful QTouch Design Matrix scanning touch panel / touchscreen for touch keyboards. Technical considerations on circuit/trace layout, component placement, especially adjacent paths to ground, coupling due to moisture films on surface. Multiple key sensing (compare multi-touch). ASK Adjacent Key Suppression for unintended (weaker) secondary touches by registering key with strongest signal, not all keys above threshhold. Clear ITO on PET clear plastic with silver connections. [RamosG04a] Ramos, Gonzalo; Boulos, Matthew; Balakrishnan, Ravin "Pressure Widgets", Proc. CHI 2004, April 24..29 2004, Vienna, Austria, pp. 487-494 Investigation of uses for pressure-sensitive touchscreen/tablet input: Wacom Intuos tablet, Rekimoto SmartSkin, and Mitsubishi DiamondTouch. Four gestures: Click/Tap, Dwell/press-and-hold, quick release, and stroke, recognized by pressure profile over time. [RamosG05a] Ramos, Gonzalo and Balakrishnan, Ravin "Zliding: Fluid zooming and sliding for high precision parameter manipulation", Proc. UIST '05, October 23-27, 2005, Setalle, pp. 143-152 Pressure-sensitive (electrostatic?) touch screen, gestures (non-shape) combine both location/pointing and pressure: i.e. fine-control slider controls finer resolution when user presses harder while dragging the stylus. [RaymondSA05] Radmon, Stephen A. "Source Documents on Handheld Devices for Clinical Trials", SBIR Contract NIMH N44MH22051 for PHS-2001-1 (026), "Electronic Source Documents" SBIR report on project to put psychological questionaires on Tablet PCs: minimal use of electronic ink capture [RichterPJ05a] Richter, Paul J.; Cairns, Darran R.; and Bottari, Frank J. "Touch Input Sensing Device", United States Patent Application 2005/0073507 A1, April 7, 2005 4-Wire (?) Resistive sheet capacitive touchscreen with stylus, with flexible glass on top. [RingelM04a] Ringel, Meredith; Ryall, Katy; Shen, Chia; Forlines, Clifton; and Vernier, Frederic "Release, Relocate, Reorient, Resize: Fluid Techniques for Document Sharing on Multi-User Interactive Tables", Proc. CHI 2004, April 24-29 2004, Vienna, Austria, pp. 1441-1444 Usability study of multi-user / multi-touch user interfaces for sharing documents: compare two-finger/twouser "release" double-touch gesture with Milekic share/transfer gesture. Uses DiamondTouch touch table with identification of users. [RobbinsDC05a] Robbins, Daniel C.; Sarin, Raman K; Horvitz, Eric J.; and Cutrell, Edward B. "Advanced Navigation Techniques for Portable Devices", United States Patent Application 2005/1095154 A1, September 8, 2005 Zoom by zones using keypad: button to show zones (laid out same as keypad) on displayed image, press matching button to zoom in to that zone. May show world view of original image on top (semitransparent). Uses Press-and-hold/pressing-and-holding/holding-and-releasing on button for temporary zoom. [RogersY04a] Rogers, Yvonne; Hazlewood, William; Blevis, Eli; and Lim, Youn-Kyung "Finger Talk: Collaborative Decision-Making Using Talk and Fingertip Interaction Around a Tabletop Display", Proc. CHI 2004, 24-29 April, Vienna Austria, pp. 1271-1274 Collaborative software for multiple users using multi-touch DiamondTouch touchscreen table: cites also to Rekimoto Smartskin, other tabletop displays. Users tended not to use interface simultaneously (social interaction), taking turns. Many gestures/manipulation were directed to inter-human communication in conjunction with speech, rather than manipulating calendar objects. [RossiterJ05a] Rossiter, Jonathan and Mukai, Toshiharu "A Novel Tactile Sensor Using a Matrix of LEDs Operating in Both Photoemitter and Photodetector Modes", Proc. 4th IEEE Int'l. Conf. on Sensors. IEEE Sensors, Irvine, California October 30 - November 3, 2005, pp. 994-997 Use LED "in reverse" as an optical sensor for a touchscreen display using one set of LEDs. Uses deformable / elastomeric layer on top, light from one LED in emitter mode is reflected and detected by adjacent LED in detector mode, then the other way around, compute ratiometric position between the two. Prototype is 4x4 matrix. Cites to Albert Einstein's 1905 paper on the photoelectric effect. Compare with Abileah. [RowleyH04] Rowley, Henry; Bennett, John; Abdulkader, Ahmad; and Slavik, Peter "Reducing and Controlling sizes of Prototype-Based Recognizers", United States Patent Application 2004/0002986 A1, January 1, 2004 For a training-based (handwriting) recognizer using fixed prototypes, method of determining the prototype which has the greatest contribution to error, and reducing the training set size by removing that prototype. Further reduction by removing prototypes with the least increase in error when they are removed.



[RowleyH05] Rowley, Henry; Bennett, John; Abdulkader, Ahmad; and Slavik, Peter "Reducing and Controlling sizes of Prototype-Based Recognizers", United States Patent 6,970,882 B2, November 29, 2005 For a training-based (handwriting) recognizer using fixed prototypes, method of determining the prototype which has the greatest contribution to error, and reducing the training set size by removing that prototype. Further reduction by removing prototypes with the least increase in error when they are removed. [RyuJ04a] Ryu, Jonghyun and Kim, Gerard Jounghyun "Using a Vibro-tactile Display for Enhanced Collision Perception and Presence", Proc. VRST '04, November 2004, Hong Kong Experiment in vibro-tactile haptic feedback for simulating collision/contact between user and virtual reality objects. Multiple vibrators strapped on to different parts of arms, legs. Works better if also used with stereo or binaural 3D sound for 3D location. [SandburgB04] Sandburg, Brenda "Judge Torpedoes Dead Inventor's Patent Claims", Available at www.law.com Cognex lawsuit against Jerome Lemelson "submarine patents" on machine vision from 1950's. [SantoroO04a] Santoro, Ovid and Lagermann, Klaus "System and Method for Simultaneous Display of Multiple Information Sources", United States Patent 6,724,403, April 20, 2004 Display/refresh information in from different sources at same time in separate tiled windows, arranged in a grid. States that present art does not auto-update (?). Compare with TV split-screen, or split frames in HTML? [Sattler05] Sattler, Michael "Newton: The Dawn of PDAs", www.GeekTimes.com/michael/techno/computing/hardware/products/apple/newton/index.html Personal history of past work at GO/Penpoint, Newton PDA MessagePad, MessagePad 2000 / MP2K development manager Sandy Bennett, [SchwesigC04a] Schwesig, Carsten; Poupyrev, Ivan; and Mori, Eijiro "Gummi: A Bendable Computer", Proc. CHI 2004, April 24-29, Vienna, Austria Graspable user interface UI with flexible tablet and 2-D two-dimensional digitizer touch sensor on back side. Graspable/organic user interfaces. [Seiko04a] Seiko Instruments "Seiko InkLink Handwriting System", jdjstore.com, 2004 Distributor product listing for Seiko InkLink electronic pen: acoustic/sonic digitizer. [SensAble04a] SensAble Technologies, Inc. "SensAble Phantom Omni User's Guide", SensAble Technologies, 2005 Mechanical 6DOF digitizer, with 3D force haptic feedback, desktop size. [SharpJL05a] Sharp, Jeffrey L. "Touch Sensor System to detect Multiple Touch Events", United States Patent 6,856,259, February 15, 2005 Detect multiple touches overlapping in time (simultaneous) by comparing with known actual touch events: multi-touch. [ShenC04a] Shen, Chia; Vernier, Frederic D.; Forlines, Clifton; and Ringel, Meredith "DiamondSpin: An Extensible Toolkit for Around-the-Tablet Interaction", Proc. CHI 2004, 24-29 April 2004, Vienna, Austria, pp. 167-174 Software toolkit for multi-touch multi-user interaction on a multi-touch tabletop (e.g. DiamondTouch table). Shows simulated documents/papers stacked: compare with Wang Freestyle (which was single-user)? Can display Java Swing windows at any rotation by rotated frame of reference. Also: electronic video file of demonstration. [Siemens04a] Siemens "Siemens to show prototype PenPhone: Integrated handwriting recognition technology enters written data on mobile phone", InfoWorld, 2004/02/09 Siemens prototype handwriting pen(?) for smartphone "may be used on any surface" as digitizer with handwriting recognition, as alternative to thumb-typing SMS messages. Few details. Handwriting recognition for composing SMS message. Compare with IBM Simon? [SignerB05a] Signer, Beat "Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces", Ph.D. Thesis, ETH Dissertation 16218, Switzerland, 2005 Thesis on general topic of electronic paper / digitally augmented paper / smart paper: Paper++ pen technology: also Seiko InkLink acoustic (sonic) digitizer, WACOM two-stylus multi-touch tablet, SmartDesk multi-hand project, Bar Code grid (compare with Anoto) using inductive pen. Notes that Microsoft Universal Pen (u-Pen) similar to Anoto, interactive books (Sony LIBRIe, LeapPad) [Silberman04] Silberman, Steve "The Hot New Medium: Paper. How the oldest interface in the book is redrawing the map of the networked world.", Wired.com, April 2001, archive at www.wired.com Tomas Edso and Mats-Petter Petterson devised dot-displacement pattern, Hugosson used larger and smaller dots Business model was to be a charge on all transactions that involved Anoto paper pattern, not directly on the licensing of technology Interview about Anoto with Christer Faehraeus, mentions comparison with Microsoft Tablet PC, which overshadowed their introduction at ComDex [SilberschatzA05a] Silberschatz, Avi; Calvin, Peter Baer; and Gage, Greg "Operating System Concepts, 7th Ed.", John Wiley and Sons, Inc., 2005 Upper-level course textbook on operating systems design and architecture: Unix (Mach, UNIX BSD), Windows (XT/2000,64). Historical notes on Multics protection domains / ring system, ACLs and capabilities;



demand paging 1960 on B5500 and Atlas. [SilverbrookK04a] Silverbrook, Kia; Lapstun, Paul; and Lapstun, Jacqueline Anne "Method and System for Accessing the Internet", United States Patent 6,813,039 B1, November 2, 2004 Optical digitizer for handwriting connecting directly to a Web page on the internet: web page with links would be printed, not displayed dynamically. Compare with Sekendur and Anoto. [SilverbrookK04b] Silverbrook, Kia; Lapstun, Paul; and Lapstun, Jacqueline Anne "Method and system for collaborative document markup", United States Patent 6,766,942, July 27, 2004 Using an Anoto-like paper pattern optical digitizer, allow multiple users to generate mark-up a document together, while communicating/discussing via telephone or video conference. [SilverbrookK04c] Silverbrook, Kia and Lapstun, Paul "Computer System Interface Surface with Processing Sensor", United States Patent 6,766,944, July 27, 2004 Using an Anoto-like paper pattern optical digitizer, includes description of using parallax of pattern to correct for tilt offset, compute theta, phi, and rho angles. [SilverbrookK04d] Silverbrook Research "Press reports related to Silverbrook Research, Australia", (various) Kia Silverbrook: Past association with Canon until 1994. Business model is to file patents in Australia and enforce them internationally. See also Hyperlabel and patent application with DigiMarc on security document authentification. [SmartTech04a] SMART Technologies Inc. "Actalyst digital signage touchscreens", www.smarttech.com (In German): DViT (Digital Vision Touch) overlay touchscreen for large displays, optical touchscreen using high-precision cameras in frame. [SmartTech05a] SMART Technologies "SMART digitizer product information", www.smarttech.com SMART board interactive whiteboard digitizer; Symposium interactive pen display tablets; AirLiner wireless slate table (relabeled Wacom product?) Whiteboard product: Includes software (Bridgit) for live annotation of displayed documents, PowerPoint presentations [SpraulVA05a] Spraul, V. Anton "Computer Science Made Simple", Broadway Books Div. of Random House, 2005 Overview of computer science concepts. Distinguishes between Alpha Blending (fade effect) and BitBLT / bit blits. [StahovichTF04a] Stahovich, T.F. "Segmentation of pen strokes using pen speed", Proc. AAAI 2004 Fall Symposium: Making Pen-Based Interaction Intelligent and Natural For electronic ink editing and cursive recognition, segment pen strokes into lines and arcs using dwell points (pauses) and point of high curvature. Goal is to match user's intent, not exact ink pattern. Compare to Rubine, PenCept segmentation patterns. [StallingsW05a] Stallings, William "The Windows Operating System: excerpt from Operating Systems: Internals and Design Principles, 5th Edition", Prentice Hall, 2005 Windows OS architecture: "modified" micro-kernel architectures of Windows 2000 and later. Multiple data streams in files to support MacIntosh. References to Windows 3.0 memory mapping and segementation. [StefikMJ05a] Stefik, Mark J. and Pirolli, Peter L. "Content Rendering Respository", United States Patent 6,963,859, November 8, 2005 DRM usage rights management, usage rights control on permissions for what rendering/use is granted to a document or object [SwigartS05a] Swigart, Scott "Easily Write Custom Gesture Recognizers for Your Tablet PC Applications", www.microsoft.com, publication date November 2005 Writing custom recognizer for gestures for Windows XP Tablet PC edition SDK. Refers to open-source SIGER (Simple Gesture Recognition Library) from sourceforge.net. Gestures include Microsoft Tablet PC gestures: curlicue, circle, square, star, gesture, plus left and right bracket gestures. Includes unit testing against canned gesture input. Recognition apparently based on chain codes of vector increments, complete strokes only. [Synaptics04] Synaptics Incorporated "Capacitive Position Sensing", Synaptics Incorporated, San Jose California Descriptions of Capacitive, Inductive, and resistive tablet technologies from Synaptics. Describes combining capacitive (finger-touch) and inductive (stylus) to work with both, or to work with resistive SPIRAL (TM) Pen, powered via induction from tablet [TabletPC04a] Microsoft "Microsoft Experience Pack for Tablet PC", Microsoft.com, April 4, 2004 Tablet PC application utilities: Ink Desktop: note-taking on desktop. Snipping Tool: Screen grab plus ink annotation (and voice a la Freestyle?). Ink Crossword. See also Microsoft Education Pack for Windows XP Tablet PC Edition, Microsoft Office XP Pack for Tablet PC, OneNote note taker. [TabletQuestions04] TabletQuestions discussion list "Lack of Pen Gesture Applications", www.tabletquestions.com, May 26, 2004 Posting that so few TabletPC applications "think in ink" (a.k.a. pen-centric). Mentions Corel Grafigo and Wavefront as exceptions, because they took the whole application from the TabletPC Perspective [Tactiva05a] Tactiva.com "Tactiva TactaPad Technology", www.tactiva.com, fetched 2005



Effectively transparent touchscreen by having multi-touch surface on desktop with small camera above, shadow images of hands/fingers are shown on display. Multi-touch tablet digitizer includes pointaddressable tactile output what provides haptic feedback only where surface is being touched. See also Fish, D.E. [Tactiva05b] Tactiva.com "Tactiva TactaDraw Introduction to the TactaPad", www.tactiva.com, fetched 2005 Video demonstration of TactaPad technology. Effectively transparent touchscreen by having multi-touch surface on desktop with small camera above, shadow images of hands/fingers are shown on display. Multitouch tablet digitizer includes point-addressable tactile output what provides haptic feedback only where surface is being touched. Not only multi-touch: shows pinch and stretch gestures, also simultaneous use of multiple tools with multiple fingers. Because of one-to-one mapping of touch tablet to display, and shadow image of fingers, tools/menus are at same physical position on touch surface, and have tactile feedback about location, selection, detent, etc. See also Fish, D.E. [Tactons05a] Tactons Website "Tactons: an investigation of multimodal interaction with tactile displays", www.dcs.gla.ac.uk/~eve/TactonsWebsite/tactons.html University of Glasgow: research group on haptic (touch-based) human-computer interaction using tactile cues combined with auditory (cross-modal) input, vibrotactile transducers, pin arrays (compare with Braille display and Optacon aid for visually impaired) [Tekscan05a] Tekscan, Inc. "Tekscan FlexiForce Force Sensors User Manual", Tekscan Inc, South Boston, Massachusetts, 2005 Force/pressure sensing touch matrix (pad or tablet) using matrix of conductors and force-sensitive conducting ink [Telautograph05] Telautograph: Wikipedia Encyclopedia "Telautograph", en.wikipedia.org/wiki/Telautograph History of the Telautograph, 1890 precursor to Fax/Facsimile device and to digitizing tablets: Elisha Gray [TexasInstruments05a] Texas Instruments Incorporated "Using resistive touch screens for human/machine interface", Texas Instrument Incorporated, 2005 Description of single-ended (absolute) and differential/ratiometric measurments on resistive touchscreen controllers, settling time, bad-data (wild point measurement) rejection. Covers four-wire touch screens, mentions five-wire touchscreens. [TheAge05] The Age / Fairfax digital "Go files anti-trust suit against Microsoft", www.theage.com.au Press article in Australia on GO / Microsoft anti-trust lawsuit: quotes Bill Gates and Jerrold Kaplan [Thom-Santelli04a] Thom-Santelli, Jennifer "Effects of a Multitouch Keyboard on Wrist posture Typing performance and Comfort", Master's Thesis, Cornell University NY, 2004 Ergonomics evaluation of MultiTouch Keyless Keyboard from Fingerworks (human factors) [TognazziniB04a] Tognazzini, Bruce "Make Your Tungsten T3 Palm a Monster Machine", www.asktog.com, August 2004 Review of Tungsten Palm T3 hand-held computer: with Graffiti handwriting recognition: suggests kinetic scroll, faster stroke scrolls farther. [TomsHardware05] TomsHardware.com "Wacom stellt besonders kleinen Pen für PDA und Smartphones vor", http://www.tomshardware.com Wacom Super Slim Pen, very small form-factor penabled (Wacom term for pen-enabled) electromagnetic digitizer for PDA and Smartphones market. Refers to "Flight Point" functionality for moving the cursor without touching the screen (proximity sensing) a.k.a. hover [Topeka05] Topeka Computing Museum "The Topeka Computing Museum", http://www.gateman.com/museum/t3.html Has functional Pencept Penpad "white board" digitizing tablet with handwriting recognition from 1986 [TornilB04a] Tornil, Bertrand and Baptiste-Jessel, Nadine "Use of Force Feedback Pointing Devices for Blind Users", Proc. UI4ALL, 2004 Haptic/force feedback for blind users: Wingman Force Feedback Mouse and PhanToM 3D electromechanical digitizer with force feedback. Best they could do was tell blind user about bounding boxes in 2D or in 3D, not texture. Notes that devices for blind users also useful for "visually occupied" users. [Toshiba04a] Toshiba America Electronic Components "Toshiba debuts first full-color system on glass (SOG) input display with image capture technology", Toshiba America, May 25, 2004 touchscreen/"input display" integrated LCD display and 2D matrix of optical pixel sensors together. One application is laying business card on display to scan it in. [TouchInternational05] Touch International, Inc. "Touch International Inc.", http://www.touchinternational.com touchscreen digitizer "Digital Ink" -- successor to Electronic Ink?. Capacitive and resistive digitizer in one unit for both finger and stylus tablet input -- e.g. multi-touch. [TsangMH04a] Tsang, Michael H. "Correction of alignment and linearity errors in a stylus input system", United States Patent, 6,727,896, April 27, 2004 Correction table for tablet/touchscreen by trapezoidal correction of grid locations across the entire surface.



Compare with Scriptel? [TseE05a] Tse, Edward; Shen, Chia; Greenberg, Saul; and Forlines, Clifton "Enabling Interaction with Single User Applications through Speech and Gestures on a Multi-user Tabletop", Mitsubishi Electric Research Laboratories Report TR2005-130, December, 2005 Multi-touch desktop extended to multiple users, touch input can identify particular user. Case studies are Google Earth and Warcraft III. Incorporates speech and gesture input. [UllmannCN04a] Ullmann, Cristi N.; Wynn, Allen C.; and Brown, Michael W. "Rubber Band Graphic User Interface control", United States Patent 6,677,965, January 13, 2004 Rubber-band user interface for sliders and other controls: pull pointer/stylus away from control, see thin stretched rubber band, scrolls faster. Bring closer, rubber band is short and fat, scrolls more slowly. Touchpad PDAs. Compare with Apple bounce-back? [UllmerB04a] Ullmer, Brygg "Tangible User Interface / TUI Design Aspects", Lecture slides 2004 Overview of design principles and aspects for tangible user interfaces. Cites to Metadesk. Counterparts in GUI and TUI: icons/phicons, selectable trays/menus, etc. Does not include tablettop interfaces or touchscreen direct manipulation. [VIPDATA05] VIPDATA "VIPDATA/VPen Product Information", www.eu-vipdata.com (see also OTM Technologies) VPEN product: Virtual Input Pen: cell phone with built-in optical digitizer pen, reads differential image of surface: contrast with Anoto? Mentions use in multimodal user interface for military users: compare with Cohen reference (Natural Interaction Systems LLC) [VanNess05] VanNess, Shawn and Wakeam, Jamie "Windows Vista Tablet PC: Tablet Enabled Applications", downloads.microsoft.com Powerpoint presentation: PRS 315 from PDC05 conference (?): describes Ink Analysis API as superset of handwriting text recognition. Ink data types and stylus input in WinFX Windows Presentation Foundation WPF. Mentions handwriting / ink input annotation over video [VassalloK04] Vassallo, Ken "Tablet PC Software Design Considerations", Microsoft Tech-Ed Presentation, Microsoft Mobile Design Conference MPC 331, February 25, 2004 UI User Interface guidelines for tablet PC developement: Hand Obstruction of menus and scrollbars; use electronic/digital ink as data type (ink-centric) instead of handwriting recognition; editing of ink with Windows ink edit control can be awkward; targeting/positioning with stylus/pen is imprecise; describes difference between mouse click/double-click and gesture tap/double-tap; digitizer performance affected by EMI electromagnetic interference, and accuracy bad at edge of screen/tablet; mentions visual parallax error [VernierF04b] Vernier, Frederic; Shen, Chia; and Lesh, Neal B. "Circular Graphical User Interfaces", United States Patent 6,791,530, September 14, 2004 Tabletop touchscreen user interface, with all objects/windows rotatable by users. Circular table. Menu ribbons (curved) at edges of tables for particular users, on-screen virtual keyboards. Cites to DigitalDesk, ConnecTables, Wellner, i-LAND.. Mentions physical coffee mugs not as part of the invention, but the invention matches normal collaborative work by letting users place coffee mugs on the tabletop display. [VlackK05a] Vlack, Kevin; Mizota, Terukaau; Kawakami, Naoki; Kamiyama, Kazuto; Kajimoto, Hiroyuki; and Tachi, Susumu "Gelforce: A Vision-based Traction Field Computer Interface", Proc. CHI 2005', Portland, OR, p. 11541155 Multi-touch, force/pressure tactile sensing touchscreen, optical/video sensing from below of deformation/change in position of small colored marker balls in flexible gel sheet. Two layers of balls of different colors increase force resolution. Traction field (X/Y/pressure) maps similar to Rekimoto sensing proximity/contact area. (Rekimoto not cited) [VogtF04a] Vogt, Florian; Chen, Timothy; Hoskinson, Reynald; and Fels, Sidney "A Malleable Surface Touch Interface", Proc. SIGGRAPH '04, p. 36 Proposes flexible/malleable/bendable touch membrane, which would allow true stretch in the input surface: provide passive haptic feedback Mentions Smartskin, Fingerworks, and MTC Express as possible multi-touch technologies. [VonHagen05] von Hagen, Bill "Bill von Hagen's Computer Collection: ... life in the past lane ...", www.computermuseumofpa.com Collection include Pen-based computers/PDAs/calculators Apple Newton, Casio Z-7000, EO 440/880, GO Corporation Hyde(286)/Lombard(386), others [VonWrede04a] von Wrede, Caspar and Laskov, Pavel "Using classification to determine the number of finger strokes on a multi-touch tactile device", Proc. 12th Euro. Symp. on Artificial Neural Networks, Bruges, April 2004, pp. 459-555 Describes limitations of ambiguity (?) of Synaptics touchpads for multi-touch, better commercial solutions from Tactext Controls and Fingerworks, which are stated to have more complex hardware and lower spatial resolution (?). Algorithms for disambiguating multiple touches, by processing and interpolating



images from grid of touch sensors. [Voorhees04] Voorhees, Mark "Go Fish: How Fish and Richardson has climbed to the top of our annual list of most active IP practices", American Lawyer, May 2004 Review of patent law firm growth by acquisition: Fish and Richardson: Microsoft client [Voy05] Voy Wiederhold "Personal Digital Assistants (PDAs) Timeline", wwweb.stanford.edu/pub/voy/museum/pictures/display/0-2-PDAs.htm Photos of Penpoint GO/Lombard unit, Palm units from Gates Computer Science building, Stanford University. File also includes list of exhibits at the Computer History Museum, 2005. [WIPTE05] WIPTE "Call for papers, WIPTE 2006, First Workshop on the Impact of Pen-based Technology on Education", www.purdue.edu/wipte Mentions direct support by Microsoft Research for Tablet PC to education. Corporate Sponsors include Thomson Corporation, www.thomson.com [W3C04a] W3C www.w3.org "Ink Markup Language", W3C Working Draft 28 September 2004, version www.w3.org/TR/2004/WD-InkML-20040928 www.w3.org/TR/inkML XML Exchange format specification for electronic ink data interchange: handwriting and gesture recognizers, signature verifiers, ink-aware (pen-aware?) applications on Web. Includes pen angle, pen tip force (pressure), sampling mode, sampling rate, screen context. Compare with JOT binary standard. [Wacom04a] WACOM Europe "OQO setzt Penenabled-Technologie von Wacom in neuem leistungsfähigen Personal Computer im Taschenformat ein", www.wacom-europe.com Press release on OQO using Wacom electromagnetic digitizer [Wacom05a] WACOM Europe "The History of WACOM", www.wacom-europe.com History of Wacom: 1983 CAD Tablets later pressure-sensitive tablets for graphics and pre-press, desktop publishing, GO/PenPoint [Wacom05b] WACOM Co., Ltd. "Annual Report 2005, Wacom Co. Ltd.", www.wacom.co.jp Wacom Annual report 2005: mentions products for electromagnetic digitizer for the PDA market: W8802 Pen-sensor chip for mobile phones, Smartphones, PDAs and digital cameras, in addition to Tablet PC market. Super Slim Pen for PDAs introduced in April 2005. Wacom Penabled refers to custom-engineered digitizer components. [Wacom05c] WACOM Co., Ltd. "Wintab cursor mask example program, WinTab backgrounder, WinTab specification 1.1", www.wacomeng.com API Description of WinTab tablet digitizer / window management with virtual tablet frames. "Notes for Tablet-aware application programmers". Relevant to Schumer. [WalkerG05a] Walker, Geoff "The Last Word: The Future of the Tablet PC", Pen Computing Magazine / Veritas et Vitus, December 2005, pp. 34-37 Essay on lack of market success of Microsoft Tablet PC, and that other pen computers (touchscreen computers) will come. [WangJ04a] Technology Review "Universal Pen: if Jian Wang had his way, Everything would be Digital", Technology Review, May 2004 universal pen / uPen: Optical digitizer using printed "random" pattern on printer: compare with Anoto. [WangJT04a] Wang, Jintao and Canny, John "FingerSense - Augmenting Expressiveness to Physical Pushing Button by Fingertip Identification", Proc. CHI 2004, april 24..29, Vienna, pp. 1267..1270 Experimental prototype for which finger is used (FingerSense) on physical button. Prototype uses optical tracking, colored tags on fingers. 2nd prototype using FTIR frustrated total internal reflection fingerprint sensor (not multi-touch touchscreen) on each button. Cites to FingRing/fingerRing. [WardJR05a] Ward, Jean Renard "Protection of Identification Documents using Open Cryptography", United States Patent Provisional Filing 20050132194, June 5, 2005 Attaching cryptographic signature or other signature means to a document or manufactured item to identify the provenance, and thus the manufacturing station etc. of the document or item [WardJR05b] Ward, Jean Renard "Detection of system compromise by correlation of information objects", United States Patent Provisional Patent Filing, Cylant.com, filed September 22 2005 Detect hidden rootkits in real time by instrumenting for internal inconsistencies (violation of invariant conditions): e.g. there is an executing process, but it's parent .EXE file does not show up as a file on the filesystem. Real-time version of comparing two views of the data structures of a system. [WestermanW04a] Westerman, Wayne Carl "System and method for recognizing touch typing under limited tactile feedback conditions", United States Patent 6,677,932, January 13, 2004 multi-touch surface (MTS) touchpad used for full-keyboard (QWERTY) input, where finger touches do not need to be at exact position of the button locations on the touchpad. Hypothesizes home row key locations based on touches, uses best match. [WestermanW05a] Westerman, Wayne and Elias, John G. "Method and apparatus for integrating manual input", United States Patent 6,888,536, May 2, 2005



Multi-touch multi-finger gestures: two/three/four/five finger zoom/scale/rotate, pinch/anti-pinch, fingerchord gestures [WillemsD05a] Willems, Don; Rossignol, Stephane; and Vuurpijl, Louis "Mode detection in online pen drawing and handwriting recognition", Proc. ICDAR '05 Eighth Int'l. Conf. on Document Analysis and Recognition, pp. 31..35 Mode detection (handwriting, gesture, types of shapes triangle, circle, line etc.) recognition prior to classifying particular gesture. Based on pen trajectory and some shape classifcation. First step level is handwriting vs. shape, shapes then by shape (e.g. triangle/square/ellipse) vs. line/arrow. References to chain codes. [WilliamsHL05a] Williams, Hilary Lyndsay (nee Robinson) "Portable Computers", United States Patent 6,956,564, October 18, 2005 SmartQuill: pen digitizer with accelerometers sensitive to movement in gravity, small display. Handwriting recognition, built-in calendaring and other applications. Additional buttons (several) on pen housing for application user interfaces. Compare with Hugh Crane accelerometer pen, Anoto. See also paper file, additional references in electronic file back to 1997. [Williamson05a] Williamson, Peter H.; Lui, Charlton E.; and Altman, Dan W. "Method and System of Handling the Selection of alternates for Recognized words", United States Patent Application US 2005/0096914 A1, May 5, 2005 For low-confidence matches on handwriting recognition, display menu list with the various matches to user when the confidence score for the result is not high. User may set confidence threshold. User is shown ink image of the ambiguous word along with the alternates, makes selection. Compare with Chinese/Kanji input which shows alternate choices for the same input characters or strokes [Williamson05b] Williamson, Peter H.; Hawley, Nina; and Lu-Porter, Li "Implementing Handwritten Shorthand in a Computer System", United States Patent Application US 2005/0180633 A1, August 18, 2005 Handwriting recognition practically adaptive, in that user can define character text abbreviations to be expanded to larger text or to invoke an applications program or function. Interpreter sensitive to context of writing (e.g. which application/input it was written in?). No apparent relation to recognition of Pittman or Gregg shorthand, etc. [WilsonAD04a] Wilson, Andrew D. "TouchLight: An Imaging Touch Screen and Display for Gesture-Based Interaction", Proc ICMI '04, October 13-15, 2004, State College. Pennsylvania, pp. 69-76 Transparent touchscreen/whiteboard, projects video image of graphics and video image of counterpart user on same surface. Touches and gestures recognized by image processing of brighter light where hand/finger is in contact with surface: compare with Jeff Han FTIR. Cites to Clearboard. [WilsonAD05a] Wilson, Andrew D. "PlayAnywhere: A compact Interactive Tabletop Projection-Vision System", UIST '05, October 23-27, 2005, Seattle, Washington, USA Tabletop digitizer tablet, comparing to interactive table and whiteboard wall imaging touch screens. Optical finger tracking using the endpoints of the figures in a camera image. Touch is detected by heuristics that when in touch, finger will obscure its own shadow, so can also detect hover/proximity. Describes gesture of two hands moving apart to make an image large by zoom. Cites earlier work on TouchLight. Includes video (in file). [Wimmer04] Wimmer, Raphael; Holleis, Paul; Kranz, Matthias; and Schmidt, Albrecht "Thracker - Using Capacitive Sensing for Gesture Recognition", Proc. 6th International Workshop on Smart Applications and Wearable Computing, Lisbon Portugal, July 2004 Electrostatic/capacitive digitizer to recognize 3D gestures, hand movement gestures in front of a display in proximity: zooming, scrolling, and a "pick up" pinching gesture with the fingers for direct manipulation [Winograd05] Winograd, Terry and Verplank, Bill "HCI at Stanford University", ACM Interactions, Vol 12. No 5, September and October, 2005 Description of courses for master's program in human-computer interaction at Stanford: mentions haptic interaction [WirtgenJ04a] Wirtgen, Joerg "Nachserviert: Microsoft's Tablet PC Edition Version 2002", www.heise.de, September 28, 2004 Tablet PC Service Pack 2 (2002?), context-sensitive syntax rules for recognition of URLs and other formatted text in input fields, electronic ink in Web Browser input through web controls, In-place TIP (Tablet PC Input Panel) with writing pad mode, write-anywhere: see also Microsoft file. Remote-control RDP extended so that recognition on Tablet PC as controller may be used with remote system that does not have Tablet PC input, instead of just accepting stylus as mouse replacement. Input fields in applications automatically (?) extended with tap icon to bring up TIP for handwriting recognition input. [WobbrockJO04b] Wobbrock, Jacob O.; Myers, Brad A.; Aung, Htet Htet; and Lopresti, Edmund F. "Text Entry from Power Wheelchairs: EdgeWrite for Joysticks and Touchpads", Proc. ASSETS '04, October 18-20, 2004, Atlanta. pp. 110-117 Edge Keyboard / EdgeWrite unistroke alphabetic handwriting using joystick or touchpad, user moves to



corners and sides of small hand-held touchscreen with a frame: recognition depends only on order of corners, not on continuous contact. Intended for one-handed handwriting input, for persons with disabilities (wheelchair, use same joystick as wheelchair control) [WobbrockJO05a] Wobbrock, Jacob O. and Myers, Brad A. "EdgeWrite: A New Text Entry Technique Designed for Stability", Proc. Rehabilitation Engineering RESNA 28th Intl'l Annual Conference, Atalanta Georgia, June 23-27, 2005 Edge Keyboard / EdgeWrite unistroke alphabetic handwriting using joystick or touchpad, user moves to corners and sides of small hand-held touchscreen with a frame: recognition depends only on order of corners, not on continuous contact. Intended for one-handed handwriting input, for persons with disabilities (wheelchair, use same joystick as wheelchair control) [WobbrockJO05b] Wobbrock, Jacob O.; Aung, Htet Htet; Myers, Brad A.; and Lopresti, Edmund F. "Integrated Text Entry from Power Wheelchairs", Behavior and Information Technology, Vol 24, No 3, pp. 187-203, May 1 2005 Edge Keyboard / EdgeWrite unistroke alphabetic handwriting using joystick or touchpad, user moves to corners and sides of small hand-held touchscreen with a frame (Synaptics Touchpad, with side buttons): recognition depends only on order of corners, not on continuous contact. Intended for one-handed handwriting input, for persons with disabilities (wheelchair, use same joystick as wheelchair control) [Wohn05] Wohn, KWang Yun "Inside the Light Pen / Gun", Course notes for CS480, Korea Advanced Institute of Science and Technology Design and characteristics of light pens [WuM05] Wu, Mike; Shen, Chia; Ryall, Kathy; Forlines, Clifton; Balakrishnan, Ravin "Gesture Registration, Relaxation, and Reuse for Multi-Point direct-Touch Surfaces", Mitsubishi Electric Research laboratories (MERL) Technical Report TR2005-109, October, 2005 Two-handed (bimanual) continuous gestures: multi-touch. See 2006 paper same title. [WuX04] Wu, Xiaojie "Achieving Interoperability of Pen Computing with Heterogeneous Devices and Digital Ink Formats", Master's Thesis, University of Western Ontario, London, Ontario, Canada, 2004 JOT, UniPen, W3C Ink Markup Language, electronic ink formats. Mentions Pen computing APIs: IBM CrossPad API and Microsoft Tablet PC API. IBM C++ Ink Manager SDK, 2001. Document model of electronic ink, Walker Pattern. [YaoP04a] Yao, Paul "Add Support for Digital Ink to your Windows Applications", MSDN Magazine, 2004 Tutorial article on electronic ink/stroke collection in Windows Tablet PC APIs. Lists components for handwriting/gesture recognition: Input Panel Keyboard, Input Panel Writing Pad, Input Panel Character Pad, Pen Input Panel (TIP Tablet Input Panel: compare PenWindows), InkEdit control, Ink-Enabled Window. [YfantidisG05a] Yfantidis, Georgios "Blind Interaction Technique for Touchscreens: "A full keyboard under the fingertip"", M.Sc Thesis, Univ. of Tampere, Finland, April 2005 Single-finger touchscreen digitizer gesture/writing input for the blind: Gesture Driven Software Button GDSB -- pop-up pie menus (marking menu) with characters, tap and slide (press and hold) gesture on a set of pie menus combined with adaptive press-and-hold/dwell time to transition to a further menu to get a full character set: dwell time is adaptive/changeable. Some gesture input combined with key input on keyboard (left and right arrow keys). EdgeWrite and unistroke constrained handwriting. Note: pie menu and GUI user interface need not be visible, as this is an input technique for the blind. Extensive review of earlier work on blind input on touchscreens, in particular for mobile devices such as Palm PDAs and cell phones: Unistroke, Unigesture, touchscreen typing and constrained handwriting, auditory feedback, keyboard/menu layouts using frequency of characters (such as on IBM Simon). No reference to screenreaders such as VoiceOver or JAWS, nor to Milekic. [YogaratnamK04a] Yogaratnam, Kumanan "Method and Apparatus for Operating a Virtual Keyboard", United States Patent 6,677,933, January 13, 2004 Virtual keyboard component, implemented in Java environment. Software/virtual keyboard with semitransparent character displays over application image. [ZimmermanJ04a] Zimmerman, John and Martino, Jacquely Annette "Touch-Screen Image Scrolling System and Method", United States Patent 6,690,387, February 10, 2004 Dynamic scrolling of lists or information based on speed of scrolling: list continues scrolling (momentum/intertia), user can stop scrolling by pressing and holding, or select with a click while list is scrolling ("touch-marking" known in scrolled display technology) while scrolling is coasting to a stop. End of abstract: press-and-hold. [Zloter05] Zloter, Yitzhak and Shenholz, Gideon "Digitizer Pen", United States Patent 6,876,356 B2, April 5, 2005 Mechanical design of the Pegasus digital pen: mentions ultrasonic/acoustic digital transducers pen. Mentions "disappearing ink" in disclosures, as means to eliminate problem of permanent marks from ink that is required for human reading of marks while writing



biblio03 (Prev page)

Main Page







Main Page


[AbileahA06a] Abileah, Adiel; den Boer, Willem; and Green, Pat "Light Sensitive Display", United States Patent 7,053,967, May 30, 2006 Integrated (in-cell) Active Matrix LCD (AMLCD) display and optical touch sensors detecting ambient light. Can detect both less light (darkened) and more light (brighter). Detect finger, or light from light-emitting stylus (light in tip): multi-touch? Integrated TFT infra-red (not visible light) optical sensors (in-cell) in AMLCD display, used as touchscreen sensors. 1.1mm cover glass to absorb finger pressure. Detect fingers from ambient light, or lighted stylus ("light pen"): uses cone of light from stylus to determine Z height from area. Center of gravity interpolation to compute fine position. Shows multi-touch / multi-finger detection. [AbileahA07a] Abileah, Adiel and Green, Pat "Optical Sensors Embedded within AMLCD Panel: Design and Applications", Proc. EDT 2007, San Diego, CA, August 4, 2007 Integrated TFT infra-red (not visible light) optical sensors (in-cell) in AMLCD display, used as touchscreen sensors. 1.1mm cover glass to absorb finger pressure. Detect fingers from ambient light, or lighted stylus ("light pen"): uses cone of light from stylus to determine Z height from area. Center of gravity interpolation to compute fine position. Shows multi-touch / multi-finger detection. Cites to earlier 2003/2004 papers. [AckermanD07a] Ackerman, Dan "Asus R2H Ultramobile PC", reviews.cnet.com, Jan 29, 2007 Asus R2H UMPC: Ultra-mobile PC (Microsoft term) compared with OQO and Sony VAIO UX Premium, UMPC with no tablet or handwriting input, and not physical keyboard (on-screen keyboard only) [AckermanD07b] Ackerman, Dan "Band of the Hand: UMPCs reviewed", crave.cnet.com Sony VAIO UX390 UMPC: Note: runs Windows Vista, with Tablet PC software expecting proximity sense, but device has a resistive-film touchscreen digitizer: Shake / Tablet PC Input Panel gesture thus does not work. Lack of proximity cursor makes poor registration accuracy of touchscreen more of a problem. OQO Model 2: UMPC running Tablet PC. Comments that for UMPC small tablet PC, a 1.5" touch-pad is a better pointing device than a touchscreen or digitizer. [Adesso06] Addeso Inc. "Cyberpad Digital Notepad and Graphics Tablet", www.adesso.com Digitizer built into clipboard-style device, capture handwriting notes with real ink as electronic ink, notes can be uploaded to computer at later time. Compare with Anoto. mentions riteMail, handwriting



recognition with riteShape and riteScript for shape recognition (rectangle, circles, etc.) Automatic data entry using paper forms. inkNote, note-taking with electronic ink, "intelligent page break" (compare to Raamsdonk?) [AgarawalaA06a] Agarawala, Anand and Balakrishnan, Ravin "Keepin' it real: Pushing the Desktop Metaphor with Physics, Piles and the Pen", Proc of CHI 2006, pp 1283-1292 Touch-sensitive (or pen-digitizer) desktop GUI "BumpTop": simulating physics of actual cards, showing 3-D icons being placed in a three-dimensional space. Limited use of gestures: Lasso, which then brings up piechart menu. Pressure/tip-force on stylus selects different functionality in operations. Lasso-and-cross: drag through a "widget" which affects what operation is performed when stylus is lifted. Piles of simulated papers and cards: compare with Freestyle at Wang. Flick gesture is velocity-depending: faster flick object moves faster, with simulated acceleration/decelleration friction. Paper bumps (and bounces?) like physics? Commercial product: www.bumptop.com [AgarwalA07a] Agarwal, Ankur; Izadi, Shahram; Chandraker, Manmohan; and Blake, Andrew "High Precision Multi-touch Sensing on Surfaces using Overhead Cameras", Proc 2nd Annual IEEE Int'. Workshop on Horizontal Interactive Human-Computer Systems, 2007 Optical multi-touch tabletop using stereo cameras above surface. Uses stereo perspective of multiple points on fingertip to determine touching, simple disparity stereo calculation not fine enough. [AgrawalSC06] Agrawal, Subhash C.; Wimer, Scott M.; Young, Jonathan H. "Method and System for Detecting Intrusive Anomalous Use of a Software System using Multiple Detection Algorithms", United States Patent Application 2006/0085854, April 20, 2006 Cylant detection mechanism for software infection, combining multiple examination methods/models in the analysis. Describes the system call interceptor and "clicks" as datums. Previous Munson work used arbitrary code execution points, rather than system calls. [AhoAV07a] Aho, Alfred V; Lam, Monica S.; Sethi, Ravi; and Ullman,, Jeffery D. "Compilers: Principles, Techniques, and Tools", Addison-Wesley, 2007 2007 version of the "Dragon Book" by Aho on compiler design. Includes sections on parallelism, locality, interprocedural analysis. Nothing on Lexx and Yacc. [AlvaradoC07a] Alvarado, Christin and Davis, Randall "Sketch-Based interfaces: Techniques and Applications -Multi-Domain Sketch Understanding", Presentation slides, SIGGRAPH 2007 domain-flexible recognition systems for sketch recognition: recognize components such as line, vertex (high curvature, low speed motion), ellipse, arc, polyline via a primitive recognizer/fragmenter (parse), match to shape descriptions for shapes of domain (Electronics, architecture, etc.) using constraint-based pattern matching. [Amazon07] Amazon.com "Kindle: Amazon's New Wireless Reading Device", www.amazon.com, November 10, 2007 Electronic book using e-ink display, no stylus or digitizer, small keyboard buttons only [AmesB06] Ames, Ben "Palm pays Xerox $22.5 million in Patent suit", MacWorld, June 28, 2006. Available at www.macworld.com Unistroke lawsuit between Xerox and Palm / US Robotics / 3Com / ACCESS Co. / PalmSource [AnalogDevices07a] Analog Devices, Inc. "AN-929 Application Note: Tuning the AD714X for CapTouch Applications", Analog Devices, www.analog.com, 2007 Controller chip for capacitive touchscreen sensors: tuning process for clamp and offset, bulk/stray capacitance, sensor sensitivity. Wiring for buttons, sliders, wheels (like original iPod), grid arrangement for matrix keypads. [AndersonR07] Anderson, Richard; Anderson, Ruth; Davis, Peter; Linnell, Natalie; Prince, Craig; Razmov, Valentin; and Video, Fred "Classroom Presenter: Enhancing Interactive Education with Digital Ink", IEEE Computer, September 2007, Vol 40 Number 9 pp. 56..60 Whiteboard system for use during lectures using Digital Ink (Microsoft term for Electronic Ink). Teacher and students can mark-up comments submitted by student in electronic form. Using note-tacking application from Microsoft on Tablet PC. See also Tront. [Answers06a] Answers.com "Technology: Graphics Tablet; 1988; Mouse", www.answers.com, fetched 2006 Reference to stylus versus puck for tablet: actual source is Computer Desktop Encyclopedia, (c) 1998 The Computer Language Co. Inc. See also topic "mouse - technology": cites mouse data as relative motion only, tablet is absolute. Scriptel digitizer in 1988 [AppleInsider06a] AppleInsider "Apple Applies for Touchscreen Gesture Patent / Inside Apples latest iPod touchscreen patent filing", AppleInsider, February , 2006 / September 30, 2005 Press articles on original Apple multi-touch gesture and touchscreen hardware patent application. [AppleInsider07a] AppleInsider "Touch sensor firm issues legal warning to Apple over iPhone", AppleInsider, February 20, 2007: available at forums.appleinsider.com Quantum Research threatens infringement patent lawsuit for Apple iPhone. Not clear whether Quantum's



UI patens, or multi-touch digitizer technology was involved. See file on Quantum. [AppleInsider07b] AppleInsider "Apple developing configurable multi-touch gesture dictionary", AppleInsider, August 2, 2007: available at forums.appleinsider.com Apple iPhone patent application for a dictionary of multi-touch gesture/chord input. Compare with chorded keyboard of Buxton? [ApplianceDesign06a] Philipp, Hal "Novel microwave oven has interactive display that uses durable glass touch controls", Appliance Design, February 2006, pp. 14-17 Quantum Research capacitive touchpad/buttons in Whirlpool Velos oven. Author is managing director of vendor. [ApplianceDesign06b] Levin, Mike "Tactile Turn: Multifunction rotary encoders increase design options", Appliance Design, February 2006, pp. 18-21 Rotary control with programmable haptic touch feedback variable for multiple functions: Barrier at end of turn, constant force continuous, detent, hill (increasing force until peak, then drop off), compound/combination effects such as detents of different force levels. Author is V.P. of vendor. [AssureTec06] AssureTec Systems, Inc. "AssureTec System: Innovating Technologies", www.assuretec.com Identification products: mentions Ted Kuklinski/Pencept/Penpad. [Atmel06a] Atmel / Quantum Research Group "QMatrix Technology White Paper", Quantum Research Group 2006 Digital charge-transfer / QT capacitive touchpads and capacitive touch surfaces, mutual capacitance. Patented 1999 for kitchen appliance controls (Velos Whirlpool). Narrow gate times (microsecond) treat water films as RC network, thus avoids false touches from water on surface. ITO film wires on curved surface. [AtwoodM06a] Atwood, Margaret "LongPen", Margaret Atwood, 2006 Collection of press references to LongPen: remote signing station for signing books, including live video and audio chat for interaction with fans who want their books signed by the author. (Margaret Atwood comments that physical travel is bad for the environment) [BachnerEF06a] Bachner, Edward F. III; Major, John; and Du, Xin "Wireless intelligent personal server", United States Patent 7,149,511, December 12, 2006 Remote file access/synchronization in a PDA: may transmit file updates/edit/changes/deltas only. Includes description of portfolio/Refalo-style PDA with two fold-up screens. [BalakrishnanR07a] Balakrishnan, Ravih "Curriculum Vitae for Ravi Balakrishnan", Ravih Balakrishnan, U. Toronto, 2007 Publication list and CV for Ravih Balakrishnan: note groupware interactive graphics, user interfaces for 3D displays. [BarrettG06] Barrett, Gary L. "Curriculum Vitae for Gary L. Barrett", Gary L. Barrett Touch International: reviewer for Agulnick patent [BarrettG07a] Barrett, Gary "A tough touch challenge", Veritas et Visus: Touch Panel, February 2007 ExtremeTouch: projected capacitive (versus surface capacitive) touch panel using resistive sheet. Surface capacitance: AC voltage applied four corners, ratiometric measurement of current drawn to finger from all four corners. Refers to capacitive grid as projected capacitive, higher accuracy than a resistive sheet, multi-touch. [BatesonM06a] Bateson, Melissa; Nettle, Daniel; and Roberts, Gilbert "Cues of being watched enhance cooperation in a real-world setting", Biology Letters, FirstCite Earl Online Publishing, 2006. Available at www.journals.royalsoc.ac.uk (partial copy) Psychological impact: pictures of eyes over an "honor system" payment box result in greater compliance with social standard to pay [BedersonB06a] Bederson, Ben; Karlson, Amy; and SanGiovanni, John "Interfaces Supporting One-Handed Use of Small Devices", www.cs.umd.edu/hcil/mobile, archived 2006 AppLens, LaunchTile, and other user interfaces for semantic zooming of information on very small touchscreen displays. Includes links to video presentations [BellD07] Bell, Donald "Apple iPod Touch - product review", reviews.cnet.com, September 14, 2007 Review of iPod Touch. Few mentions of multi-touch gestures from iPhone, other features are what are seen as significant. [BenkoH06] Benko, Hrvoje; wilson, Andrew D.; and Baudisch, Patrick "Precise Selection Techniques for MultiTouch Screens", Proc. CHI 2006, April 22-27, 2006, Montreal, Quebec, Canada Two-finger precise positioning by using slider or stretch gestures in multi-touch display. Contrasts with WIMP interfaces. Cites to Buxton early work on two-handed input. Simulates proximity sensing by detecting area of contact (Simulated Pressing SimPress) of fingertip. Dual-finger midpoint to generate an offset cursor, so that finger does not obscure display. Dual finger stretch, one finger points, the other zooms. Also: video file of demonstration. [BennettD07] Bennet, Drake "Patently Obvious", Boston Globe, Sunday, May 6, 2007, page E1



History of standards for granting of patents, refers to "Hindsight bias" on obviousness [BergquistJ07a] Bergquist, Johan "Display circuit with optical sensor", United States Patent 7,184,009, February 27, 2007 Optical phototransistor sensors integrated into LCD display as touchscreen, addressing lines shared between pixels and optical sensors. Compare to Abileah (2002 Filing) [Berrnier07] Berrnier, Marc "Adventures in Cybersound: Vannevar Bush", Available at www.acmi.net.au Memex *did* represent Hyptertext, but description is not clear. Voder device for speech recognition (concept) [BesacierG07a] Besacier, Guillaume; Rey, Gaetan; Najim, Marianne; Buisine, Stephanie; Vernier, Frederic "Paper Metaphor for Tabletop Interaction Design", Proc. HCII Human Computer Integrated Interfaces, 2007 Multi-touch multi-user collaborative desktop digitizer user interface: Mentions UI user-interface directmanipulation gestures of peeling and and piling of simulated documents -- peeling by bouncing virtual document against an edge to show the virtual other side of the document; putting documents into virtual slots (like mail slots) to send them (via email, to a copier, etc.), tearing/stretching and regeneration to copy. Compare to kinetic scrolling? [Blickenstorfer07a] Blickenstorfer, Conrad H. "Dell Latitude XT: Dell finally release a Tablet PC convertible", www.ruggedpcreview.com/3_slates_dell_latitude_xt.html, December 2007 Announcement of Dell Tablet PC using transparent digitizer tablet component from N-trig [Blickenstorfer07b] Blickenstorfer, Conrad H. "Motion leapfrogs competition again", www.ruggedpcreview.com, March 26, 2007 Motion Computing LE1700 tablet PC using electrostatic WriteTouch N-Trig digitizer instead of Wacom electromagnetic tablet: resistive touchscreen and electrostatic stylus digitizer combined [BloombergD06a] Bloomberg, Dan "Leptonica: Rasterop (a.k.a. Bitblt)", www.leptonica.com, November 23, 2006 Overview of rasterop/raster-op/bitblt operations for compositing/moving/combining graphics images, cites to BLT block transfer instruction on PDP-10. Includes min/max, affine transforms shear/rotation, scaling, transation in rasterop operations. See also other leptonica documentation. [BostonGlobe06] Boston Globe "Business Notebook: Brighter outlook", The Boston Globe, Wednesday, October 4, 2006 Page E2 Nintendo DS hand-held game machine with digitizing tablet [BostonGlobe07a] Boston Globe (from Bloomberg News) "Microsoft get $1.5b verdict tossed: US judge negates Alcatel-Lucent win in MP3 patent case", The Boston Globe, Tuesday August 7, 2007, page D2 Lucent/Microsoft patent dispute concerning MP3 patent licensing. Court rules after jury verdict that Microsoft had a license due to a chain of earlier corporate acquisitions by various parties (Fraunhofer Institute). [BostonGlobe07b] Boston Globe (from Associated Press) "Amazon.com launches its own e-book reader", The Boston Globe, Tuesday, November 20, 2007 Page C2 E-ink electronic ink e-book product Kindle. Allows mark-up, but no notation about digitizer or stylus. [BoulletM06] Boullet, Marc "Mozilla Firefox Extensions: All-in-One Gesture Extension", perso.wanadoo.fr/marc.boullet Mouse gesture plug-in for Mozilla Firefox web browser. Includes additional mouse extensions using alternative mouse buttons, mouse scrollwheel. Subsumes Mouse Gestures by Optimoz, Rocker navigation, Autoscrolling. Allows for mouse-gesture customization -- gesture macro definitions? Scrollbar Anywhere: hold right mouse button invokes scrolling anywhere (compare with mouse wheel?) [BreedDS06a] Breed, David S. "Interactive Vehicle Display System", United States Patent 7,126,583, October 24, 2006 Touchscreen user interface in automobile, functions are enabled when user selects a functions group using the touchscreen. Compare to Apple swipe gesture? / unlock [BrewsterR07a] Brewster, Hon. Rudi M. "Order granting summary adjudication of no infringement of U.S. Patent No. 5,347,295", United States District Court, Southern District of California, Civil Action No 02CV2060-B(CAB), March 7, 2007 Patent ruling that use of gestures on a touchscreen is not covered by patent on gestures in an operating system when gestures are made using a digitizer. Distinguishes between touchscreen and digitizer for purposes of the patent. [BrewsterR07b] Brewster, Hon. Rudi M. "Order granting-in-part and denying-in-part motion for summary judgment of no indirect infringement of U.S. Patent No. 5,347,295", United States District Court, Southern District of California, Civil Action No 02CV2060-B(CAB), March 21, 2007 Patent ruling that use of gestures on a touchscreen is not covered by patent on gestures in an operating system when gestures are made using a digitizer. Distinguishes between touchscreen and digitizer for purposes of the patent. [Bricklin07a] Bricklin, Dan "Gesture, the iPhone, and Standards: A Developer's Questions",



www.bricklin.com/gestures.htm, 24 October 2007 Essay by Dan Bricklin on standardization of gestures in pen-computing. Describes special considerations on shape of Tap gesture. Mentions accessibility problems of iPhone and other systems to visually impaired. Flick gestures in PenPoint and on iPhone -- no mention of Microsoft Tablet PC flick gestures. Intellectual property/patent positions on gesture and multi-touch input, including speculation about Bill Buxton and Microsoft on "Multi-touch systems I have known and loved" position on prior art. [BrouhonP07a] Brouhon, Patrick "Hybrid Sensing Techniques for Position Determination", United States Patent 7,269,531, September 11, 2007 Optical pattern tablet/stylus. Position is intermittently determined from a glyph pattern on a surface, between times by integrating/tracking relative motion. (Glyph bed may be obscured by human-readable marks or overprinting.) Compare with Anoto? Cites to Silverbrook, and to correspondence from Sekendur. [BrownLM07a] Brown, Lorna Margaret "Tactons: Structured Vibrotactile Messages for Non-Visual Information Display", Ph.D. Dissertation, Dept. of Comp. Sci, U. Glasgow, Scotland, April 2007 Study of optimal recognizability of vibrotactile icons (tactons), tactile equivalent of Earcons in audio. Recognizable to users by three factors: rhythm, spatial location, and roughness+intensity. Hardware is TACTAID VBW32 haptic actuator (vibrator) or EAI C2 Tactor, attached to fingertip. [BuisineS07a] Buisine, Stephanie; Besacier, Guillaume; Najm, Marianne; Aoussat, Ameziane; and Vernier, Frederic "Computer-Supported Creativity: Evaluation of a Tabletop Mind-Map Application", Springer Verlag, August 24, 2007 (abstract only: see Besacier also) Multi-touch multi-user collaborative desktop digitizer user interface: Mentions UI user-interface directmanipulation gestures of peeling and and piling of simulated documents -- peeling by bouncing virtual document against an edge to show the virtual other side of the document; putting documents into virtual slots (like mail slots) to send them (via email, to a copier, etc.), tearing/stretching and regeneration to copy. Compare to kinetic scrolling? [BusinessWeek07a] Business Week "Apple's Magic Touch Screen", Business Week, March 15, 2007 Apple multi-touch/multipoint touch screen patent application based on Westerman/Fingerworks technology acquired by Apple. Manufacturer is Balda/TPK Holding/Optera/Magna International. [BusinessWire07] Business Wire / Gale Group "N-trig Opens Taiwan Office", BNET.com, August 20, 2007 N-trig multi-touch transparent electrostatic tablet digitizer, producing office started in Taiwan [Buxton06] Buxton, William "Directory of Sources for Input Technologies/Manufacturers Addresses", www.billbuxton.com Adaptive/Assistive technologies, character recognition, Pens, Digitizing Tablets, Eye and Head Movement Trackers, Joysticks, Lightpens, Mice, Stylus Devices, Touch Screen, Haptic Force Feedback devices, etc. [Buxton07] Buxton, William "Multi-Touch Systems that I have Known and Loved", www.billbuxton.com Apparently prompted by recent developments, such as the iPhone, with multi-touch capabilities. Bill Buxton had done earlier work on two-handed input (see other references). Mentions that touch-screens are not accessible to the blind or visually impaired. (NOTE: Not so: consider VoiceOver accessibility features of all iPhones, iDevices.) Finger on touchscreen obscures more of the view than would a stylus. Cites N-Key rollover in typewriters at multi-touch. First multi-touch system by Nimish Metha using optical digitizer with frosted-glass surface in 1982, "Flexible Machine Interface". Cites 1983 Soft Machines at Bell Labs, but not earlier 1973(?) work by Kaplow, Buxton's own multi-touch tablet 1985, Wacom 1992 tablet that could use both a puck and a stylus. [BuyensK07a] Buyens, Koen; Scandariato; Riccardo; Joosen, Wouter "Process Activities Supporting Security Principles", Proc. COMPSAC 2007 31st Ann. Int'l. Computer Software and Applications Conference, July 24..27, 2007, pp. 281..292 Essay on application of computer security principles in software design methodologies. [CIC06] CIC Corporation "CIC Company Information", www.cic.com Company information on CIC/Communications Intelligence Corporation: mentions sale of CIC Jot handwriting recognition to PalmSource in 2002 as Graffiti-2 [CardSK06a] Card, Stuart Kent; Gossweiler, Richard Carl III; Woodruff, Allison Gyle; and Mackinlay, Jock Douglas "Navigation Methods, Systems, and Computer Program Products for Virtual Three-Dimensional Books", United States Patent 7,139,982, November 21, 2006 3D (2 1/2 D) display of a virtual book, where user can flip and scroll pages that appear to be stacked left and right. Page images include virtual tabs, like a dictionary, and scrolling to flip pages. Also shows virtual bookcases (Microsoft Bob?) (Wang Freestyle on pages?) [ChanK07a] Chan, Kee-Yip Kitagawa, Kelly; and Manduchi, Roberto "Experiments in Planar Haptic Contour Exploration", Proc. WHC 07, 2nd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems, pp. 604-606 2-D haptic Wingman Mouse from Logitech (out of production), using Immersion TouchSense API to simulate edges of polygon: notes that adding sound feedback helps. Difficult for users however to



recognize (form mental map) of the polygon's shape: should compare with teaching orientation to the newly blind at the Carroll Center? [ChannelRegister07a] Orlowski, Andrew "Psion: The last computer. Secrets of the Sony we never had", www.channelregister.co.uk/2007/06/26/psion_special, June 26, 2007 History of Psion pocket computer (Series 5) with touchscreen and custom OS, circa 1994. [ChaudhriI06] Chaudhri, Imruan; Christie, Greg; Ording, Bas "Virtual Input Device Placement on a Touch Screen User Interface", United States Patent Application, 2006/0033724 A1, February 16, 2006 Simulated Devices displayed on a multi-touch touch screen: keyboard, video, folders, etc. Compare with Kaplow, Buxton, or Wang Freestyle portable. Sound mixer / music mixer using touchscreen: Apple Gesture recognition for zoom, pan, rotate, scroll, page turn. Confused in some press reports with FTIR / Frustrated Internal Reflection multi-touch input tablet, which came out about the same time from a different group [CherubiniM07a] Cherubini, Mauro: Venolia, Gina; and Ko, Andrew K. "Let's Go to the Whiteboard: How and Why Software Developers Use Drawings", Proc. CHI 2007, May 2007, pp. 557-566 Not sketch recognition, but study of the human communication in sketches in various technical subjects: one point is that sketches are beneficial because they are abstract, semantics not precise, leads to additional ideas. Sketches not saved as documentation / documentation lost. implementations mentioned include "Tablet PC" (Microsoft) [ChesnutC06a] Chesnut, Casey "tabletStrator", www.mperfect.net/tabletStrator Prototype of gesture commands for controlling video player, and electronic ink annotation on video, Windows Media Player on Tablet PC. Gestures include square, left and right semi-circle mentions electronic ink to speech translation [ChiuYC07] Chiu, Yen-Chang; Yeh, Ting-Hao; Chien, Yung-Lieh; and Lin, Shih-Chi "Capacitive Touchpad Integrated with Key and Handwriting Functions", United States Patent 7,274,353, September 25, 2007 Capacitive/electrostatic touchpad with zone-based handwriting/gesture recognition on a (simulated) keypad, mouse control. Compare with Refalo and calculator? [ClarityConsulting07a] Clairty Consulting "There's something about WPF: iPhone like unlock screen", blogs.calrity.con, 2007 Posting about implementation of kinetic scrolling / flick list (a.k.a. touch panning) [CockburnA06a] Cockburn, Andy; Karlson, Amy, Bederson, Benjamin B. "A Review of Focus and Context Interfaces", University of Maryland technical report 2006-06 Earlier version of 2008 ACM paper, same authors. [CravottaR07a] Cravotta, Robert "Recognizing Gestures: Blurring the Line Between Humans and Machines / Recognizing gestures: interface design beyond point-and-click", EDN Magazine, August 16, 2007, pp. 44-ff Review of gesture (hand-gesture) user interfaces in products. HP India GKB (gesture keyboard) for Devanagari and Tamil handwriting scripts/alphabets. Six-degrees-of-freedom Wacom digitizer tablet. Cites IBM Simon as forerunner of predictive keyboard entry in a touchscreen PDA to iPhone. Cites Buxton several places: multi-touch invented 25 years previously. [DRSTacticalSystems07] DRS Tactical Systems "Hammerhead rugged tablet computers", Product literature on Walkabout ruggedized tablet computers, available at www.drs-ts.com Ruggedized tablet PCs with integrated tablet and display: product line originally from Walkabout. Models include both a Pen Tablet PC version (proximity sensing, two-button stylus, electromagnetic?) and touchscreen version (no proximity sense) [DavidsonPL06a] Davidson, Philip L. and Han, Jefferson Y. "Synthesis and Control on Large Scale Multi-Touch Sensing Display", Proc. 2006 Int'l. Conf. on New Interfaces for Musical Expression (NIME06), June 4-8, 2006, Paris, France, pp. 216-219 Multi-touch sensors that have continuous homogeneous interaction surfaces for dynamic contextualization (i.e. arbitrary widgets) and bi-manual complex gestures. Cites to Lemur, JazzMutant, Westerman/Fingerworks and MTC Express for multi-touch touchscreens or digitizers. [DavisR07a] Davis, Randall "Magic Paper: Sketch-Understanding Research", IEEE Computer, September 2007, pp. 34-41 Sketch recognition: not just prettying, but draw objects (e.g. wheels, attachment points) and animate them automatically. Speaks of fragile properties e.g. verticality of a line: slight tilt makes it no longer be perceived as having verticality. [DavisR07c] Davis, Richard and Simpson, Rosemary "2007 Invited Workshop on Pen Computing", Brown University, RI, March 26-28, 2007: pen.cs.brown.edu/MSpccWorkshopReport.doc Brown University pen-centric computing lab sponsored by Microsoft, Andy Van Damm: Summary of pencentric as pen is more than a substitute for the mouse (focusing on smooth drawing / electronic ink). Cites PenPoint, Apple Newton, Pen Windows, Wang Freestyle without commercial success, mentions Palm Pilot and Anoto Pen (patterned paper). Attributes failure to limited functionality for price. Cites to Tablet PC, games such as Line Rider.



[DavisRC07b] Davis, Richard C.; Saponas, T. Scott; Shilman, Michael; and Landay, James A. "SketchWizard: Wizard of Oz Prototyping of Pen-Based User Interfaces", Proc. UIST '07, Oct 7-10, Newport RI (Author's version of published paper) Defines "Wizard of Oz" system. Prototype for pen-based/mark-based/sketch-based user interface. Paper prototyping of pen-based system. [Decuma07] Decuma "Decuma: Capturing your hand", www.oresundit.com Press information on Decuma handwriting character recognition for Japanese and Chinese, using Geometrical Invariant Technology, GIT See note on Hermansson. Character recognition for Tablet PCs, PDAs, mobile phones [Dell07a] Dell Inc. "Dell Latitude XT Tablet", www.dell.com Lap top / slate computer with dual-mode (electrostatic/capacitive) digitizer: for stylus/pen or finger touch, with proximity sensing. Press reports mention need to add touch-input GUI to Windows. [DesturaGJa06a] Destura, Galileo J. a.; Hector, Jason R.; and Roosendaal, Sander J. "Touch-input Active Matrix Display Device", United States Patent Application, November 23, 2006 Active matrix LCD integrated with same active matrix TFT switching circuit for independent touchscreen pixels: may be applied to capacitive or optical sensing on each pixel. References. Dwell/multi-touch. [DieWelt07] Die Welt "Apple hat Probleme mit iPhone-Lieferanten", Die Welt / Online, June 26, 2007. Available at www.welt.de Apple multi-touch iPhone: touchscreen produced by Balda in Germany, production transferred to TPK in Taiwan. [Divecha06] Divecha, Vic "The World of Smartboards, Symposiums is about to change", mblog.lib.umich.edu/~rdivecha Commentary on user-interface multi-touch tablet digitizer using FTIR / frustrated internal reflection: Video demonstration. Some comments are skeptical of utility of multi-touch input [DoveAP07a] Dove, Andrew P. and Burr, Kent A. "Wearable computer in a process control environment", United States Patent 7,230,582, June 12, 2007 Wearable computer with video camera. Process control application. [DresevicB06a] Dresevic, Boding and Kallay, Michael "Dynamic Rendering of Ink Strokes with Transparency", United States Patent 7,091,963, August 15, 2005 Electronic ink is semi-transparent (translucent), like a high-lighter, darker where electronic ink mark crosses over another. Cites to JOT standard. [E-Ink06a] E-Ink Corporation "High Resolution Displays", Product Information: www.eink.com E-Ink electronic ink product: paper-like display at 170 pixels/inch, not a digitizing tablet. Mentions Sony Reader using E-Ink Imaging Film. [E-Ink06b] E-Ink Corporation "E-Paper de digitale krant op digital papier", Product Information: www.eink.com E-Ink electronic ink product: paper-like display at 170 pixels/inch, not a digitizing tablet. IBBT consortium: includes I-merge, Hypervision, Philips, De Tijd (newspaper) for electronic books. [E-Ink06c] E-Ink Corporation "Ink In Motion", Product Information: www.eink.com E-Ink electronic ink product: paper-like display, not a digitizing tablet. Can show dynamic graphics, use for signage. Hardware from Neolux, Midori Mark or Teraoka business partners. Later product information in file for color e-ink. File also includes AM-100 EPD Kit electronic paper display prototyping kit, Active matrix EPD prototype kit. [Economist07a] Economist Magazine "Computing Section: How Touching: "Haptic" technology is gradually bringing the neglected sense of touch into the digital realm", The Economist Magazine, March 10, 2007, special section Article on Haptics: cites Samsung touch-screen phone, SCH-W559; Karon MacLean, researcher at University of British Columbia; Vic Viegas, patents on haptic feedback at Immersion Inc., vibrotactile feedback and VibeTonz (hardware vibration device); Vincent Hayward Centre for Intelligent Machines at McGill, Gabriel Robles-De-La-Torre of International Society for Haptics; no mention of Bill Buxton [EderyYM06a] Edery, Yigal Mordechai; Vered, Nimrod Itzhak; and Kroll, David R. "Malicious mobile code runtime monitoring system and methods", United States Patent 7,058,822, June 6, 2006 Malware checking on downloaded code (.EXEs, JavaScript, Java applets, etc.) by firewall or filter putting the downloads on a separate machine, examining them or letting them run, then enforcing a security policy or action. Compare with deep packet inspection. [Egger06] Egger, Markus "Find New Meaning in Your Ink With Tablet PC APIs in Windows Vista", msdn Magazine, May 2006, pp 91-100 Microsoft article which used term ink analysis as replacement for gesture recognition or handwriting recognition: describes Windows APIs for ink, mentions lack of APIs for drawing recognition (sketch recognition) Some comments are skeptical of utility of multi-touch input [EidMA07] Eid, Mohamad A.; Mansour, Mohamed; El Saddi, Abdulmotaleb H.; Iglesias, Rosa "A haptic multimedia



handwriting learning system", Poster Session: Proc. Intl. workshop on Educational Multimedia and Multimedia Education; Augburg, Germany, 2007. pp. 103-108 Teaching handwriting in various languages with visual and audio feedback: haptic feedback refers to forces to guide the hand back to the correct path, but few details on hardware: sensable.com, mpbtechnologies.ca [ElektronikPraxis06a] Elektronic Praxis "Quantum QField Resistive Touchscreen Eine Schicht mit Struktur: ITO auf der Panel-Rückseite lässt Akkus länger leben", Elektronik Praxis, 28 September, 2006 (in German) description of fabrication of touchscreens using ITO / indium tin oxide for resistive touchscreens with two layers: one resistive, one conductive. [EloTouch06a] Elo Touchsystems "Acoustic Pulse Recognition(APR): Breakthrough New Touch Technology from Elo TouchSystems", www.elotouch.com, 2006 Touchscreen using acoustic pulses: successor to force/pressure sensitive touch screen by EloGraphics. No coating required on glass like resistive touchscreens. Similar to Surface Acoustic Wave (SAW) touchscreen, but uses sonic profile of acoustic pulse with frequencies that disperse at different speeds. [EloTouch07a] Elo Touchsystems "CarrollTouch Infrared Touchscreens", www.elotouch.com, 2007 Datasheet for optical/infrared touchscreen. Emitters and detectors are building into a low-profile frame that fits around the top surface of the display. [FairmanCM06] Fairman, Christopher M. "(The principal obscene word of the English language)", Monograph, March 2006, Center for Interdisciplinary Law and Policy Studies, Working Paper No. 39, Ohio State University, Moritz College of Law Scholarly investigation of F**K, the principal obscene word of the English language. Also well known for being rejected by law review on the basis of the title alone. See also Stone, 1956 [FinePointInnovations06a] FinePoint Innovations "Digitizers - How They Work", FinePoint Innovations, www.finepointinnovations.com, 2006 FinePoint electromagnetic digitizer "active" tablet, contrasted with "passive" resistive digitizer touchscreen (with spacer dots) [Fingerworks06] Fingerworks Inc. "iGesture Pad Product Information", www.fingerworks.com Multi-touch Touchpad digitizer tablet with built-in gesture recognition: one model shows a numeric touchpad that appears similar to the Casio PF-8000 calculator. Gestures are relative motions of more than one fingertip, not shapes: MyGesture editor for defining gestures, and setting acceleration thresholds. Devices provided simulated keyboard output, thus could be used to replace keyboards on various systems (compare with Penpad 320?) TouchStream LP appears similar to Refalo (?). File also covers FingerBoard split-hand multi-touch keyboard. In Mouse Emulation, two finger gesture for point, three-finger tap gesture for double-click, four-finger scroll gesture. [FleischmannM07a] Fleischmann, Monika; Strauss, Wolfgang; Li, Yinlin; Groenegress, Christoph "Gesture-based input device for a user interface of a computer", United States Patent 7,312,788 B2, December 25, 2007 Hardware patent on electrostatic/capacitive non-contact digitizer, similar in principle to a theremin, to sense in-air gestures. Requires use to stand on on a grounding pad or reference pad. Hardware for iPoint device from Fraunhofer Institut? Includes one claim relating to detecting termination of a gesture by not moving. [FlipStart07] www.flipstart.com "Super Compact PC for the Mobile Professional", www.flipstart.com Product information and press information on FlipStart, competitor to OQO UMPC / Tablet PC. Uses small touch-pad rather than a tablet digitizer or touch screen. Also small joystick and thumb wheel. [FlippinBits06a] Flippinbits.com "TWiki.Main.DevelopmentToolsList", www.flippinbits.com List of compilers, IDS's, languages, source code, libraries, other software tools for Palm OS Developers [FlyWorld07a] flyworld Inc. "FLY fusion (TM) Pentop computer: The Ultimate High-Speed homework System", www.flyworld.com, 2007 Product information: Anoto optical digitizer pen with special paper, software for MP3 player, games, homework application. Electronic ink notetaking, handwriting conversion to digital text via character recognition, FLY World application. [FlyWorld07b] flyworld Inc. "FLY fusion (TM) Pentop computer: User Manual", www.flyworld.com, 2007 Anoto optical digitizer pen with special paper, special application. Control Panel is special printed menu/panel with Anoto code recognized by application firmware in pen: notes on/of, language, games, math, music, mp3 player control (including scroll wheel). Printed keyboard with anoto pattern; arrow/menu scroll buttons. Write copyright symbol (gesture) to define custom menus: on/off in rectangle (gestures), Help in rectangle. Handwriting recognition for block characters, uses language context for upper/lower case assistance. Scheduler, note-taking (w/wo sketch recogniton/handwriting recognition). Hand-drawn keyboard, drums. Recogizes 2D two dimensional mathematical writing. Built-in dictionary translator (e.g. EnglishSpanish) with handwriting recognition. [FlyWorld07c] flyworld Inc. "FLY fusion (TM) Pentop computer: Fly World Application User Manual Version 1.0 for



Windows XP and Vista", www.flyworld.com, 2007 Anoto optical digitizer pen with special paper, special application. Details on pre-printed patterned notebooks for recording and indexing notes (with recorded audio) to particular pages. On-line store for downloading additional applications. [ForsbergA07a] Forsberg, Andrew; Holden, Loring; Miller, Timothy; and Zeleznik, Robert "The Music Notepad for Tablet PC", graphics.cs.brown.edu, fetched 2009 Port of Music Notepad handwriting recognition editor for musical notation to Tablet PC with funding form Microsoft to be Tablet PC PowerToy "Tablet PC Music Composition Tool" [Fraunhofer06] Fraunhofer Institut "PointScreen: Wie von Zauberhand", Fraunhofer Magazin, January 2006, pp 16-18 (in German) article on Gesture-based wall mounted digitizer (low resolution), which senses hand motions and gestures while standing near a projection screen. Non-contact whiteboard. Sensing is electromagnetic/electrostatic, similar to the Theremin musical instrument [FreehandSystems06] Freehand Systems Inc. "MusicPad Pro Plus product information", www.freehandsystems.com Electronic EBook to display musical scores, page turning controlled by foot paddle, allows handwriting stylus to annotate music. Marlin Eller. [FriedlandG07a] Friedland, Gerlad; Hürst, Wolfgang; and Knipping, Lars "Educational Multimedia Systems: The Past, the Present, and a Glimpse into the Future", Proc. EMME '07, Sep. 28, 2007, Augburg, Germany History of distance learning/correspondence courses, going back to Isaac Pitman (shorthand) in 1840. Retail pen computers and Tablet PC started distance learning / whiteboard market in 2002. Chalkboard PowerPad in 1980s by Atari and Commodore C64. Note: press and hold: has citation to Momenta User's Reference Manual 1991. [FukuchiK06a] Fukuchi, Kentaro "Concurrent Manipulation of Multiple Components on Graphical User Interface", Ph.D. Thesis, Tokyo Institute of Technology, October 26, 2006 Multi-touch (or multi-pointing) user interface requires concurrent multiple input streams and multiple input focus for manipulation, unlike customary/traditional WIMP GUIs. Cites to Fitzmaurice for space multiplex (concurrent manipulation of objects) versus time multiplex (sequential manipulation of multiple objects). Favors absolute pointing (tablet) over relative pointing (mouse). Bulldozer manipulation -- pushing objects around, detect shape of hand (shapetape). Multi-touch prototypes include optical tracking of colored squares and 2D barcodes, electrostatic/capacitive sensing (SmartSkin) for moving multiple animation objects, multiple laser pointers. Potential field manipulation user interface (bulldozer), both repulsive and attractive. [FukuchiK07a] Fukuchi, Kentaro and Rekimoto, Ken "Marble market: bimanual interactive game with a body shape sensor", Proc. ICEC 07 6th Int'l. Conf. on Entertainment Computing, pp 374..380 Multi-touch/multi-user prototype game using SmartSkin tablet and bulldozer arm gestures to move/herd animated marbles into targets. [GawandeA07a] Gawande, Atul "The Power of Negative Thinking", New York Times, May 1, 2007 Political essay on medical problems in patient care at Walter Reed Army Hospital: essentially seems to say that care improved when staff focused on addressing the systemic failures rather than focusing on their successes. [GeaghanBO07a] Geaghan, B. "Remote Touch Simulation Systems and Methods", United States Patent 7,236,161, June 26, 2007 Probably testing of electrostatic tablets: Conductors placed mechanically on a capacitive digitizer so that touches can be simulated for calibration or testing. [Geekheim06] geekheim.de "Sacha's Pen Computing: Archive for the "Pen Computing" Category", www.geekheim.de Ink Gestures for JumpingMinds : includes single, double and triple flick-like gesture for underline, bold, capitalize: compare with GO PenPoint flick gestures. Left and right flick for indent and outdent, letter P for paragraph, etc. Pen-one biometric pen: includes fingerprint scanner in pen, not signature verification. EInk technology for ink display from IRex Technologies Photograph of NCR System 3130 tablet/pen computerising Windows 3.1 for Pen Computing and MS DOS Anoto Fly Pentop computer -- marketed as children's educational toy [GestureTek07a] Baard, Mark "Be as high-tech as a Tom Cruise character", The Boston Globe, December 24, 2007, p.E10 Hand-gesture recognition using infrared (?) sensors in a digitizer bar: commercial product. Described as touch-free gesture input utilizing computer, compared with Minority Report, pointing to a projector screen [Gorbunov06] Gorbunov, Artem and Voronezh, Roma "Infoscroller, or Scroll bar in the submission of content in the form of graphic symbols", Blog posting 31.10.2006, artgorbunov.ru Russian patent on scroll bar that has highlights for sections of text found by global search: compare to US



patent by Hill on same feature? [GordonGB07a] Gordon, Gary B. "Apparatus and Method for Controlling a Screen Pointer", United States Patent 7,167,162, January 23, 2007 Controller implements inertial/kinetic scrolling, generates simulated inertial (friction) momentum after finger lifted from touchpad, motion stopped by touching. [Gouy-Pailler07] Gouy-Pailler, Cedric; Zijp-Rouzier, Sophie; Vidal, Sylvie; and Chene, Denis "A Haptic Based Interface to Ease Visually Impaired Pupil's Inclusion in Geometry Lessons", Universal Access in Human-Computer Interaction: Applications and Services. Springer Verlag, Berlin/Heidelberg, 2007. ISBM 978-3-540-73282-2, pp 598-606 (abstract only) use of haptic (force-feedback?) device to enable blind/visually-impaired pupils to learn plane geometry. Suggested using gestures rather than keyboard for input. [Greene07a] Greene, Kate "Wall-Size Touch Screens: Multi-touch displays advance", Technology Review, March/April 2007, page 23 Jeff Han at NYU Courant Institute of Mathematical Sciences, FTIR / frustrated internal reflection multitouch touch screen. Quotes Bill Buxton of Microsoft Research. Company "Perceptive Pixel". [Greene07b] Greene, Kate "A Better Touch Screen", Technology Review, November/December 2007, page 15 Microsoft/ Mitsubishi rear-mounting touch screens on the back of a large PDA, with transparent images of user's fingers: simulated device only as a research prototype [Greene07c] Greene, Kate "Touch Screens for Many Fingers", Technology Review, January 18, 2007 multi-touch digitizer comparison of Jeff Han / Perceptive Pixel (optical frustrated/total internal reflection), Mitsubishi DiamondTouch, and iPhone / Fingerworks [GridBoard06] GRidBoard: Rob's GRidBoard "Rob's GRiDBoard", www.pd.com/gbmenu.html Listing of GRiDPad and similar tablet computers available for sale: drivers, parts, manuals: listing as of Feb 2006 [Griml07] Griml, Guy "Startup has the future at its fingertips", HAARETZ on-line magazine, December 14, 2007. www.haaretz.com/hasen/spages/934096.html Press announcement of N-trig touch and stylus electrostatic digitizer, incorporated in Dell Tablet PC. Referred to generally as "multi-touch", not clear that capabilities for detecting two finger touches simultaneously is operational. Original company name for N-trig was "Game Along", founded by Meir Morag. [GrossmanT06a] Grossman, Tovi; Hinckley, Ken; Baudisch, Patrick; Agrawala, Maneesh; and Blakakrishnan, Ravin "Hover Widgets: Using the Tracking State to Extend the Capabilities of Pen-Operated Devices", Proc. CHI 2006, April 22-27, 2006, Montreal, Quebec, Canada. pp. 861..870 Hover widgets: perform gestures in hover/proximity to select command, then touch stylus to tablet to select target. Compare to Pen Windows "shake" gesture in hover (not cited). Refers to proximity sensing as "tracking state". Cites Tablet PC to Wacom electromagnetic digitizer. Cites to marking menus. [HallerM07a] Haller, Michael "Pen-based Interaction", SIGGRAPH '07 courses, 2007. A Tutorial notes. Finger-operated touch screens advantageous over mouse. 3D pen-based interfaces. 6DOF six degrees of freedom stylus digitizer from Terminal Display systems. Longer discussion of Anoto optical digitizer pen on visual digital desk, desk surface also patterned with Anoto pattern. Plawa.com Scrivo.1 optical mouse/pen (see file) [HammondTA07a] Hammond, Tracy Anne "LADDER: A Perceptually-based Language to Simplify Sketch Recognition User Interface Development", Ph.D. thesis, M.I.T., 2007 Constraint-based language for sketch recognition, both for editing and for recognizing symbols (not handwriting). Many references on sketch recognition running on Microsoft Tablet PC platform. [HanJ06a] Han, Jeff "Multi-Touch Interaction Research: Bi-manual Multi-point and Multi-user interactions on a graphical interaction surface", cat.nyu.edu/~jhan/ftirtouch/index.html, 2006 FTIR internal reflection tabletob: multi-touch, multi-user experimental actions. Force-senstivie/pressuresensitive (area of contact?). Also LED touch display using LEDs as direct sensors, 3D digitization video suit plus instrumentation of user's breathing. 3D volumetric display using projected light on suspended dust particles, fast (1000fps) eye tracker. Also animated napkin sketch, real-time replay of drawing (compare with Wang Freestyle and Anoto) [HavivY07a] Haviv, Yaron and Corem, Guy "Filtered Application-to-Application Communication", United States Patent 7,216,225, May 8, 2007 Direct communication between applications/processes without going through the operating system calls that would usually handle sockets and other connections: to bypass network firewall overhead. [HawkinsJC06a] Hawkins, Jeffrey Charles and Haitani, Robert Yuj "Method and apparatus for accessing a contacts database and telephone services", United States Patent 7,007,239, February 28, 2006 User interface on touchscreen phone with permanent icons for dial pad, speed dial, call history, contact book/address list. Names looked up by minimum uniqueness.



[HeiseMobil07] Heise Mobil "Ultra-Mobile PC: Die nächste Generation [Update]", Heise Mobil Web magazine: 15.03.2007, 03.03.2007 UMPC Tablet computers from Gigabyte, Samsung Q1 Ultra 1024x600 LCD, touchscreen. Mentions Windows XP Touchpack user interface [HellerM06a] Heller, Martin "UMPCs and the Nokia 770 Internet Tablet", byte.com March 27, 2006 Comparison of Nokia 770 internet webpad tablet with UMPC Ultra Mobile PC / Tablet PC from Microsoft. Nokia runs Linux, has limited handwriting recognition. Mentions dial keys modified thumb-type soft keyboard on UMPC. [HendersonDA07a] Henderson, Daniel A. "Method and apparatus for improved personal communication devices and systems", United States Patent 7,251,318, July 31, 2007 Smartphone/cellphone with additional messaging/communication features: paging, identity information. Cites to General Magic / Telescript in a personal communicator. Numerous smartphone references. Note: original CIP priority date 1994 [HermanssonP07a] Hermansson, Per-Olof "Recognition for Merlin (Swedish) and Gabelsberger (German) Shorthand", Mass Marketing Software AB, Horred, Sweden. www.massmarketing.se Reference to Decuma doing Japanese and Chinese handwriting character recognition with Anoto pen, Decuma then acquired by Canadian company [Hildreth06] Hildreth, Even and MacDougall, Francis "Multiple Camera Control System", United States Patent 7,058,204, June 6, 2006 GestureTek Inc.: Hardware for gesture input: Two video cameras to compare background, to detect hand for hand gesture input. Compare to "put-that-there" system from M.I.T., iPoint from Fraunhofer Institut. [Hildreth07] Hildreth, Even and MacDougall, Francis "Video-based Image Control System", United States Patent 7,227,526, June 5, 2007 GestureTek Inc.: Hardware for gesture input: Two video cameras to compare background, to detect hand for hand gesture input. Had "presence" zone for detecting when a person was standing in front of a large monitor, and a "gestures" zone for detecting positions of hands during gestures. [HinckleyK07a] Hinckley, Ken "InkSeine: Find your Stuff with Ink", Web posting, research.microsoft.com, 7/25/2009 Ink Seine: Search through electronic ink notes using background handwriting recognition, includes Video presentation. Compare with Dan Bricklin at Slate in 1992? Other papers refer to phrasing (chunking and phrasing? See IBM 1988) in multi-stroke selection gestures. [HinckleyK07b] Hinckley, Ken; Zhao, Shengdong; Sarin, Raman; Baudisch, Patrick; Cutrell, Ed; Shilman, Michael; and Tan, Desney "InkSeine: In Situ Search for Active Note Taking", Proc CHI 2007, Apri8l 28-May 3, 2007, San Jose, USA Ink Seine: Search through electronic ink notes using background handwriting recognition. Gestures include lasso, breadcrumb (pend up, stroke down); drag, tap; pigtail on empty space to bring up menu, lasso+pigtail. Compare with Dan Bricklin at Slate in 1992? Other papers refer to phrasing (chunking and phrasing? See IBM 1988) in multi-stroke selection gestures. [HinckleyK07c] Hinckley, Kennneth P.; Sinclair, Michalel J.; Pierce, Jeffrey Scott; and Horvitz, Eric J. "Method and apparatus using multiple sensors in a device with a display", United States Patent 7,289,102, October 30, 2007 Proximity/tilt sensor on smartphone/PDA to determine how user is holding device, tilt sensor to rotate display automatically, tilt scrolling. Cites to Rekimoto tilt operations on PDA. [HodgesS07a] Hodges, Steve; Izadi, Shahram; Butler, Alex; Rrustemi, Alan; and Buxton, Bill "ThinSight: Versatile multi-touch sensing for thin form-factor displays", Proc. UIST '07, pp. 259-268 Multi-touch optical touchscreen: mount array of infra-red transmitters and receivers (coarse) behind and LCD, detect reflection of infra-red reflective fingers and objects through LCD. Bi-cubic interpolation to interpolate to higher accuracy for "center" of a finger touchpoint. Cites to Rekimoto. [HornBL07a] Horn, Bruce L. "Computer System for Automatic Organization, Indexing and Viewing of Information from Multiple Sources", United States Patent 7,275.063, September 25, 2007 Displaying lists or tables of objects (e.g. .JPGs, .MPG3, etc.) automatically sorted by different attributes, and sub-trees (e.g. folders) can be expanded or contracted in the list. Refers to sticky paths for references to objects. [HornbyT06] Hornby, Tom "The Story Behind Apple's Newton", lowendmac.com, Feb. 7 2006 History of development of Apple Newton: mentions Amstrad Pen-Pad, PenPoint. Interesting tidbit: At 1993 CeBIT in Hannover, Newton managers saw Amstrad Pen-Pad 600, similar form factor to Newton at lower price. At 1993 CES, apple representers saw PenPoint tablets in audience, which stole the thunder of the Apple Newton demonstration. (This was before Macworld Boston on August 2, 1993) [HotellingS06a] Hotelling, Steve; Strickon, Joshua A.; Huppi, Brian; Chaudhri, Imran; Christie, Greg; Ording, Bas; Kerr, Dunan Robert; and Ive, Jonathan P. "Gesture for Touch Sensitive Input Devices", United States Patent Application 2006/0026536 A1, February 2, 2006



Multi-point multi-finger gestures on a multi-touch tablet: pan, zoom, etc. The examples are more like manipulations of simulated devices than markings: e.g. rotating a wheel. Zoom gesture is to touch two fingers to surface, and move them in or out. Bimanual gestures, two gestures at same time. Compare with Bill Buxton simulated devices, Tog video early 1990's, Wellner. [HotellingS06b] Hotelling, Steve; Kerr, Duncan Robert, Ording, Bas; Ive, Jonathan P.; Kennedy, Peter J.; Fadell, Anthony M; and Robbin, Jeffrey L. "Proximity Detector in Handheld Device", United States Patent Application 2006/0161871 A1, July 20, 2006 Performing user interface gesture actions when a finger is detected in proximity by a capacitive digitizer: [HsuC07a] Hsu, Cheng and Boonjing, Veera "Natural language interface using constrained intermediate dictionary of results", United States Patent 7,177,798, February 13, 2007 Natural language query for databases (search engines) using multiple language dictionaries [HugossonO06a] Hugosson, Ola and Ericson, Petter "Systems and Methods for Information Storage", United States Patent 7,072,529, July 4, 2006 Describes Anoto pattern for an optical "smart paper" digitizer, using modified virtual raster pattern marks: position determination using fourier analysis [HullJJ07a] Hull, Jonathan J.; Erol, Berna; Graham, Jamey; Ke, Qifa; Kishi, Hidenobu; Moraleda, Jorge; and Van Olst, Daniel G. "Paper-Based Augmented Reality", Proc. 17th Int'l. Conf. on Artificial Reality and Telexistence, 2007 PDA viewer for printed documents, uses nGram filters (both horizontal and vertical) on text patches, or similarity filters on images (versus database) to detect and generate clickable URLs to related information in the display. One application referred to as Clickable Paper. Compare with DataGlyphs? [Hullender07a] Hullender, Gregory and Gounares, Alexander "Ink Gestures", United States Patent 7,305,630 B2, December 4, 2007 Ink gestures recognized using Chebychev Polynomials and Bayes net. Refers to gestures including alphabetic characters, and automatic recognition of gesture shape in the midst of text input via handwriting. Refers to an ink gesture being included in a stroke: gesture is part of a larger stroke? In description, lists example gestures and their semantics: e.g. braces (discontinuous selection), circle-tap, flick (scroll), double-right-flick for scroll to end, double-left-flick, triple-up-flick, etc. Appears to be same basic set of gestures as Microsoft "Unimplemented Glyphs". Compare with GO/PenPoint flick and other gestures [IOGEAR07a] IOGEAR "IRISPen executive 6 User's guide", iogear.com hand-held optical scanner, includes OCR software for text and for handwriting recognition (static/off-line character recognition), bar-code recognition. IRISPen translator 6 for translating text in other languages, including Chinese. Includes speech synthesis for recognized text -- useful as accessibility technology? File contains additional references. See also IRIS file. [IPhoneNews07a] iPhone-news.eu "So funktioniert das iPhone TouchScreen", http://iphone-news.eu, July 4, 2007 Multi-touch touch screen on on iPhone consists of front-mounted capacitive/electrostatic touch panel in front of the display: shows additional protective layer in front of the capacitive touch panel. UI patent on circular wheel, simulated devices, two-finger gestures for zoom/relax, simulated on-screen keyboard. [IPhoneNews07b] www.iPhone-news.org "iPhone Touchscreen von Fingerworks", http://www.iphone-news.org, February 7, 2007 Cites Fingerworks as vendor for multi-touch capacitive digitizer used in Apple iPhone. Apple acquired Fingerworks in 2005: products iGesture Pad and iGesture NumPad. Founders were John Elias and Wayne Westerman, University of Delaware. [IRIS07a] I.R.I.S. "IRISPen executive 6 (Windows) User's Guide", I.R.I.S., 2007 Hand-held OCR scanner in (largish) pen format. Handprinting/Handwriting recognition for number/digits and upper-case/capital block letters. User can define subset (compare Pencept). See also IOGEAR file. Windows, Mac, Mini documentation also in file. [IRIS07b] I.R.I.S. "IRISPen translator 6 (Windows) User's Guide", I.R.I.S., 2007 Hand-held OCR scanner in (largish) pen format. Scans also right to left (left-handed, or Hebrew). Scanning multiple lines. See also IOGEAR file. [IRexTechnologies07] iRex Technologies BV "iLiad Electronic Reader ER 0100", iRex Technologies BV Electronic Book using e-Ink display technology with touchscreen digitizer and stylus. Uses Wacom penabled stylus, electromagnetic digitizer: see also Super Slim Pen. Mentions specifically use of writing to add annotation mark-up by multiple users. Stereo audio output. [IsaacsonBass06] Isaacson Bass, Portia "Portia Isaacson Bass, Ph.D. C.V.", basseconomics.com C.V. for Portia Isaacson Bass: mentions early reviews of PenPoint/PenWindows approximately 1991 on likely large business impact pen computing and electronic ink [Itablet06] ITablet "iTab - The Mac Laptop Evolved", www.itablet.theplaceforitall.com Modified Mac LapTop to be a tablet PC: touchscreen replaces mouse, on-screen simulated keyboard but no



handwriting recognition, no provision for it. Does not describe any handling for electronic ink data compared to mouse driver input. [IwamotoT06a] Iwamoto, Takayuki and Shinoda, Hiroyuki "Two-dimensional Scanning Tactile Display using Ultrasound Radiation Pressure", Proc. 14th Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Alexandria VA, March 25..26, 2006, pp. 57..61 Proposal and design report for ultrasound tactile display using acoustic radiation pressure. Display is octoganal array of 320 ultrasound transducers, does not do 3D three dimensional, single focal point (vertically projected above 2D array) about 1 mm. Protective glove for ultrasound on skin (?). [JainV06a] Jain, Vidit "Handwriting Recognition, Read-Ink Technologies", Vidit Jain CV non-parametric probabilistic approach using various machine learning techniques for online cursive handwriting recognition [JarrettRJ06a] Jarrett, Robeert J; Duncan, Richard J. and Rimas-Ribikauskas, Emily K. "Scrolling Web Pages using Direct Interaction", United States Patent Application Publication 2006/0048073 Dynamic/auto (kinetic?) scrolling with simulated inertia: for handwriting / electronic ink input, list is automatically scrolled to desired position for input [JenkacH06a] Jenkac, Hrvoje; Stockhammer, Thomas; Wen, Xu; and Abdel Samad, Wissam "Efficient video-ondemand services over mobile datacast channels", Journal of Zhejian University, China, 2006, Vol 7 No 5, pp 873884 Encoding of Video-on-Demand in DVB-h video television transmission systems, FLUTE Protocol [JoguetP07a] Joguet, Pascal and Largillier, Guillaume "Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen", United States Patent Application US 2007/0198926, August 23, 2007 (See also PCT /FR2005/00428) Multi-touch tablet (of any type: multi-contact bidirectional sensor matrix), software to implement sliders, keyboards, potentiometers, and other controls with visual feedback (visual return) of direct manipulation of controls for musical instruments with real-time control and input. Refers to point clouds for multi-touch capacitive touchpad. [JohnsonZ07a] Johnson, Zac "Microsoft Surface", Wikispaces Classroom posting, 2007, available at zacjohnson12.wikispaces.com Internal design of Microsoft Surface multi-touch multi-user tabletop, rear projection and rear infrared camera. Cites to "Microsoft surface diagram: how it all works". [JumpingMinds06] JumpingMiNDS Inc. "InkGestures", www.jumpingminds.com Ink Gestures for JumpingMinds: plug-in for Microsoft Word on Tablet PC. Gestures include single, double and triple flick-like gesture for underline, bold, capitalize: compare with GO PenPoint flick gestures. Left and right flick for indent and outdent, letter P for paragraph, etc. Support ink annotations, ink comments, selection of which gestures/commands will be included in the input. [KableRG07a] Kable, Robert G.; Kable, Adam T.; Heringer, Lawrence J; and Wilson, Brent B. "Pen Apparatus, System and Method of Assembly", United States Patent Application 2007/0195069, August 23, 2007 Tablet stylus design: uses spring coil also as antenna to reduce components. Resistive film digitizer also sensing finger touch or stylus. Includes discussion of long history of resistive film touchscreens and digitizers. [KaikurantaT06a] Kaikuranta, Terho "Method for Preventing Unintended Touch Pad Input due to Accidental Touching", United States Patent 6,984,137, January 10, 2006 Cell phone with touchpad, lock state where certain functions in UI user interface are disabled in software unless a particular software determination unlocks the UI [KaltenbrunnerM07a] Kaltenbrunner, Martin and Bencina, Ross "reacTIVision: A Computer-Vision Framework for Table-Based Tangible Interaction", Proc. ACM TEI '07, February 2007, Baton Rouge LA Rear-projection tabletop touchscreen with rear camera, detects (bottom side of) objects on surface as well as fingers, can recognize identity marks / fiduciary markings on object. d-Touch software (open source) used for recognizing special markings. [KeatonP07a] Keaton, Patricia and Chowdry, Amit K. Roy "Method and apparatus for apparatus for generating three-dimensional models from uncalibrated views", United States Patent 7,289,662, October 30, 2007 Compute three-dimensional model from multiple views at different angles (or object at different angles) from a single camera. Edge detection to find features for matching in different views. Emphasizes no need for registration. Compare with Chen 2010 and structure-through-motion, etc. [Keks06] Keks "Anleitung für den Zauberwürfel: Lösung für den Rubik's Cube", www.keks.de/wuerfel Solution for Rubik's cube puzzle [KellyJPJ07a] Kelly, John P.J. "Kelly Technology Group", www.kellytechnologygroup.com Expert Witness / Intellectual Property firm, Lucent/Microsoft patent case concerning Agulnick patent. John P.J. Kelly, Principal. [KentJ06a] Kent, Joel; Adler, Robert; and Adriani, Paul M. "Acoustic Touch Sensor with Laminated Substrate",



United States Patent 7,006,081, February 28, 2006 Acoustic digitizer/touchscreen using Love waves and Rayleigh-like waves, using polymer sheet between rigid layers. Uses phrase "opaque touchscreen" -- not transparent. [KimJE07a] Kim, Ji-Eun; Sunwoo, John; Son, Yong-Ki; Lee, Dong-Woo; and Cho, Ii-Yeon "A gestural input through finger writing on a textured pad", CHI '07 extended abstracts, 2007, pp. 2495-2500 Flexible wearable touchpad digitizer, textured pad causes vibrations when finger is used to draw gestures on the pad. Pad has sections (pads) with different physical frequency of texture, thus vibration sensor on wrist can detect which section user is moving finger on by frequency of vibration. Piezo electric sensor, unistroke (Graffiti) recognition for digits. Cites to 2002 and 2001 gesturepads and gesturewrist interfaces. [KisA06a] Kis, Attila "Tactile Sensing and Analogic Algorithms", Doctoral dissertation, Peter Pazmany Catholic University, Budapest, 2006 very fine-scale piezo-resistive tactile sensing elements (taxel) sensing not just force, but moments and angles of forces. Part of larger review of human touch sensing: stroking, rubbing, palpation (feel of something when pressed with fingers over its area). Visual imaging of surface: compare with Rekimoto 2002 figures on pressure/contact in multitouch tablet. Earlier effort used Tactilus commercial product. Compare with TacTex products (not cited), Jung? [Klotz06] Klotz, Dr. Karlhorst "Magische Tische", Spiegel Magazine, June 24, 2006. Available at www.spiegel.de 42 inch Diamond Touch / DiamondTouch table with digitizer and display from Mitsubishi in demonstration in a pub in Salzburg "MayDay Bar". Also mentions similar Stehtisch Gispen XS from Emilie Tromp in Holland, Philips Entertaible, Panasonic digitizer tablet table in Tokyo. Compare to Microsoft table digitizer with multiple touch input. [KrasnerJ07a] Krasner, Jeffrey "Washable computer gives doctors a clean slate", Boston Globe newspaper, March 5 2007, page E1 MedTab: Washable/waterproof Tablet PC slate computer by Emano Tec, sealed so that it can be disinfected for hospital use Wayne Bailey, Dr. John Halamka, Bob Caspe of Emano Tec, Newton, Massachusetts [KrausMG06a] Kraus, Mark Gerard and Nguyen, Vu Xuan "Method and system for detecting multiple touches on a touch-sensitive screen", United States Patent 7,023,427, April 4, 2006 Touchscreen (grid) method to detect when more than one touch, and warn user not to do multiple touches. [Krazit07a] Krazit, Tom "Palm cancels first-generation Foleo", C-net news.com, September 4, 2007: Available at http://news.com.com Announcement that Palm Foleo full-keyboard PDA by Jeff Hawkins is being discontinued: Linux-based Palm OS with handwriting recognition in the works [Krazit07b] Krazit, Tom "Rock bottom for Palm and Hawkins?", C-net news.com, September 4, 2007: Available at http://news.com.com Palm Foleo full-keyboard PDA by Jeff Hawkins is being discontinued: Linux-based Palm OS with handwriting recognition in the works. [KunteRS07] Kunte, R. Sanjeev and Samual, R. D. Sudhaker "A simple and efficient optical character recognition system for basic symbols in printed Kannada text", "Sadhana" (Indian publication), Volume 32 Part 5, October 2007, pp 521-533 Neural-net/adaptive OCR recognition system for machine-printed south-Indian languages: Kannada, Telegu. Claims 99% recognition rate, not clear whether this is per character or in context. [LargillierG07a] Largillier, Guillaume "Developing the First Commercial Product that Uses Multi-Touch Technology: JazzMutant 2004", SID Information Display, December 2007, pp. 14-18 JazzMutant/Stantum multi-touch touchscreen technology 2003, prevents masking/shadowing behavior by setting all other columns of matrix/grid to high impedance except for the one column being energized with capacitive/electrostatic or resistive signa. Compare to Kaplow? [LeeKB07a] Lee, KWang-Bok "Data Displaying Apparatus and Method", United States Patent 7,232,058, June 19, 2007 Pop-up menus for displaying sub-properties (layers) on a small touchscreen display [LengelingG06a] Lengeling, Gerhard and Ribinstion, Jonathan Jake "Touch-sensitive electronic apparatus for media applications and methods therefor", United States Patent Application 2006/0022956A1, February 2, 2006 Using multi-touch/multi-point touchscreen to display music controller device: virtual keyboard, sliders, knobs, etc. Compare with JazzMutant? [LevyDH07a] Levy, David H. "Fast typing system and method", United States Patent 7,175,438, Feburary 13, 2007 Touchscreen typing / spelling dictionary: look up words by first, last, and a middle character, to account for accidental/wrong touches while typing. [LiebermanK07a] Lieberman, Klony; Sharon, Yuval; Naimi, Eyal; Maor, Yaniv; Tsachi, Mattan; Levy, Amiram;



Arnon, Boas; and Turm, Amichai "Data Input Devices", United States Patent 7,242,388, July 10, 2007 Project display image of keyboard (or any other user interface: piano/music keyboard, etc.) on table (or any other surface can be turned into a touchscreen/digitizer), read user's touches on keyboard optically using stereoscopie cameras. Touch detected by shadow sharpness of finger in projected image. Surface can be transparent (translucent) and cameras underneath (total internal reflection), or shadow coalescence. Projected image can be pre-distorted, e.g. for conical projection onto surface to appear flat. Projector can be "spot" projector, synchronized with sensor, to simplify imaging detection. [Lightpen07] Lightpen "Light pen product information circa 2007: multiple vendors in file", Tauris, www.fastpoint.com, etc. Lightpen Vendors include Tauris (germany), Fastpoint.com, others. Mention of CRTs versus LCD displays. [LimaE07a] Lima, Eduardo "Yet another post on "Kinetic" scrolling", blogspot.com, July 30, 2007 Third-party implementation by Chris Lord of MokoFingerScroll project, kinetic scrolling similar to that of Apple iPod and iPhone user-interface. [LinJC07a] Lin, Jao-Ching; Huang, Shyh-In; Chu, Lin-Abel; and Shen, Chung-Yi "Method for identifying a movement of single tap on a touch device", United States Patent 7,180,506, February 20, 2007 Tap-gesture detecting using two time spans/thresholds and a maximum distance threshold. [LinJC07b] Lin, Jao-Ching; Huang, Shyh-In; Chu, Lin-Abel; and Shen, Chung-Yi "Method and controller for identifying a drag gesture", United States Patent 7,184,031, February 27, 2007 Drag-gesture detecting using two time spans/thresholds and a minimum distance threshold. [LinJC07c] Lin, Jao-Ching; Huang, Shyh-In; Chu, Lin-Abel; and Shen, Chung-Yi "Method and controller for identifying double tap gestures", United States Patent 7,190,356, March 13, 2007 Double-tap-gesture detecting using two time spans/thresholds and a minimum distance threshold. [LiuJ06a] Liu, Jun; Pinelle, David; Sallam, Samer; Subramanian, Sriram; and Gutwin, Carl "TNT: Improved Rotation and Translation on Digital Tables", Proc. Graphics Interface 2006, pp. 25-32 GUI to rotate images on a digital tabletop display: gestures/direct manipulation to rotate and/or translate images (manipulable objects) in a tabletop display. Multifinger palm gesture. [LongeMR06a] Longe, Michael R. and van Meurs, Pim "Virtual keyboard system with automatic correction", United States Patent 7,030,863, April 18, 2006 Dictionary correction on small virtual keyboards on touchscreen, match is not by letters per se but by words whose sequence of positions (center of virtual keys) is closest pattern to actual positions typed by user (which may mis-target key). Cites to projection/projected keyboards on surface, peratech/electrotextiles fabric/pressure-sensing touch sensors. Compare with stroke targetting on PenPoint, Buxton virtual keyboards. [LoprestiDP06a] Lopresti, Daniel P. "Daniel P. Lopresti: Publications", www.cse.lehigh.edu/~lopresti/publications.htnl, fetched 2006 See: generative biometric attacks on signature verification / handwriting biometrics. See other publications. [LuJ06a] Lu, Jin "Multi-point Touch Pad", United States Patent 6,995,752, February 7, 2006 Multi-point touchpad digitizer using X/Y array of strain gauges below a compliant layer: argument is that it senses touches between sensor pad better than force-sensing resistors, which only respond when touched on the sensor. Also states that capacitive/electrostatic multi-touch pads (before 2001) only responded to direct physical touch. [MacDailyNews07a] MAC Daily News "Apple's Advantage: iPhone multi-touch patent", MAC Daily News, June 20, 2007: available at http://www.macdailynews.com Press article on multi-touch technology from Apple iPhone, but does not specifically state which patent, or whether it is on the hardware technology [MacEssentials06a] Mac Essentials "Hinweise auf Tablet Mac mehren sich", Mac Essentials, March 10, 2006, available at www.mac-essentials.de Press article on possible Tablet MAC pen computer, based on iGesture technology from Fingerworks (later iPad) [MacEssentials07a] Mac Essentials "Rubrik Apple: Das Lexikon der Fingersprache", Mac Essentials, August 2, 2007, available at www.mac-essentials.de Press article on Multi-touch gestures with iPhone, cites Fingerworks. Cites 300 gestures/chords, states that the dictionary functionality is the gist of the patent claims. Examples include two-fingers down for copy, up for undo, etc. [MacEssentials07b] Mac Essentials "Die Zukunft liegt zwischen den Fingern", Mac Essentials, January 22, 2007, available at www.mac-essentials.de Press article on Multi-touch gestures with iPhone, technology originating at Fingerworks. Mentions separate development by Jeff Han of New York University. Cites Bruce Tognazzini that the idea (?) was obvious ("hanging in the air") 25 years previously. [MacRumors06a] MacRumors.com "Apple Refining the Touch Screen Interface? Evidence for a Tablet Mac?",



macrumors.com, March 10, 2006 Report on Apple filing touchscreen patents, including for multi-touch. Includes comments on acquisition of Fingerworks iGesture multitouch by Apple, Lemur multi-touch music instrument panel, links to Jeff Han FTIR multitouch demonstration, [MackeyBL07a] Mackey, Bob Lee and Golovchenko, Mykola "Capacitive Sensing Apparatus", United States Patent 7,218,124, May 15, 2007 Capacitive touch tablet using shaped traces on single side of a substrate with no crossovers: various patterns sinusoidal width, varying concentric offset circles, traces tapered in opposite directions every other trace. Compare with charge-ratio, or HP sinusoidal grid? [Macwelt07a] Meiners, Ole "Touch me! John Elias Fingerworks", Macwelt, August 3, 2007 News article on Apple filing for patents based on multi-touch technology aquired from FingerWorks. Dictionary of about 300 multi-touch gestures. John Elias, Wayne Westerman, Myra Haggerty. [MainI07] Main, Ian J.B.; and Ward, Jean Renard "Detection of System Compromise by Per-Process Network Modeling", United States Patent Application Publication 2007/0169192 A1, July 19, 2007 See also related application 2007/0067623 A1. Detection of rootkits and unauthorized software / virus by instrumenting network activity by individual processes, and detecting violation of invariant conditions or highly unusual behavior. Similar to an application firewall, but with analysis done in opposite direction. [Makowski06a] Makowski, Dirk "Fujitsu Stylistic 1000", www.winhistory.de Technical information and photographs of Fujitsu Stylistic 1000, Windows for Pen Services 1.0, Windows for Pen Services 2.0 (a.k.a. PenWindows) Web site contains numerous screen shots, simulations of early Microsoft OS versions, including Windows Bob and DOS [Makowski06b] Makowski, Dirk "Windows 3.1 for Pen Computing 1.0", www.winhistory.de Screen shots of Windows for Pen Computing 1.0 on Windows 3.11. Trainer application for handwriting recognizer, User training program (WinTutor) to learn how to use handwriting input objects and tools. Shows Pen Services 1.0 gestures: check-mark for edit text, curlicue for cut and un-do. [MalikS07a] Malik, Shahzad "An exploration of multi-finger interaction on multi-touch surfaces", Ph.D. Thesis, Grad. Dept. of Computer Science, University of Toronto, 2007 VisualTouchpad touch surface: vision-based large-area touchpad : two cameras watch hands over a black (paper) surface, detect 3D position of fingertips and hands. Says can recognize gestures based on configuration of hand better capacitive multi-touch sensors, since it see the entire hand. Image translation and rotation by dragging a single corner of a rectangular image. Clutching gesture. Thumb slider combined with pointing gesture with index finger. Self-revealing gestures by onscreen display. [MartenV06a] Marten, Victor and Wang, Wei "Data acquisition from capacitive touch pad", United States Patent 7,075,523, July 11, 2006 Circuitry/hardware for capacitive grid touchscreen, using multiplexer and multiple switching stages to address sense lines. [Matthey07] Matthey, Florian "Quantum Research will klagen", MACWelt magazine, February 21, 2007. Available at www.macwelt.de Quantum Research files patent suit on touchscreen interface for iPhone against Apple: patent not specified, described as related to hardware with touchscreen under the surface cover [MaxwellJW06] Maxell, John. W. "Tracing the Dynabook: A Study of Technocultural Transformations", PhD Dissertation, Factully of Education, University of British Columbia, Canada, November 2006 History of Dynabook and its impact on education, contrast of liberal democratic society by Alan Kay with resulting development. Does not focus on impact on graphical user interfaces. Special mention of FOSS "Free and open software" conflict between sharing of development and corporate ownership of software intellectual property with Richard Stallman in 1990's. [MetatlaO06a] Metatla, Oussama and Harrar, Lila "Listening to Graphs & Hearing Diagrams", Proc. 1st Ibn Badis Scientific Annual Conference, IBSAC 2006, London UK, July 8, 2006 Auditory display / sonification (vs. audio feedback): speculative overview of general goals of auditory display of graphica or visual information, for situations in which user's vision is otherwise occupied. [Microsoft06a] Microsoft Corporation "Microsoft Tablet PC - Working with the Pen Tip", msdn.microsoft.com Description of numerous differences between mouse and stylus/pen GUIs as part of education of "mouse" GUI developers to develop applications of Tablet PC and pen-computing: primarily choice of "visible" cursors, obscured view of cursor preventing hover feedback [Microsoft06b] Microsoft Corporation "Microsoft Tablet PC - Design Recommendations", msdn.microsoft.com Description of numerous differences between mouse and stylus/pen GUIs as part of education of "mouse" GUI developers to develop applications of Tablet PC and pen -computing. Ink data, use of handwriting recognition, pen as both pointing and input device. Mentions problem of stylus not being suitable for external monitor system and corresponding need to switch between mouse and pen. Mentions that Tap gesture often results in some motion, which does not occur with a mouse. See also 2003 version



[Microsoft06c] Microsoft Corporation "Microsoft Tablet PC - Visual Design for Tablet PC", msdn.microsoft.com calibration, parallax, pen position, and positional feedback of Tablet to Screen, not required with mouse. Tablet tap gesture more difficult than mouse click. [Microsoft06d] Microsoft Corporation "Microsoft Tablet PC - Making the Pen Work", msdn.microsoft.com Application gets both mouse messages for pointing/selection, and separate message for pen data (electronic ink). Mouse data is for pointing, electronic ink is additional data not supported by mouse data. [Microsoft06e] Microsoft Corporation "Welcome to Windows Mobile!", msdn.microsoft.com Introduction to Windows Mobile OS for Palm OS Developers: Windows Mobile 5.0. Explicit reference to Windows Mobile OS [Microsoft06f] Microsoft Corporation "Standalone Device Emulator 1.0 with Windows Mobile OS Images", msdn.microsoft.com Device Emulator for Windows Mobile 5.0 OS devices with Visual Studio 2005. Explicit reference to Windows Mobile OS [Microsoft06g] Microsoft Corporation "MSDN Search: Windows Mobile operating system", msdn.microsoft.com List of references to Windows Mobile Operating System in MSDN documentation as of June 2006 [Microsoft06h] Microsoft Corporation "The Windows Vista Product Guide -- Beta 2", www.microsoft.com Description of Beta 2 version of Windows Vista OS with Tablet PC functionality: Additional visual feed back on gestures to avoid perception that action was not completed. Multiple selection triggered by hover of stylus over files/object (requires proximity sensing?), Pen Flick gestures (compare with GO/PenPoint): scroll up, page up, undo, page down, delete, etc. For touch screen, brings up simulated/virtual mouse device a.k.a. Touch Pointer. Snipping Tool from Experience Pack for Windows XP Tablet Edition 2005. Trainable handwriting recognition. Gesture erase scratch-out, strikethrough, vertical scratch-out, circular/angled scratch-out. Compare with Wang Freestyle laptop? [Microsoft06i] Microsoft Corporation "Windows XP Tablet PC Edition 2005 Hardware Requirements", www.microsoft.com For Windows Tablet PC (prior to UMPC Ultra-mobile PC), digitizer required to be 1000 pts/inch or 5 times the display resolution, 100 (preferably 133) points/second, proximity sensing required. Note: on UMPC, requirements changed, proximity no longer required to run Tablet PC OS on UMPC. Note: confuses resolution and accuracy? Ctl-Alt-Del must be separate mechanism. [Microsoft06j] Microsoft Corporation "The Microsoft Office System: A Guide for Educators", msdn.microsoft.com, 21-March-2006 Educator's tutorial on office: includes sections on OneNote electronic ink notetaking app, Research Pane, writing guides (baselines for writing) for Tablet PC. [Microsoft06k] Microsoft Corporation "Tablet PC Platform SDK: Tablet Input", msdn.microsoft.com, 21-March-2006 Excerpted from "Building Tablet Applications", Rob Jarvet, Microsoft Press Distinguishes between system gestures / pen actions, on a par with mouse actions such as click and drag, versus ink-based gestures such as scratchout, up arrow, and curlique. Mentions pen-inversion, the "eraser" end of a two-ended stylus (compare with Wang Freestyle). Cites to WinTab specification, states that WinTab API was not adequately standardized. Wisptis.exe: previous Windows Ink Services for Pen as former name for Tablet PC. Mouse emulation by mapping pen/stylus actions to mouse actions, points out problem with tool tips because stylus cannot be held stationary out of contact. (Differences between mouse input and digitizer input) [Microsoft06l] Microsoft Corporation "Announcing the Ultra-Mobile PC", msdn.microsoft.com General announcement of Ultra-Mobile PC (UMPC) running Windows XP Tablet PC Operating system. Includes design tips on programming for small display, also notes about "accidental activation" from touch screen. Note that UMPCs hardware might not support hover. Specific reference to problems with edge effects of the digitizer, especially towards the bottom of the display (hardware design?). "Touch Improvements" pack to adjust UI design from default mouse/tablet settings to work better with Tablet PC (e.g. larger target windows). UMPC display emulator for development. Real-Time Stylus API. OrangeGuava pen-centric / ink-driven desktop. Persisting Ink on the Web. [Microsoft06m] Microsoft Corporation "Brown University, Microsoft Research Open Research Center on PenCentric Computing: Alliance to explore pen-based computing", Press release, March 20, 2006, www.microsoft.com Andries van Dam, Microsoft Center for Research on Pen-Centric Computing. Funding for projects related to Tablet PCs and UMPCs for gesture recognition: mentions Music and chemistry applications. [Microsoft06n] Microsoft Corporation "Ultra-Mobile PC: Microsoft Origami", Microsoft Corp, 2006 UMPC promotional video: hand-held touchscreen (not stylus) hardware for Microsoft Tablet PC. Available 2014 at www.youtube.com/watch?v=lVLhvn4cvnQ [Microsoft07a] Microsoft Corporation "Practice using pen flicks", Windows VISTA OS control panel: April 2006 Pen-flicks: radial "flick" gestures for Forward/Back/Drag-up/Drag-down, similar to Penpoint and to radial gestures. Also has octant radial gestures for Delete/Copy/Undo,Paste: user can assign different meanings to the gestures. Compare to Kurtenbach94 (Bill Buxton?)



[Microsoft07b] Microsoft Corporation "RecognitionProperty Member", msdn.microsoft.com, Nov 2007 Stroke Segmentation documentation for TabletPC InkRecognizer / InkRecognizerBase class in .NET [Microsoft07c] Microsoft Corporation "Features Explained: Tablet PC Support", www.microsoft.com, Dec 2007 Windows Vista Tablet PC support: gestural feedback (shape of gesture?) in single-tap, double-tap, and right-click. Mentions both pen/stylus support and touch-screen support (separately?), pen-flick gestures drag-up, drag-down, delete, undo. Brings up simulated mouse (compare with Wang patent?) for touchscreen to aid in relative-motion input with finger on touchscreen. [Microsoft07d] Microsoft Corporation "BUGCODE_USB_DRIVER: posting to Microsoft TechNet forum concerning Vista drivers", http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=1714403, June 10, 2007 Posting concerning Vista/Longhor device drivers: includes dump of driver info on N-Trig tablet digitizer driver ntrigdigi.sys, dated 11-02-2006, N-trig Innovative Technologies, N-trig tablet digitizer in-box driver [Microsoft07e] Microsoft Corporation "Tablet stylus barrel button or right button in Silverlight", http://forums.silverlight.net, May 6, 2007 Posting concerning stylus barrel button or right mouse button: request that context menu not be forced for these events, agreement from Microsoft. Compare with mouse emulation in tablet drivers (Synaptics, etc.) [Microsoft07f] Microsoft Corporation "The History of Microsoft Surface", Microsoft Presskit Release, May 2007 Microsoft surface tabletop touchscreen, original prototype 2003. Frequent mention of Bill Gates' approval. No information on design, or history of tabletop systems [MillerP06a] Miller, Paul "Sony gets official on new Vaio UX Micro PC", Endgadget.com, May 16, 2006, 2006 Sony Viao not-a-UMPC handheld Windows PC with slide-out keyboard: compare with OQO? [MillerP06b] Miller, Paul "Some iPhone touchscreen roots 'splained by Fingerworks inventors", Mitsubishi Electric Research Laboratories Press Release: June 16, 2006 Fingerworks electrostatic digitizer/touchscreen used in Apple iPhone, multi-touch digitizer. [Mitsubishi06] Mitsubishi Electric Research Laboratories "HoverPen - New Method for Text Input with a Handheld Device", Mitsubishi Electric Research Laboratories Press Release: June 16, 2006 Stylus is a projector (similar to laser pointer): digitizer is an camera reading the projected light. Currently using a unistroke handwriting recognizer. Part of "Off the Desktop Interaction and Display" research. Mentions as digitizer addition to cellphone and PDAs. http://www.merl.com [Morag07a] Morag, Meir and Perski, Haim "Transparent Digitizer", United States Patent 7,292,229 B2, November 6, 2007 N-trig transparent digitizer, using an organic conductor instead of ITO indium tin oxide, cordless stylus excited by loop around digitizer similar to Wacom. Electromagnetic (?) or electrostatic/electric field (?) digitizer tablet (not clear). Part of design that also senses finger touch via electrostatic sensing. [MoscovichT07b] Moscovich, Tomer "Principles and Applications of Multi-touch Interaction", Ph.D. thesis, Brown University, 2007 User interface techniques for continuous multi-touch input, especially parallel control of multiple parameters (e.g. position and orientation, grouping and moving objects). Multi-finger (same hand) versus bi-manual (two hands) gestures: e.g. gestures using two fingers of one hand (circle) versus two fingers on separate hands (narrow ellipse). [MotionComputing06a] Motion Computing "LE-Series and LS-Series Tablet PCs Microsoft Windows XP Tablet PC Edition 2005 User Guide / L Series UserGuide", Motion Computing, 2006 Tablet PC. Describes LS electronic digitizer as a non-touch screen, only responds to pen, also refers to Touch Screen: e.g. Touch Screen Hover Mode using a function key. Press-and-hold and tap-and-drag both gestures (?). [MotionComputing07a] Motion Computing "Motion LE1700 Tablet PC Series -- Quick Reference Guide for Windows Vista", Motion Computing, 2007 Tablet PC with Windows Vista, Quick Reference. Press-and-hold / tap or touch and drag touchscreen. [MotionComputing07b] Motion Computing "Motion LE1700 Tablet PC Series -- Quick Reference Guide for Windows XP (with pen extensions)", Motion Computing, 2007 Tablet PC with Windows Vista, Quick Reference. Press-and-hold / tap or touch and drag touchscreen. [MotionComputing07c] Motion Computing "Motion LE1700 Tablet PC User's Guide: Windows XP", Motion Computing, 2007 Tablet PC User's guide: mostly hardware features including SIM/phone card slot. Press-and-hold / tap or touch and drag touchscreen, or use Function button and tap. [Motorola06a] Motorola Corporation "MotoMING A1200 MotoManual (2006 version)", www.motorola.com MotoMING A1200 GSM cellphone:compare with 2005 manual. Built-in stylus for touchscreen. Handwriting recognition dual area for single characters, time-out or swap writing in the two areas. Compare with Wang GUIDE? Subset/mode selection alpha, numeric, shift-capitalize, special characters. Document viewer with drag gesture for scrolling. Two-stroke (tap-plus-line) for zoom gesture. tap and hold gesture for pop-up menu on item. Tape and hold on virtual keyboard/keypad: e.g. tap and hold on zero in phone pad to enter



international calling coded. Dwell/press and hold gesture. [Mozdev.org06a] Mozdev.org "Mouse Gestures: rapid execution of common browser commands with mouse movements", http://optimoz.mozdev.org/gestures Mouse gestures for Mozilla / Firefox browser, Thunderbird Mail, and Chatzilla; Close-Tab (down-right), New-Tab (up), Open links (end with Right-up-left). Recognition based on changes in direction? [Mozdev.org06b] Mozdev.org "Screenshots: MozGest 0.3.5 Final", http://optimoz.mozdev.org/gestures Mouse gestures for Mozilla / Firefox browser, Thunderbird Mail, and Chatzilla. Points out confusion of gesture input versus selection of text (because UI is "gesture-enabled", and not "gesture-centric")? [Mozdev.org06c] Mozdev.org "Gesture Exchange", http://optimoz.mozdev.org/gestures Mouse gestures for Mozilla / Firefox browser, Thunderbird Mail, and Chatzilla. Gesture recognition apparently based on changes in direction. Certain gestures involve right mouse button, mouse wheel. [Mueller-Prove07a] Mueller-Prove, Matthias "Eine kurze Geschichte der Software-Ergonomie", www.mprove.de, fetched May 12, 2007 Historical highlights including 2002 presentation on Memex, SketchPad, contains detailed photo of Memex by Vannevar Bush: See 2002 version by co-author [MultiTouchScreen07] Multi-Touchscreen.com "iPhone uses Multi-touch Technology", http://multitouchscreen.com, July 2, 2007 Article on Apple multi-touch UI patent application 20060026535 -- confuses published application versus granting of patent. Highlights two-finger gestures for Zoom and Unzoom. Cites multi-touch UI work by Jeff Han as prior art. [NRW07] www.media.nrw.de "Touchscreen für iPhone kommt aus NRW", Press release by state government of Nordrhein-Westphalia, available at http://www.media.nrw.de Apple multi-touch iPhone: touchscreen produced by Balda in Germany, production transferred to TPK in Taiwan. [N-Trig07a] N-Trig Corporation "N-Trig DuoSense (TM) product information", Available at www.n-trig.com Capacitive/electrostatic digitizer technology for either stylus/pen or finger input, basically tablet and touch screen in same unit as dual-mode touchscreen: describes it as multi-touch. No power required for stylus, appears to use induced voltage similar to Wacom (?) electromagnetic. Driver included in Windows Vista package. Cites stylus electrostatic sensor for palm rejection: less clear on touch sense. [N-Trig07b] N-Trig Corporation "Dell places 12.1-inch touch screen notebook panel order with AUO", www.ntrig.com press release, November 20, 2007 Public announcement of N-trig with Dell Tablet PC. Capacitive/electrostatic digitizer technology for either stylus/pen or finger input. [NandaG07] Nanda, Gauri "Mobile Wakeup Device", United States Patent Application 2007/0036034 Alarm clock which walks/rolls away when it rings, making the sleeper have to get up to chase the clock [NaoneE07a] Naone, Erica "Computing on Paper: Livescribe's smartpen turns a sheet of paper into a computer", Technology Review, December 13, 2007 Anoto digital pen (tablet paper) with limited multimedia functions added: voice recording with two microphones to improve quality of recording while taking notes. Founder formerly worked on Fly Pentop computer (www.flyworld.com) [NationalInstruments06a] National Instruments "Getting Started with the LabVIEW Touch Panel Module for Windows CE, Version 8.5", National Instruments, 2006 Touchscreen / touch panel for LabVIEW visual programming language. File contains further touch screen references to 2001. [Neo07a] Kooi, Koen "Kinetic scrolling on the neo1973", Youtube.com, Fetched Dec 2014 Brief video of kinetic scrolling (coasting + decelleration) on smartphone, similar to feature on iPhone/iPad. [Neonode07a] neonode.com "Neonode N2 user guide", www.neonode.com, Fetched Dec 2007 Touch screen smart phone: has sweep gestures across different regions of the screen: e.g. diagonally in different direction to change virtual keyboards or close an application, from center out in a direction to scroll, left or right across top of bottom, or sides of display screen. [NerdWatchMuseum07a] Nerd Watch Museum "Wrist Watch Computers", www.blogspot.com, September 08, 2007, 2007 Collection of computerized wrist watches: included Seiko/Epson RC-20 of 1985 with touchscreen digitizer user interface and 2000 bytes of storage [NetBeans06a] NetBeans project, Java.net "Mouse Gestures v1.0", Java.net, 2006: NetBean project Mouse gestures used in NetBeans IDE for Java, circa 2006. Has minimum movement for a line, flick gestures in multiple directions, adjustable angle parameters. Most gestures consist of lines, or combinations at 90-degree angles. [NewsletterArchive07] Newsletter Archive "Help Plan the Future of Tablet PC", www.newsletterarchive.org: Mobile PC Newsletter May 02, 2007



New features for Tablet PC, Ultra-Mobile PC / UMPC ink annotation, ink recognition, TEO 3.0 personal planner, OneNote Calendar. Tablet Factory announces sales of word dictionaries for Tablet PC. [NishiboriY06a] Nishibori, Yu and Iwai, Toshio "Tenori-On", Proc. 2006 Int'l Conf. on New Interfaces for Musical Expression (NIME06), Paris, France, pp. 172..175 Musical intstrument interface, using 16x16 2D array of combined LEDs for display and topical touch sensors. Inclues tilt/accelerometer, duration of touch. User pushes to set switches, system scans input. Talks about direction of touch, no details. No pressure/force sensing, no mention of multi-touch. [Nokia07a] Nokia "Beyond Multi-touch: Nokia Patens 3D Touchless Control Interface", www.symbianfreak.com/news/008/01/touchless_device_patent_from_nokia.htm Commentary on S60 No-touch / touchless electrostatic touch UI device from Nokia. Allows in-air threedimensional 3D gestures. [NorthK06a] North, Ken and D'Souza, Henry "Acoustic Pulse Recognition Enter Touch-Screen Market", Information Display, Vol 22 No 12, December 2006, pp. 22-25 Acoustic Pulse Recognition / bending wave sonic touchscreen in contrast to SAW surface acoustic wave, can use plain glass without a coating. Synonyms flexural plate wave and loest order anti-symmetric Lamb wave -- wave disperses, higher frequencies propagate faster, also reflect from edges. Recognizes moment of touching (multi-touch) and dragging (which also causes characteristic acoustic signal), but not multiple points of static contact. Position can be determined by characteristic vibration pattern of location on glass. [Numonics07a] Numonics Corporation "PI-1500 Product Information", Available at www.interactivewhiteboards.com, www.numonics.com Digitizing tablet integrated with a stand-alone desktop display, USB interface, intended as a small whiteboard device. Electromagnetic. Direct mouse compatibility over USB (via plug-and-play driver for absolute vs. relative position?) [Numonics07b] Numonics Corporation "Interactive Display product information", Available at www.interactivewhiteboards.com, www.numonics.com Electromagnetic tablet/display whiteboards [Numonics07c] Numonics Corporation "Freedom Writer product information", Available at www.interactivewhiteboards.com, www.numonics.com Electromagnetic tablet/display whiteboards: very small unit used as a remote-control input for a tablet PC for presentation, using infrared connection, About the size of a large PDA held horizontally. [Numonics07d] Numonics Corporation "Mobilepresenter BT IIproduct information", Available at www.interactivewhiteboards.com, www.numonics.com Electromagnetic tablet/display whiteboards: very small unit used as a remote-control input for a tablet PC for presentation, using Bluetooth. About the size of a slate. [OQO06a] OQO Inc. Study "Tablet PC Edition of Model 01+ hand-held computer", www.oqo.com Hand-held tablet PC running Windows XP Tablet PC edition -- previous product ran regular Windows XP, no electronic ink: predates Origami [OTMTechnologies06a] OTM Technologies "Optical Translation Measurement VPen Device", OTMTechnologies.com, fetched 2006 VPen electronic digitizer pen about the size of an Anoto pen, no special patterned paper. Handwriting (printing?) recognition built in, tilt sensing can be used as joystick. Laser and optical sensor in tip to measure "absolute" (?) position on surface? Mentions barcode scanning and OCR text scanning, 3D sensing. Also motion sensors (TMD Remote MOtion Detection) using projected laser pattern and optical sensors. Product names NaviLite and Light Speed Innovation. See also Kinrot, Lederer. [OdomG06] Odom, Gary "Toolbar Manipulations", United States Patent 7,036,087 B1, April 25, 2006 Tool bar re-configures automatically, changing/re-arranging toolbar menu items depending on what tool user has selected and is using most frequently: file contains press reports on patent assertion against Microsoft. See also www.patenthawk.com (filed under Odom): patenthawk.com [OlsenDR07] Olsen, Dan R. Jr. "Principles of Interactive Systems", Textbook published electronically at Computer Science Department http://icie/cs.byu.deu, Brigham Young University, 2007 Chapter 19: Text input. text entry should measure not speed, but account for correction steps also. Includes alternative keyboards (one-handed), stylus/digitizer input using various keyboard/button layouts, pop-up menus. Cites Goldberg Unistrokes, T-Cube pie menus, Edge-write mechanically constrained handwriting on PDA, Quikwriting and Cirron circular layout with one stroke moving from character to character. Otherwise not much information on handwriting recognition, script. Chapter 20: Digital Ink. Short review of digitizers: resistive film, ultrasonic, Anoto, DiamondTouch capacitive, optical path digitizers. Visual parallax on rear-projected displays. Described multi-stroke gestures (segmentation) as a "messy problem". Zone recognizers for handwriting. Labels handwriting recognizers as "vector classifiers", cites elastic matching of Tappert. Digital/electronic ink as data type (Slate?), credits later Tivoli system from at Xerox Parc by Moran/Kurtenbach. Longer discussion of sketch input and sketch editing, ink annotation on



text. Chapter 21: Selection. Fitt's law and menus, extension to Steering Law where path (of mouse through cascading menus) is important. Modifications to mouse behavior not to be linear or absolute, to improve path and targeting accuracy. Short mention of digitizer tablet selection gestures. Chapter 7 "Abstract Widgets" missing from electronic file [Ording06] Ording, Bas "Operation of a Computer with Touch Screen Interface", United States Patent Application Publication, 2006/0053387 A1, March 9, 2006 multi-touch user interface, simulated keyboard with multiple keys, refers to using "pre-determined behavior" of the user's touch. Uses area (multiple points) of finger touch as an indication that the user is pressing down harder, therefore uses touch area as a surrogate for the pressure/force for the user. [OrdingB07a] Ording, Bas "Scrolling List with Floating Adjacent Index Symbols", United States Patent Application 2007/0150830, June 28, 2007 When scrolling, show section/group/index displays next to scrolling with swipe gesture, bounce when hitting end of list. Bounce-list at end of scrolling on Apple iPod/iPad. [Origami06a] Origami Product Announcements "Origami Product Announcements", www.cebit.de Press announcements of the Origami project, a.k.a. UMPC Ultra-Mobile PC. Small form-factor Tablet PC, many models with no built-in keyboard. Mentions use of single characters as macros/gestures -- compare with Pencept/CIC 1985. Has pop-up virtual keyboard in two halves, for the thumbs. [Origami06b] Origamiproject.com "Origami Project", http://origamiproject.com Press announcements of the Origami project, a.k.a. UMPC Ultra-Mobile PC. Mentions TouchPack: DialKeys touch keyboard from Fortune Fountain Ltd., and "Touch Improvements" setting changes for Tablet PC software [Origami06c] Origami project "Origami Project File", http://origamiproject.com Correspondence and technical file for Origami project: version of Tablet PC with resistive film touchscreen, pop-up keyboards for two thumbs. Multiple demos of Origami sub-subnotebook computers at CeBIT 2006. Compare with OQO. [OuyangTY07a] Ouyang, Tom Yu "Recognition of Hand Drawn Chemical Diagrams", Matser's Thesis, M.I.T., May 2007, Dept. of EE/CS Sketch recognition for chemical diagrams. Uses domain knowledge to refine inconsistencies (e.g. atomic valence) and improve recognition. [PCWelt07] PC-Welt magazine "Apple's iPhone patent multi-touch advantage", www.pcwelt.de, June 20, 2007 Press-article quoting iSuppli on market for multi-touch touch screens, one Apple patent (on UI only?) [Palm07a] Palm Computing "Palm: Benutzer-handbuch: Ihr Palm Centro Smartphone", www.palm.com User manual for Palm Centro Smartphone (in German): simple touchscreen [PalmOS06a] PalmSource "Palm OS Emulator / Palm OS Simulator", www.palmos.com Emulator for Palm OS handheld touchscreen PDA, on Windows/Mac: executes M68000 instructions and ROM images, based on open-source Copilot, enhanced by Palm. Simulator runs Palm OS over Device Abstraction Layer compiled for native Intel x86 (Windows), PACE Palm Application compatibility Environment. Also noted Graffiti, Graffiti 2 handwriting recognition. [PalmOne06a] PalmOne "Ways to Enter Data into a Palm Device: Write it, type it, beam it - it's your choice", www.palmone.com Graffiti 2 handwriting recognition. Also mentions on-screen keyboard, Note Pad to scribble notes in electronic ink [PalmOne06b] PalmOne "Graffiti 2 writing", www.palm.com Graffiti 2 handwriting recognition. Examples of unistroke (?) shapes from Graffiti 2, which is a version of the CIC handwriting recognition system. Includes description of Palm OS Simulator and Palm OS Emulator, based on "copilot" project by Gref Hewgill [PalmOne06c] PalmOne "PalmOne LifeDrive: Getting Started", www.palmone.com PalmOne touchscreen PDA. Manual button on side for screen rotation. MP3 player (Rhapsody), PIM organizer, Office. Installable third-party applications. [PalmSource06a] PalmSource "Palm OS: Getting Started / Languages and Environments", www.palmos.com Palm OS languages: Java, "C", C++, Java, Visual Basic. file includes additional material on MetroWorks, M68000 ARM processors, Garnet and Cobalt versions of Palm OS, Palm OS Developer Suite. See also Palm Simulator and Emulator files. [Panjit06a] Panjit Touch Screen "Resistive multi-touch screens", www.panjittouchscreens.com Resistive matrix multi-touch touchscreens. [Parascript07a] Parascript Inc. "Intelligent Recognition technology - the final frontier", www.parascript.com Interactive character recognition ICR (a.k.a. on-line handwriting recognition). Describes a form of chain codes where script writing it broken into eight possible writing motions that combine in appropriate sequences. Uses term "motion theory" for handwriting dynamics / graphonomics. Describes segmentation problem in script: "clear" vs. "dear".



[Patalong06] Patalong, Frank "Der Anfang vom Ende gefällter Bäume", Spiegel Magazine, June 23, 2006 E-Ink electronic paper. Mentions prototype project for newspaper publication in electronic book format: De Teij, New York Times, Westdeutsche Allgemeine Zeitung (in PDF format?). Electronic book hardware from iRex: product name Iliad [PdaParts07] PdaParts.com "Pam Tungsten T3, Tungsten T5, Palm TX & LifeDrive Digitizer/Touch Panel", www.pdaparts.com Replacement glass tablet digitizer / touch screen for Palm PDAs: manufacturer not identified, shows internal construction [PenComputing06a] Pen Computing Magazine "Pen windows / WebPads / Tablet PCs", www.pencomputing.com/PenWindows (date approximate) Background article: pen computers used in vertical markets such as sales force automation, utilities, health care. Mentions PenWindows/Microsoft's Windows for Pen Computing arising after PenPoint/GO received attention. Vendors using CIC PenX alternative to Windows95. History of Microsoft's efforts at PDAs, TabletPC, WebPad, Smart Displays (Project Mira), UMPC. [PenenbergAL07] Penenberg, Adam L. "Can't Touch This", Fast Company magazine, Issue 112, February 2007, page 86: www.fastcompany.com Press article on Jeff Jefferson Han and multi-touch optical technology demonstrated at TED 2006, Monterey California: company name is Perceptive Pixel. Competition is TouchLight with Microsoft, and DiamondTouch. Also Panasonic, Accenture [PepperComputer07] Pepper Computer "Pepper Pad 3 Web Computer", www.pepper.com, www.hanbitamerica.com Pen computer, similar to UMPC Tablet PCs, running Linux OS: software for Amacon.com MobiPocket electronic E-books, TV remote control [PfleegerCP07a] Pfleeger, Charles P. "Security in Computing, 4th Edition", Prentice-Hall, 2007 Tutorial and reference on encryption and PKI, intrusion detection, user authentication, secure/trusted computer systems, copy protection, DRM, legal issues of (excessive) computer security [PhilippH06a] Philipp, Harald "Keyboard with reduced keying ambiguity", United States Patent 6,993,607, January 31, 2006 Capacitive touch keyboard: report key with greatest capacitive signal, to get better location of key. Compare with peak profiling measurement of Rekimoto? [Philips06] Philips "Entertaible concept: combination of electronic gaming and traditional board games", www.research.philips.com Entertaible from Philips: Multi-touch "smart" tablet used for playing board games. Multi-touch can sense up to forty-five fingers or touch points. Compare with Diamond Desk of Mitsubishi [PittmanJA07a] Pittman, James. A. "Handwriting Recognition: Tablet PC Text input.", IEEE Computer, September 2007, Vol 40 Number 9 pp. 49..54 Special issue on Tablet PC uses (mostly whiteboards) in college education. Article gives a brief and very Microsoft-centric view of pen-computing history starting in early 1990s. Recognizer uses word dictionary and time-delayed neural network TDNN. Microsoft/Vista lexical dictionary has been expunged of offensive words. [PittmanJA07b] Pittman, James. A. and Manu, Mitica "Handwriting Recognition Using Neural Networks", IEPO Patent Application Publication WO/2007/002377, PCT/US2006/024451 Neural Net handwriting recognition. Refers to use of languages, starting and blocking masks. [Pixelsumo06] Pixelsumo "Multi-Touch Interaction Roundup", www.pixelsumo.com Review of: Apple application on musical input using multi-touch; Jeff Han multi-touch digitizer using frustrated internal reflection; TactaPad; SmartSkin which uses capacitive touchscreen tablet and projection from above of images of playing cards, etc. [PocketPCMagazine06a] Smartphone and Pocket PC Magazine "What is Windows Mobile? Introduction to Microsoft's Windows Mobile OS", www.pocketpcmag.com Buyer's guide to Windows Mobile and Pocket PC products: reference to Windows Mobile OS. [PocketPCMagazine06b] Pocket PC Magazine "Tips & Tricks: Windows Mobile, Pocket PC, and Smartphone", www.pocketpcmag.com descriptions of gesture: UpDown, DownUp, UpRight/Tab, etc. Reference to Windows Mobile OS as successor to Windows CE OS [Pothier07] Pothier, Mark "In digital age, more t's are crossed poorly", Boston Globe, Sunday May 6, 2007, Page a1 Review of current education (2007) in handwriting styles / penmanship. Zaner-Bloser handwriting method, transition from block printing to cursive. Teaching of cursive/connected-writing declining, mentions legibility training that teaches quick separated characters rather than connected cursive. Cites legibility/handwriting studies by Kathering Boles, Karen Conrad, Barbara Getty, Kate Gladstone (handwriting repair consultants).



[PrevalentDevices06a] Prevalent Devices LLC "The Phraze-It Keyboard 2.0 for Windows Mobile and Pocket PC PDAs", www.prevalentdevices.com, 2006 See also one-hand gestures: small keyboard for one-finger typing on PDA instead of using a stylus and touchpad digitizer. Uses few keys by modal switching between vowels and consonants. [PrevalentDevices06b] Prevalent Devices LLC "Phraze-It One-Handed Finger-Vowel Digital Texting Method", www.prevalentdevices.com, 2006 One-hand gestures/signs for finger spelling, using only six hand gestures. Computer input is via an electronic glove, not a digitizing tablet. [PreyJG07] Prey, Jean and Weaver, Alf "Tablet PC Technology: The Next Generation", IEEE Computer, September 2007, Vol 40 Number 9 pp. 32..41 Introduction to special issue on Tablet PCs in education (mostly college level), mostly talks about sketch understanding / sketch recognition (chemical diagrams, circuit diagrams). Refers to Alan Kay Dynabook as early form of Tablet PC (not so?), Proof-of-concept Tablet PC in 1999 based on Transmeta TM5800 processor. [PryorTR06a] Pryor, T. R. "Programmable tactile touch screen displays and man-machine interfaces for improved vehicle instrumentation and telematics", United States Patent 7,084,859, August 1, 2006 Touchscreen (not surface) instrumentation control panel for automobile [Quantum06a] Quantum Research Inc. "QT1106 Demo Code: QT1106 Qwheel / Qslide / Qtouch IC", Quantum Research Group, Hamble, Great Britain Capacitive touch screen digitizer: "C" driver code: "C" driver code Additional material in paper files: Quantum acquired by Amtel: see Amtel file. [Quantum06b] Quantum Research Inc. "Sensitivity Differences When Migrating from QT100 to QT100A", Quantum Research Group, Hamble, Great Britain Capacitive touch screen digitizer: capacitive sensitivity profiles [Quantum06c] Quantum Research Inc. "QMatrix Technology White Paper", Quantum Research Group, Hamble, Great Britain Capacitive touch screen digitizer: digital charge transfer (QT) matrix scanning touch sensor, using pairs of driving/sensing electrodes, nearby finger couples to affect charge transfer (capacitance) between. [Quantum07] Quantum Research Inc. "Atmel licenses Quantum's Capacitive Touch Sense Technology", Press release, Quantum Research, September 20, 2007 Capacitive touch screen digitizer, used to simulate slide-bars at the edge of a cellular phone, rotary touch control similar to iPod. Interesting touch-sensor application on the keys of a cell phone, to allow singlecharacter writing over the keys. Technical description of capacitive/electro-static digitizer stylus sensing. [RaizG06] Raiz, Greg "ABC Scrollbar Control", www.raizlabs.com, March 27, 2006 Scrollbar with letters shown like the index in a mechanical file with items stored alphabetically. Extension to scroll bar with highlighted radio stations, or heatmap scrollbar with areas of interest higlihghted: Later (2008) posting cites to Google Chrome "Find" scrollbar. [ReadInk06a] Rai, Saritha "Is the Next Silicon Valley Taking Place in Bangalore?", New York Times, March 20, 2006, page C3 Read-Ink Technologies, founded by Thomas O. Binford: development of handwriting recognition. Mentions statistical recognition and machine learning [ReadInk06b] Read-Ink Technologies "Read-Ink Handwriting Recognition and OCR Solutions", www.read-ink.com Read-Ink Technologies, founded by Thomas O. Binford: development of handwriting recognition. Mentions statistical recognition and machine learning, unconstrained writing styles with relative independence from a lexicon. Support for handwriting input for various Indian/Asian languages [ReetzA06a] Reetz, Adrian; Gutwin, Carl; Stach, Tadeusz; Nacenta, Miguel; and Subramanian, Sriram "Superflick: a Natural and Efficient Technique for Long-Distance Object Placement on Digital Tables", Graphics Interface 2006, pp. 163-170 Flick gesture on tabletop/digital table touchscreen to move graphical objects larger distances by throwing them in any (2D) direction. Cites to Wellner, Interactive Wall. Compare with artificial reality project, PenPoint? [RekimotoJ06a] Rekimoto, Junichi; Nashida, Tatsushi; and Poupyrev, Ivan "Input Method and Input Device", United States Patent 7,088,342, August 8, 2006 Touchscreen on back of hand-held display. Touch on back is displayed at corresponding position on front. Back surface touchscreen can detect multiple/plurality of finger touches. Can have touchscreen on front also (e.g. stylus). Back surface multi-touch touchscreen uses rows of even/odd capacitive pads: touch detected by capacitance between even and odd pad: proximity sensing. [RekimotoJ06b] Rekimoto, Jun and Schwesig, Carsten "PreSenseII: Bi-directional Touch and Pressure Sensing Interactions with Tactile Feedback", Proc. CHI '06, April 22-27, 2006, Montreal Quebec Touchpad with pressure sensing (contact area), with tactile feedback (vibration) to simulate physical



button clicks. User interface uses three-level pressure sensing -- soft press / hard press threhold (off, display, activate), pressure-sensing accelleration on scrolling, positive and negative pressure using finger angle / contact position. Compare with Buxton tri-state input, pressure-sensitive mouse, dwell gesture. Cites Zeleznik pop-through mouse buttons. [Resende06] Resende, Patricia "Microsoft strengthens ties to N.E.: Software giant gives $1.2M to Brown for penbased computing", Mass High Tech, April 6, 2006, page 9 Microsoft funds Tablet PC / pen-based computing development at Brown University. Mentions ChemPad Tablet-PC application for molecule models: Microsoft Center for Research on Pen-Centric Computing, Andries van Dam, Director [RingWriter06a] RingWriter.com "RingWriter Development Note: Dictionaries", rin-writer.com, January 4, 2006 RingWriter: Linear-X writing system of ring-writer: pie menu where user draws sequentially though threeletter segments (like phone keypad), software recognizes words from dictionary (some different words may have same pie-menu combination). Paper concerns design of a word dictionary: take into account not only frequency of occurence, ("the" most frequent of all), also dictionary size, and alternative verb/adjective forms, especially in inflected language: e.g. if first-person form of verb is in dictionary (from frequent use by user), then third-person forms should be also -- because user will not understand why other forms of the verb are not present. [RingWriter07a] RingWriter.com "RingWriter User Guide for Nokia 7710", Available at www.ring-writer.com, 2010 Linear-X writing system of ring-writer: pie menu where user draws sequentially though three-letter segments (like phone keypad), software recognizes words from dictionary. If single word in dictionary, entire word can be entered with a single stroke. If multiple words from dictionary, shows pop-up menu with list of word alternatives. Also has additional gestures in center of pie-chart area for special or nonalphabetic characters, space, backspace, enter gestures similar to Pen Windows and Microsoft Tablet PC. Compare with unistroke recognizer. Other references go back to 2001. Additional papers in file, back to 2002 [RingWriter07b] RingWriter.com "The Ring-Writer input system: RingWriter User Guide for Sony-Ericsson P900/P901i pen-based smartphones.", Available at www.ring-writer.com, 2007 Linear-X writing system of ring-writer: pie menu where user draws sequentially though three-letter segments (like phone keypad), software recognizes words from dictionary. If single word in dictionary, entire word can be entered with a single stroke. If multiple words from dictionary, shows pop-up menu with list of word alternatives. Also has additional gestures in center of pie-chart area for special or nonalphabetic characters, space, backspace, enter gestures similar to Pen Windows and Microsoft Tablet PC. Compare with unistroke recognizer. Other references go back to 2001. Additional papers in file, back to 2002 [RohrabaughGB07a] Rohrabaugh, Gary B. and Sherman, Scott A. "Scalable Display of Internet Content on Mobile Devices", United States Patent 7,831,926, November 9, 2010 Convert HTML layout to resolution-independent vector display-list form, for better rendering on small (handheld/smartphone) displays. [RoschelleJ07a] Roschelle, Jeremy; Tatar, Deborah; Chaudhury, S. Raj; Dimitriadis, Yannis; Patton, Charles; and DiGiano, Chris "Ink, Improvisation, and Interactive Engagement: Learning with Tablets", IEEE Computer, September 2007, Vol 40 Number 9 pp. 42..48 Classroom Presenter and Group Scribbles shared whiteboard application on Tablet PC for college-level education, encourages cooperative engagement and discussion. [RosenbergLB07a] Rosenberg, Louis B. "Haptic Feedback Stylus and Other Devices", United States Patent 7,265,750, September 4, 2007 (See also 6,353,427, 6,088,019 Stylus with haptic feedback actuator in stylus tip: feedback can be jolt, vibration, constant force (for a constant time, or at a constant magnitude), or texture. Can also be mounted in a steering wheel axel, motion is perpendicular to steering wheel. [Rosenstein06] Rosenstein, Tatiana "Interfacedesign: In Echtzeit interaktiv agieren", netzspannung.org: MARS Projekte Gesture input in an "interactive room" using various whiteboard, theremin, touchscreen, and video capture digitizers for gesture input [RotoView07a] Innoventions, Inc. "RotoView by Innoventions: The Intuitive Display Navigation Solution for Hand Held Devices -- Fact Sheet and Evaluation System", Innoventions.com 2007 Evaluation kit (sensor card, USB connection to PC, software) Tilt and hand-acceleration sensing user interface instead of touchscreen: can view larger "virtual" display on small hand-held display, or using tilt gestures for magnify, zoom. PDA emulator software for PC. Original product: 1996. [Roush07a] Roush, Wade "Device Disorder on the Desktop", Technology Review, March 23, 2007 Article on BumpTop by Anand Agarawala at U Toronto (Bill Buxton?): desktop GUI metaphor with documents displayed on a large physical desktop with touchscreen digitizer or mouse. No reference to



Freestyle by Steve Levine, which appears similar? Cites to Lifestreams by Eric Freeman and David Gelernter, which appears similar to "Forever Directory" also by Freestyle group. [RutkowskiEJ07a] Rutkowski, Edward J. "Microsoft works on notebooks with multi-touch", www.msmobiles.com, June 21, 2007 Microsoft demonstrates commercial version of Tablet PC (Dell notebook) with multitouch input: two fingers of two hands for zoom and rotate. Mentions confusion of multi-touch with Lenovo incorporation of two digitizers in one device for touch (resistive/passive) and stylus (electromagnetic/active) digitizer tablets. [SRS06] SRS Management "Die Technology Digital Pen and Paper: FAS Group", www.srs-management.de Members of FAS Group for Forms Automation System using Anoto "digital paper": Kayentis, Meticulus, support from Hewlett-Packard printer group (for printing of the patterned forms) [SalterHC07a] Salter, Hal Christopher "Interactive game providing instruction in musical notation and in learning an instrument", United States Patent 7,174,510, February 6, 2007 Interactive touchscreen game for teaching musical notes and music notation. Has virtual on-screen keyboard (musical) keyboard, not typewriter), screen rotation automatically rotates (musical) virtual keyboard and morphs into musical notation (staff with notes). If input is touch sensitive (pressure/force sensitive touchscreen), can check for volume (forte/piano) of user against pre-programmed music shown on display. Equates velocity sensing to touch (force) sensing for user striking piano keys. [Samsung07a] Samsung "Samsung UMPC Tablet-PC computer", Samsung press information from CeBIT 2007, March 15, 2007; seven-inch display on Samsung UMPC Tablet-PC: Samsung Q! Ultra with split keyboard for thumbs to left and right of display. [Screendirect07] Screendirect.com "Light Pen Basics - How they Work", www.screendirect.com/specsheets/LPbasics.html Operating and design of Light Pens [SensAble07a] SensAble Technologies "SensAble OpenHaptics Toolkit: datasheet", SensAble Technologies, www.sensable.com, 2004-2007 OpenHaptics Software toolkit for adding haptic feedback to touchscreen integreating with OpenGL graphics toolkit/API: , 3D/6DOF electromechanical touch input systems, and other devices, Force, Torque, stiffness feedback. See also PHANTOM 6DODF haptic feedback systems from SensAble. [SensAble07b] SensAble Technologies "SensAble Phantom Haptics Devices: data sheet", SensAble Technologies, www.sensable.com, 1993-2008 Phantom/PHANTOM Omni haptic devices: 6DOF haptic feedback electromechanical/stylus input device with force feedback, desktop and larger size working volume. Optional end effectors: scissors input, thumb-pad for 7 DOF input. [ShaoJ07] Shao, Joy; Fiering, Leslie; and Kort, Todd "Dataquest Insight: Tablet PCs Are Slowly Gaining Momentum", Gartner Reports, 6 April 2007: available at www.garner.com Tablet PC market growing: Windows Vista, which integrates tablet PC extensions, will drive tablet PC sales starting in 2008 [Sharp07a] Sharp Corporation "Sharp develops a new LCD touch screen system", Sharp Corporation press release, September 1, 2007 In-cell combined touchscreen and LCD with optical sensor in each pixel of LCD display panel, combined display/touchscreen/scanner. Multi-touch sensing, can also scan objects (business cards) placed on top of panel (e.g. business card), refers also to fingerprint authentication on touchscreen. [SignerB07a] Signer, Beat; Norrie, Moira C.; Kurmann, Ueli "iGesture: A Java Framework for the Development and Deployment of Stroke-Based Online Gesture Recognition Algorithms", ETH Zurich Technical Report TR561, September 2007 Java Framework for gesture UI development: includes utilities for testing/training recognizers, and automatically (?) determining gestures which might be mutually confusable. Implementations of extended version of Rubine recognizer eRubine: chain-code recognizer based on regular expressions SiGeR / SiGrid (compare Pencept and CIC), Graffiti, Microsoft Application gestures. Note: Not to be confused with iGesture by Fingerworks, technology acquired by Apple for the iPad. [SignerB07b] Signer, Beat; Kurmann, Ueli; Norrie, Moira C. "iGesture: A General Gesture Recognition Framework", Proc. ICDAR 9th Int'l. Conf. on Document Analysis and Recognition, Vol 2, pp. 954-958 Digital pen and paper user interface / interactive paper iPaper. See also iGesture.org, on-line resource. Trainable recognition for gestures, cites to Rubine and other earlier trainable systems. [SimaSystems07a] SiMa Systems Inc. "Technology Intro: Multi-Touch/Dual Force (MT/DF) activated touch sensor", SiMa Systems, July 2007 pressure (force) sensitive multi-touch: uses light touch to track, heavy touch to activate, so that user's fingers need not lift from touchscreen surface. Determines stylus contact by size of contact area. Backmounted touch on rear surface so that display is not obscured. [SluskyRD07a] Slusky, Ronald D. "Invention Analysis and Claiming: A Patent Lawyer's Guide", American Bar



Association, 2007 (hardcopy book) Tutorial/textbook on patent claiming techniques, suggestions on dependent claims [Softpedia06a] www.softpedia.com "xGestures", www.softpedia.com xGestures mouse gestures for Mac OS X 10.3: no visual feedback / electronic ink on gestures. Recognizes each separate motion as a sub-gesture, e.g. UpDown is an Up and Down composite gesture [Softpedia06b] www.softpedia.com "Synaptics and Pilotfish to Develop Next Generation Mobile Phone concept", www.softpedia.com, 22 August 2006 Synaptics clearpad capacitive digitizer: "recognizes points, taps, shapes, complex gestures, and proximity of user's finger or cheek" in a portable phone [SongH06] Song, Hyunyoung; Guimbretiere, Francois; and Hu, chang "ModelCraft: Capturing Freehand Annotations and Edits on Physical 3D Models", Proc. UIST '06, October 15-18, Montreux, Switzerland, 10 pages Place Anoto optical digitizer paper on a physical 3-D model, and digitize on the sides of the 3D model to edit a computer model of the same model. Two pens used: one for annotating (drawing) on the model, the other for editing (cutting) from the computer model. Uses gestures: pigtail using starting point as reference location, and "C" for cut using location inside an area, "E" gesture for extrude. Cites to Sekendur pen and to DataGlyph optical pattern digitizers. [StefikMJ07a] Stefik, Mark J.; Pirolli, Peter L.; and Merkle, Ralph C. "Digital Works having Usage Rights and Method for Creating the Same", United States Patent 7,225,160, May 29, 2007 DRM usage rights management, usage rights control what rendering/use is granted to a document or object. Usage rights use a structured language. [StefikMJ07b] Stefik, Mark J.; Pirolli, Peter L.; and Merkle, Ralph C. "Content Rendering Apparatus", United States Patent 7,269,576, September 11, 2007 DRM usage rights management, usage rights control what rendering/use is granted to a document or object, based on rights granted by a digital ticket and an authorized respository/storage. Compare with Benson certificate? [Strauss06] Straus, Wolfgang; Fleischmann, Monika "PointScreen: Interaktion ohne Berührung", netzspannung.org: MARS Projekte Gesture input whiteboard from Fraunhofer Institute, technology similar to Theremin electrostatic device. In-air gestures, capacitive sensing. [SuntronicLCD07a] Suntronic LCD, Inc. "Suntronic Outdoor LCD Signage", suntroniclcd.com/touch-tech.html, 2007 Large-format touchscreen LCD displays. Options include MicroTouch DST dispersive signal technology touchscreen, TouchScreen Solutions projected capacity, Turbotouch Resistive, E-resistive, Cleartek capacitive, Infrared touch screen (optical), IntelliTouch/SecureTouch acoustic SAW. [Synaptics07] Synaptics Incorporated "Product Information: PC & PC Peripheral Products", www.synaptics.com Synaptics ClearPad: capacitive-sensing touchscreen tablet digitizer technology, used in Microsoft UMPC tablet computers. Synaptics One-Touch controller chips for capacitive sensing touchscreens. ClearPad by Synaptics: refers to two-finger (multi-touch) touchscreen, and proximity sensing (with front-mounted digitizer? compare with Scriptel). General note of Onyx Concept concept platform. [TabletPCMagazine06] TabletPCMagazine "Press News from Tablet PC Magazine, from Pen Computing Magazine, March 21 2006", www.pencomputing.com/TabletPC Multiple pen/TabletPC computer products. eMusic music reader (e-book). PhatWare PhatPad note-taking notes organizer with voice notes (compare with Wang Freestyle, Anoto?). Sales force automation CRM applications for pen/touch computers. Microsoft Origami Ultra-Mobile PC UMPC. Andries van Dam at Brown University, Microsoft Center for Research on Pen-Centric Computing. Term "pen-centric" from Slate Inc. in 1992(?). Describes van Dam as early pioneer in pen-based computing. PenCentra UMPC Tablet PC product from Fujitsu Gateway eStand music reader electronic book with Tablet PC hardware (see Ellis?) OQO Tablet PC with Windows Mobile and Windows XP Tablet PC in same device: compare with Origami announcement by Microsoft, which states that Tablet PC 1.7 (?) includes both Tablet PC and Windows Mobile APIs Plastic Logic E-Ink 10-inch plastic E-Ink display: flexible display presented as like paper Synaptics Dual Mode TouchPad combines electrostatic digitizer and capacitive touchscreen (multi-touch) FinePoint digitizer: provenance is Mutoh America, took over Kurta, now spun out to FinePoint versus Wacom virtual monopoly on tablet digitizers. Dave Berque, Workshop on the Impact of Pen-based Technology on Education WIPTE, April 6-7 2006 at Purdue [Tactiva06] Tactiva "TactaPad", www.tactiva.com Two-handed (multi-touch) touchscreen with optical projection, images (shadows) of fingers are projected into graphical image and picked up by video camera: might be considered "virtual reality" interaction [TanenbaumAS06a] Tanenbaum, Andrew S. and Woodhull, Albert S. "Operating Systems Design and Implementation, 3rd Ed. -- The Minix Book", Prentice Hall, 2006 Upper-level course textbook on operating system design, example is Minix, minimal Unix implementation with small kernel, other facilities (e.g. file system) in server processes. ACLs access control lists stored by



row, or by column [TappertCC07a] Tappert, Charles C. and Ward, Jean R. "Pen-Centric Shorthand Handwriting Recognition Interfaces", First Int'l Workshop on Pen-Based Learning Technologies, Catania, Italy, May 24-25, pp C5.1-C5.6 Description of simplified writing styles: Papyrus Allegro alphabet Graffiti, Allen alphabet, Goldberg Unistroke, Stenographic, Stenographie, Moon, Tironian. Cites to chatroom abbreviations in pen computing. [TappertCC07b] Tappert, Charles C. and Cha, Sung-Hyuk "English Language Handwriting Recognition Interfaces", Chapter 6 in "Text entry Systems", MackKenzie and Tanaka-Ishii, eds., 2007 Review of handwriting recognition: on-line/off-line, discrete/continuous/connected, shorthand, Graffiti and Papyrus unistroke neographies, text editing with gestures. [TappertCC07c] Tappert, C.C. "Shorthand and Unistroke Alphabets and Pen Computing", (private correspondence) Draft paper, not for distribution [TaylorB06a] Taylor, Brian; Layton, Michael D.; and Taylor, David "System for disposing a proximity sensitive touchpad behind a mobile phone keypad", United States Patent 7,151,528, December 10, 2006 Capacitive-grid (mutual capacitance) touch/proximity sensor behind keys of cellphone keypad, physical keys drilled through holes in touchscreen, to combine pointing/scrolling into small hand-held device. Also separate touchpad to side of keyboard for dedicated scrolling input. Touchscreen material is flexible. Continuation of 6,680,731 and 6,730,863 [TechnologyReview07] Technology Review "To Market; Longhand goes Digital", Technology Review magazine, March/April 2008, page 36 Announcement of LiveScript Pluse Smartpen: Anoto optical digitizer pen using printed pattern on paper, plus synchronized voice recording in the pen. www.livescribe.com [Tekscan07a] Tekscan, Inc. "Tekscan Pressure and Force Measurement Technology", Tekscan Inc, South Buxton MA, 2007 (fetched 2012, www.tekscan.com) Force/pressure touch sensors using matrix of conductors and pressure-sensitive conductive ink, can sense multiple touch points by scanning each node [TenGO06a] Xrgomics, Inc. "TenGO User Manual Version 2.00", Xrgomics, February 09, 2006 Small-format touchscreen virtual keyboard for PDAs/smartphone. Combination of grouped keys (e.g. 26 keys of alphabetic divided into six groups). User types/taps on keys, dictionary match to words that could be spelled with letters of those keys. User can update dictionary. [TeskaK07a] Teska, Kirk "Patent Savvy for Managers - Spot and Protect Valuable Innovations in Your Company", Nolo Press, 2007 General introduction to patents, patenting process, business strategies [TheTeacher06] TheTeacher99 "HCI (Human Computer Interface)", www.theteacher99.btinternet.co.uk/theteacher/gcse/newgcse/module6/task11.htm Gives MS-DOS as example of command-line HCI user interface, as contrasted to menus, GUI, Speech interface [ThurrottP06a] Thurrott, Paul "Judge Dismisses GO's Antitrust Case Against Microsoft", www.windowsitpro.com, July 3, 2006 Statute of limitations makes private anti-trust lawsuit by Larry Kaplan of GO moot. [TofelKC07a] Tofel, Kevin C. "iPhone's kinetic scrolling comes to the Nokia N800", ijontherun.com: August 2, 2007 Third-party kinetic scrolling for Nokia N800 smart phone, similar to kinetic/dynamic scrolling behavior of iPhone [Toshiba06] Toshiba Matsushita Display Technology Company "Finger Touch and Optic Pen", www.tmdisplay.com: Press Release October 17 2006 "system on glass" finger touch and fiber optic pen on LCD display: unclear what digitizer technology is used for touchscreen. Development announced first in 2001. Later articles refer to system-on-glass as "incell" touchscreen. [TouchTable06] TouchTable "TouchTable Product Information", www.touchtable.com Commercial company with touch-sensitive table, appears similar to several projects, such as that of Microsoft. 2006. TouchTable GIS hand gestures translated into commonly used input and navigation commands -- similar to recognition macros of PenPad? [TownsendR06] Townsend, Reed "Touch Interface for Tablet PCs", WinHEC 2006 conference, available at www.microsoft.com Capacitive/electrostatic digitizers for Microsoft Vista Tablet PCs: respond both to finger touch and to stylus, just be able to handle both forms of input. Alternative, hardware to include both an electromagnetic (stylus) digitizer and resistive touch screen for touch. Refers to Microsoft "Tablet Driver Guidance" requirements. [TrontJG07] Tront, Joseph G. "Facilitating Pedagogical Practices through a Large-Scale Tablet PC Deployment", IEEE Computer, September 2007, Vol 40 Number 9 pp. 62..68 Usefulness of searchable (via deferred/delayed recognition) of digital ink / electronic ink notes in a



classroom. Teacher can mark-up papers submitted by student in electronic form. Special mention of electronic whiteboard system using OneNote for group discussion. Using OneNote note-tacking application from Microsoft on Tablet PC. See also AndersonR. [TruittT06] Truitt, Todd R. "Electronic Flight Data in Airport Traffic Control Towers: Literature Review", U.S. Department of Transportation, Federal Aviation Administration. NTIS / William J. Jughes Technical Center, DOT/FAA/CT-05/13 ATC Air traffic control user interfaces, includes references to French studies on gesture-based input on radar screen. General commentary is that handwriting recognition is undesirable due to cognitive load and errors, simple gestures on a touch screen are advantageous (flick gestures), portable and especially handheld pen computers for air traffic control are not helpful, in part because they are distracting and require two hands to operate. Mentions that portability of data (from one display/workstation to another) is more important than portable computers in air-traffic control. [TseE06a] Tse, Edward; Greenberg, Saul; Shen, Chia; and Forlines, Clifton "Multimodal Multiplayer Tabletop Gaming", Third International Workshop on Pervasive Gamin Application - PerGames 2006, May 7th, 2006, Dublin Ireland hand/finger gestures by multiple users on a digitizer tablet table top: Refers to sub-gesture and compound gestures. Refers to "wipe" (vs swipe) gesture using fist instead of fingertip, two-hand pile-and-browse gesture. Mitsubishi Research www.merl.com [TurnerD07] Turner, Daniel "The iPhone, Cracked Open", Technology Review, Vol 110 No 5, September/October 2007, pp 30-31 iPhone runs stripped-down Mac OX X operating system: multi-touch digitizer from Balda with opticalquality glass surface resists scratching [UMPC06a] Microsoft.com "Ultra-Mobile PC", www.microsoft.com/windowsxp/umpc/hardware.mspx, www.intel.com/go/umpc Hover/proximity sensing no longer required to get Tablet PC (UMPC) Logo from Windows: Contains recommendations on target area larger for touch-screen input (no proximity indicator) See also Origami Project. Origami/UMPC project director Otto Berkes: see notes on VMI vs. AutoCad trade-secret lawsuit [UMPC06b] New York Times "A Big Question Unanswered by a Tiny PC", New York Times, May 4 2006, p. C1 UMPC Ultra-Mobile PC touchscreen computer: Windows-base UI hard to use, especially text entry/typing with special thumb-menu pop-up keyboards in corners. See also Origami Project. Origami/UMPC project director Otto Berkes: see notes on VMI vs. AutoCad trade-secret lawsuit [UMPC06c] Mosberg, Walter S. (New York Times) "Two Tech Leaders Aim For Bold New Portable, but Miss The Mark", New York Times, May 4, 2006, p. B1 UMPC Ultra-Mobile PC touchscreen computer: complaints are no keyboard, short battery life, expensive, display/screen resolution too low. Compares with OQO and Tablet PCs. Does not mention pen-enabled vs. pencentry UI. [UnemotoS06a] Unemoto, Seiji; Noguchi, Tomonori; Kameyama, Tadayuki; Shimodaira, Kiichi; Sugawara, Hideo; and Andou, Hidehiko "Touch Type Liquid-Crystal Display Device and Input Detecting Method", United States Patent 6,982,432, January 3, 2006 LCD/flatpanel display with rear-mounted touch panel on the back, to form a touchscreen. Touchpad uses two grids of electrodes. [UyM06a] Uy, Michael "Integrated Sensing Display", United States Patent Application 2006/0007222, January 12, 2006 In-cell display "touchscreen": multiple optical elements, each with own lens (and potentially different focal lengths) interspersed in display pixels. Light source can be from some of the pixels. When user looks at screen, camera does not show gaze direction off to one side (towards a separate camera). [VanDenHovenEAWH06a] Van Den Noven, Elise Anna Walthera Hendrina and Eggen, Josephus Hubertus "Device and method of browsing an image collection", United States Patent 7,152,210, December 19, 2006 Photo browsing of an image collection using a flick or other stroke gesture. Current picture is shown in window, scrolling list may be on the side. Compare with Apple bounce-back feature? [VibeTonz07] VibeTonz "What's VibeTonz (R)", www.vibetonz.com hardware vibration component used to give tactile/haptic feedback on touch interfaces, such as an Apple iPhone or PDA for touchscreen interface. See also Stephen Brewster at U. Glasgow on vibrotactile icons. [Visigo07] Visigo "Visigo Task Pad product information", www.visigotaskpad.com Scriptel PDA running Windows CE in a Tablet-PC-sized unit: has additional function buttons, appears to be a notebook-sized / Tablet PC -sized PDA, longer battery life than most Tablet PCs. Product literature shows tethered stylus [VisionObjects06] Vision Objects "MyScript online handwriting recognition technology", www.visionobjects.com Handwriting recognition trained to western languages (Danish, Finish, Norwegian, English (UK), English (US), Italian, etc., Asian Languages Simplified Chinese, Japanese InkSearch application for searching



electronic ink notes (based on confidence score of handwriting recognition?) Personal correspondence with Adrienne Tommy [Wacom06a] Wacom "PenOffice: Wacom Shareware Archive", www.wacomeurope.com/uk/downloads/shareware/pen_office.asp PenOffice by Paragraph: electronic ink handwritten annotations in MS Word an e-mail, for pressure sensitive Wacom pens (tablets) [Wacom06b] Wacom "Funktionsweise UltraPen: Wie der kabel- und batterielose WACOM-Stift funktioniert", www.wacom-europe.com Description of cordless stylus from Wacom for electromagnetic digitizer: appears to be similar technology to that used in the rear-mounted digitizer for the OQO? [Wacom06c] Wacom "PL-720 Innovatives Arbeiten in Perfektion", www.wacom-europe.com Stand-alone tablet/display from Wacom. Rear-mounted electromagnetic digitizer. Battery-free and cablefree stylus. [Wacom06d] Wacom "Beweiskräftiges Unterschreiben ohne Papier", www.wacom-europe.com Wacom digitizer/display combination, article states that it is used for "legally binding" handwritten signatures. Reference to signature verification based on writing rhythm and writing speed/velocity (dynamics). Also uses writing pressure/tip-force? [Wacom07a] WACOM Co., Ltd. "The De Facto Standard for Pen Tablet Devices is Realized using EMR (R) Technology", www.wacom-components.com/english/technology/emr.html General description of Wacom electromagnetic / magnetostrictive digitizer tablets, and the statement that they are de facto the only available component for tablet pen computers: Penabled Technology, Wacom term for avoiding EMR magnetic field noise by placement of parts and customized adjustment of operating frequencies for each integration. Mentions need to block EMR magnetic field interference from backlighting and transformers, etc. [Waddington07] Waddington, R. Paul "Digitizer Puck", members.ozemail.com.au/~cadwest1/gallery/Digitizer_Puck.html Images of typical digitizer puck from 1980s. Notes that digitizer was clear transparent glass: Scriptel? [WaldnerM06a] Waldner, Manuela "Tangible Tiles: Ein Tangible User Interface zur Verwaltung und Manipulation von digitalen Daten", Diplomarbeit, Fachhochschule-Masterstudiengang Digitale Medien, Hagenberg, Germany, August 2006 Plexiglass tiles used as tangible user interface objects on a touch-sensitive surface with overhead projection onto desktop. Cites to Digital Desk by Wellner. [WalkerG06a] Walker, Geoff "Touching Origami: Microsoft's UMPC Platform", www.pencomputing.com, March 12, 2006. Previously published in "Touch Panel Newsletter" Review of Microsoft Ultra-Mobile Personal Computer UMPC initiative: basically a slightly-smaller Tablet PC with a touch-input digitizer instead of a stylus tablet digitizer, and requirement for proximity/hover sensing not enforced by Microsoft. N-Trig capacitive digitizer, cites to previous manufacturer unhappiness at being forced in practice to a single supplier for an electromagnetic digitizer (Wacom). [WalkerG06b] Walker, Geoff "Touch screen highlights from the SID 2006 show floor", Veritas et Visus / Touch Panel, June 2006, pp 19-26 Review of new large number of touchscreen vendors exhibiting at SID: 3M Touch not at show, because SID not a good marketing opportunity. Mostly resistive-film/capacitive touchscreens, Wacom electromagnetic digitizer dominates dominates Tablet PC, dual-mode combining two digitizer types. IRTouch showing multi-touch implementation on infrared (optical) touchscreen in software. Immersion haptic feedback. [WalkerG07a] Walker, Geoff "A Holistic View of Touch", Veritas et Visus / Touch Panel, April 2, 2007 Brief repeat overview of of touchscreen vendors: transparent and "opaque" (digitizer): includes Smart Technoligies, Fuji Chimera, Fujitsu, HanWang [WalkerG07b] Walker, Geoff "Last Word: Backpedaling in Microsoft-Land: The UMPC suddenly gets real", Veritas et Visus / Touch Panel, May, 2007, pp. 49-50 Microsoft UMPC, contrasted with other ultra-mobile handheld PCs with touchscreens: FlipStart palmtop, Sony U50/U80 tablets, OQO Model 1+, portable PCs [WangCH07a] Wang, Chun-Hao; Hwang, Chiu-Pei; Wang, Wen-Tsung; and Chu, Sung-Hao "Touch Panel Fabrication Method", United States Patent Application 2007/0238294, Published October 11, 2007 Construction method for opaque touch panel by laminating layers [WangJ06a] Wang, Jian and Zhang, Chunhui "Optical System Design for a Universal Computing Device", United States Patent 7,133,031 B2, November 7, 2006 Microsoft Optical digitizer using rectangular pattern printed on paper: very extensive references to Anoto dot pattern technology from internet print-outs. Mentions including other sensors (accelerometer?) in case the printed pattern is not available, mentions ability to identify individual users (by encoding of stylus?) for



simultaneous input from multiple users. [WangJ06b] Wang, Jian; Chen, Liyong; Dang, Yingnong, and Ma, Xinoxu "Cmera-pen-tip mapping and calibration", United States Patent 7,136,054 B2, November 14, 2006 Microsoft Optical digitizer using rectangular pattern printed on paper. No citations to Anoto. [WangJ07a] Wang, Jian and Zhang, Chunhui "Universal Computing Device for Surface Applications", United States Patent 7,262,764 B2, August 28, 2007 Microsoft Optical digitizer using rectangular pattern printed on paper: very extensive references to Anoto dot pattern technology from internet print-outs. Mentions including other sensors (accelerometer?) in case the printed pattern is not available, mentions ability to identify individual users (by encoding of stylus?) for simultaneous input from multiple users. Includes instrumentation of a whiteboard eraser, with printed pattern over a conventional whiteboard. [WardJR06a] Ward, Jean R.; Barrett, David M.; Martin, Patricia A.; Mokoski; Christopher D. "User interface having simulated devices", United States Patent 5,491,496, February 13, 1996 Touchscreen/digitizer user interface using simulated devices: virtual keyboard, virtual mouse, virtual tablet. Describes highlighting/selecting an object on touch, but not carrying out action until lift or mouse-up, and to selecting a field for writing on first touch. Refers to emulating mouse by using entire touchscreen, but in relative mode not in absolute mode. [WardJR07a] Ward, Jean Renard "Detection of system Compromise by Correlation of Information Objects", United States Patent Application Publication 2007/0067623 A1, March, 22, 2007 Detection of rootkits or other system compromise by means of checking for violation of invariant conditions in real time, rather than the traditional "two views" scanning method of most rootkit detectors. One example is instrumentation of the Windows OS kernel. See also Ian Main application on similar approach to network behavior. [Waring06] Wacom "Product Review: OQO model 01 + Handtop PC", www.jiwire.com, February 17, 2006 OQO / Wacom tablet: refers to awkward need to hold Wacom stylus at 90-degree angle to digitizer. Refers to Wacom as touch screen, does not mention proximity sensing. [WeaverB06] Weaver, Barbara "The Case of the Missing Ink", Keynote address, WIPTE 2006: Workshop on the Impact of Pen-based Technology on Education Notetaking by students is a lost learning/teaching art with the advent of laptop computers. Examples of educational use of electronic ink in notetaking, grading, and teaching using Microsoft OneNote. Appears to be largely a replication of taking notes on paper. [WesselD07a] Wessel, David; Avizienis, Rimas; Freed, Adrian; and Wright, Matthew "A Force Sensitive Multi-touch Array Supporting Multiple 2-D Musical Control Structures", Proc. NIME07, 2007 Conf. on New Interfaces for Musical Expression, New York, 2007 Force-sensitive multi-touch touchpad arrays for musical expression. Array of 24 to 2x16 pads (small forcesensitive touchpads) in brick-wall overlapping arrangement. Each pad is X/Y/force VersaPad touchpad from Interlink Electronics. States that VersaPad uses resistive (?) material between two conductive layers: resistance between layers gives measure of force, but description sounds ore like capacitive touchpad with a compressible material. multi-touch touchpad array: haptic regularization -- feel/haptic feedback to musical performer is important. Because force response of Versapad is very non-linear, must be corrected to user-dependent perception. [WestermanW06a] Westerman, Wayne Carl and Elias, John G. "System and Method for Packing Multi-Touch Gestures onto a Hand", United States Patent 7,030,861, April 18, 2006 Multi-touch gestures with hand neutral (closed) and hand open, for twice as many chorded gestures. E.g. hand neutral four fingers for scroll, hand open four fingers for page up/down. [WigdorD07a] Wigdor, Daniel; Forlines, Clifton; Baudisch, Patrick; Barnwell, John; and Shen, Chia "LucidTouch: A See-Through Mobile Device", Proc. UIST '07, October 7-10, 2007, Newport, USA transparent/translucent display and digitizer, pseudo-transparent device, user interface involves touching on rear of device with fingers visible by image superimposed over display image on front. Stated to solve fat-finger (poor resolution) problem and occlusion problem. Display targeting point of fat finger image as aid in targetting See also video on file. [Wiki-how-help07] wiki-how-help.com "Tablet PcSale: list of business reports/DOS", wiki-how-help.com, fetched 2008 List of business reports on Tablet PC sales: Vista to spur tablet PC sales [Wikibooks06] Wikibooks "A Neutral Look at Operating Systems/DOS", en.wikibooks.org/wiki/A_Neutral_Look_at_Operating_Systems_DOS.html MS-DOS, PC-DOS, DR-DOS, freeDOS as keyboard only UI without a GUI interface [Wikipedia06a] Wikipedia.org "Fuzzy set", en.wikipedia.org/wiki/Fuzzy_sets Definition of Fuzzy set, many-valued logics, theory of possibility, uncertainty in engineering [Wikipedia06b] Wikipedia.org "Graffiti (Palm OS)", en.wikipedia.org/wiki/Graffiti_(Palm_OS)



Description of Graffiti and Xerox lawsuit concerning Goldberg patent 5,596,656. Xerox had demonstrated neographic Unistrokes to Palm before Graffiti. Jot handwriting recognition from CIC replaced Graffiti. Symbian UIQ pen computing tablet platform. [Wikipedia07a] de.Wikipedia.org "Lichtgriffel", de.wikipedia.org/wiki/Lichtgriffel Description of Light Pen technology: only works with CRT displays due to refresh frames, not with LCDs or other flat panel displays. (Lichtgriffel) [Wikipedia07b] en.Wikipedia.org "Memex", en.wikipedia.org/wiki/Memex Description of Memex by Vannevar Bush: mentions lack of citation to Goldberg and Townsend previous inventions. Voice/speech recognition, but not handwriting recognition. State that Memox did *not* have Hyptertext links, contrary to other historians. [Wikipedia07c] en.Wikipedia.org "Intelligent Machines Research Corporation", en.wikipedia.org/wiki/Intelligent_Machines_Research_Corporation 1951 OCR character recognition system in 1951 on Gismo [Wikipedia07d] en.Wikipedia.org "Neonode", en.wikipedia.org/wiki/Neonode optical digitizer for Windows CE palm-top computing: infrared light beams from LEDs, implies no handwriting recognition with reference to keyboard. See also Neonode file. [Wikipedia07e] en.Wikipedia.org "Sketchpad", en.wikipedia.org/wiki/Sketchpad General reference on Sketchpad drawing program by Sutherland in 1960s: mentions also T-Square by Peter Samson from 1962. [WillemsDJM06a] Willems, Don J.J. and Vuurpijl, Louis G. "Pen gestures in online map and photograph annotation tasks", Proc. Tenth Int'l. Workshop on Frontiers in Handwriting Recognition, Oct 2006, La Baule (France) Annotation of maps (marking house, humans, illustrations, handwriting recognition) classifying inputs into heirarchy of classes: dietic gestures vs. annotaions and other markings. A bit vague on how recognition was done. [WinTab06] Wacom "Notes for Tablet Aware Application Developers", www.wacomeng.com/devsupport/imbpc/gddevpc.html Documentation on Wintab Application Programmers Interface / Wintab API. Mentions Stylus vs. Airbrush vs. 4D Mouse vs. 5 button puck. Additional thumbwheel on multi-button 3-button puck for additional manipulation UI [WinfieldL07a] Winfield, Laura; Glassmire, John; Colgate, J. Edward; and Peshkin, Michael "T-PaD: Tactile Pattern Display through Variable Friction Reduction", Proc. EuroHaptics 2007 / World Haptics 2007, Jnt. Conf. and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Tsukaba, Japan, pp. 421-426 Haptic feedback on touchscreen using ultrasonic vibration, which reduces friction on surface but not perceived as vibration. Not multi-touch. Friction can simulate bumps and textures, but only when finger is moving. Simulate friction that varies with direction of motion, or speed, etc. [WobbrockJO06a] Wobbrock, Jacob O; Myers, Brad A.; and Chau, Duen Hong "In-stroke Word Completion", Proc. UIST ' 06, October 15-18, 2006, Montreux, Switzerland Word-level unistrokes, compares with Quikwriting, custom short-hand for pen-computing. Mentions shorthand unistroke based on Graffiti, EdgeWriting. [WobbrockJO06b] Wobbrock, Jacob O. and Myers, Brad A. "From Letters to Words: Efficient Stroke-based Word Completion for Trackball Text Entry", Proc. ASSETS '06, October 22-25, Portland OR EdgeWrite unistroke/graffiti gesture/character recognition using joystick input, with auto-completion / suggested completion of words at the corners. Similar to approach for Trackball EdgeWrite. User moves to edge limits when making character, somewhat similar to zone-based recognition. [WobbrockJO07a] Wobbrock, Jacob O.; Chau, Duen Horng; and Myers, Brad A. "An Alternative to Push, Press, and Tap-tap-tap: Gesturing on an Isometric Joystick for Mobile Phone Text Entry", Proc. CHI 2007, april 2007, San Jose, pp. 667-676 EdgeWrite unistroke/graffiti gesture/character recognition using joystick input, similar to approach for Trackball EdgeWrite. User moves to edge limits when making character, somewhat similar to zone-based recognition. [WobbrockJO07b] Wobbrock, Jacob O.; Wilson, Andrew D.; and Li, Yang "Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes", Proc. UIST'07, October 7-10, 2007, Newport, Rhode Island Simple template-matching recognizer for unistroke gestures, intented for use in small environments (e.g. in JavaScript where other recognizer not available). Comparison with Rubine, dynamic time warping DTW. Notes that medium-speed "accurate" gestures recognized better in all systems than slow or fast writing. Appears to deal in part with resampling for distance. Single-stroke arrow gesture, no flicks. [WuM06] Wu, Mike; Shen, Chia; Ryall, Kathy; Forlines, Clifton; and Balakrishnan, Ravin "Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces", IEEE International Workshop on Horizontal Interactive Human-Computer systems (TableTop), January 2006, pp 183..190



hand/finger gestures by multiple users on a digitizer tablet table top: Refers to sub-gesture and compound gestures. Refers to "wipe" gesture using fist instead of fingertip, two-hand pile-and-browse gesture. Mitsubishi Research www.merl.com see 2005 tech report Two-handed/bimanual gestures: Wipe Gesture (vs swipe gesture) (time based, not simple erase), Annotate (hold down with one finger, write with other), Cut/Copy-and-paste, Pile-and-Browse gestures. Uses DiamondTouch table in prototype. [Xerox06] Xerox/Palm: news article "Technology lawsuit vs. Palm, 3Com is reinstated", Boston Globe, June 9, 2006, Page E2, "Business in Brief" Xerox/Palm/3Com patent case on Unistroke/Graffiti ordered by appeals court to be reconsidered [XieX07a] Xie, Xiaofang "On the Recognition of Handwritten Mathematical Symbols", Ph.D. dissertation, U. of Western Ontario, London, Ontario December 2007 ink-stroke data (dynamic/on-line with tablet) recognition of mathematical symbols and expressions. Recognition based on HMM grouping of mark lengths, or mark curvature. De-hooking, smoothing. Cites to Ward/Kuklinski variability model. [YamamotoM07] Yamamoto, Mike "Rumor: Is Apple tablet in the works?", crave.cnet.com, November 6 2007 Crave UK posting on prototype tablet PC / pen-computer slate computer from Apple [YoshidaJ07] Yoshida, Junko "iPod's lesson: please touch", EE Times, December 12 2005 zForce optical digitizer from Neonode using infrared light beams for palm-top pen computer: implies low resolution of "several times a second"? [ZDNet07a] ZDNet "Microsoft hit with $1.5 billion patent verdict", www.zdnet.com, February 22, 2007 Jury award in Alcatel-Lucent v. Microsoft patent suite, MP3 technology. Refers to MP3 license from Fraunhofer. [ZDNet07b] ZDNet "Gallery: Cracking open the iPod Nano", www.zdnet.com Photographs of disassembly of an Apple iPod: shows Synaptics components for the butten-and-wheel interface for the iPod -- touch screen technology? [ZeleznikR06a] Zeleznik, Robert C.; Herndon, Kenneth P.; and Hughes, John F. "SKETCH: an interface for sketching 3D Scenes", SIGGRAPH 06, 2006 Courses 3D drawing interfaces based on simplified line drawings of sketch primitives. Velocity-based gestures: clickand-release (tap?), drag without pause, drag followed by tear, click near window boundary (context) followed by drag, lasso to select. Includes macro (?) capability for repeating gesture operations. [ZeleznikR06b] Zeleznik, Robert and Miller, Timothy "Fluid Inking: Augmenting the Medium of Free-Form Inking with Gestures", Proc. Graphics Interface 2006, pp. 155-162 Experimental drawing system using touchscreen gestures: combination of free-flow ink, press-and-hold to bring up marking menus, marking menus contain pictures of shape gestures that may also be used. Shape gestures recognized before ink, one example of ink drawing (smiley face) that may be mis-recognized as circle gesture. Also uses press-hold at end of a stroke gesture to terminate gesture. [Zhai06] Zhai, Shumin "Das grosse Wedeln", Bild der Wissenschaft, February 2006, p. 97 (in German) Non-character recognition ShapeWriter: pop-up keyboard with letters and characters organized so that words can be input by noting the "corner points" where the stylus is paused or changes direction. Makes use of a disambiguating dictionary, pops up a list of probable word matches [Zizzi06] Zizzi, Stephen "Encrypting File System", United States Patent 7,096,358, August 22, 2006 Encrypting file system, encryption is transparent to the user by operating on / intercepting file open or file change, various encryption algorithms can be used. biblio05 (Prev page)

Main Page







Main Page


[AFB10a] American Foundation for the Blind "Assistive Technology Timeline", www.afb.org, 2010 History of talking computers, DOS and Windows Screen readers, and other assistive technologies: Enhanced PC Talking Program by Ronald Hutchinson worked with word processors and spreadsheets 1984, outSPOKEN for Macintosh 1989, ASAP Automatic Screen Access Program "with artificial intelligence" 1990. Virtual keyboards? On-screen keyboards. [Acer10a] Acer "Acer Iconia: Dual-Display-Notebook", www.computerbild.de, 24 November 2010 Dual-touchscreen laptop with virtual keyboard or other elements in one side, regular display in other. Compare with Refalo, Microsoft Courier. Dual-touchscreen. Additional references in file. [Adapx08a] Adapx.com "Adapx Digital Paper", www.adapx.com, fetched 2008 Paper forms for Anoto-type optical digitizer pen: mentions 1200 dpi resolution of color printers certified by Anoto. Capturx capture software to upload electronic ink data directly into Microsoft OneNote, ArcGIS 9.2 Desktop geographical mapping software, and AutoCad. "Rite in the Rain" all-weather waterproof writing paper. [AllInAccess10a] AllInAccess.com "All in Access", allinacces.com/happ, 2010 Web site with archived files for DOS and other screen readers and accessibility software: include Hal Lite, ScreenPower, ASAP, Tinytalk, Proxy 7, Vocal Eyes. Also Speech synthesizers such as Dectalk. Contact addresses (?) [email protected], or David Holladay [email protected] [AllanA10a] Allan, Alasdair "Learning iPhone Programming: From Xcode to App Store", O'Reilly Publishers, 2010 (hardcopy book) Physical Book: Development guide for iPhone (iPod, iPad) devices: Compass, Proximity, Accelerometer sensors, Objective-C, Tablet-based view controller objects, etc. [AmemiyaT09a] Amemiya, Tomohiro and Sugiyama, Hiashi "Haptic Handheld Wayfinder with Pseudo-Attraction Force for Pedestrians with Visual Impairments", Proc. ASSETS '09, Cotboer 25..28, 2009, Pittsburgh, pp 107..114 Haptic direction indicator transducer gives illusion of a force/pull in one direction. Two-dimensional tactile force display. Oscillates/vibrates with asymmetric acceleration: speed fast in one direction giving sensation of pushing/pulling, slowly in opposite direction below sensing threshold. Appliaction as direction indicator



on a guide cane for the blind/visually impaired. Prototype uses OQO touchscreen hand-held computer. [AmemiyaT10a] Amemiya, Tomohiro and Sugiyama, Hiashi "Orienting Kinesthetically: A Haptic Handheld Wayfinder for People with Visual Impairments", ACM Trans. on Accessiblel Computing, Vol 3 No 2 Article 6, November 2010 Haptic direction indicator transducer gives illusion of a force/pull in one direction, used as guide display for the blind instead of audible cues (which could be problematic in noise environments, etc.). Mechanical transducer (slider/crank). Later papers have application to haptic output in hand-held smartphones. [AnalogDevices10a] Analog Devices, Inc. "Application Notes AN-925 Sensors for the AD7147 and AD7148 CapTouch Controllers", Analog Devices www.analog.com, 2010 Capacitive-to-digital A/D for single-electrode capacitive sensors (capacitance to ground). Intended for touch buttons, sliders, wheels, and matrix keypads (compare touchscreens). Matrix keypad, each button has two electrodes (connected to row and to column lines), single key detected by row + column ID. Eight sensor inputs, so up to 4x4 keypad. [AnanthaswamyA09] Ananthaswamy, Anil "Vibrating touch screen puts Braille at the Fingertips", New Scientist Magazine, 31 March 2009 Accessibility technology: Haptic display device for Nokia haptic touch-screen with vibration (similar to iPod/iPhone) devices: six pulse/no-pulse signals via vibration corresponding to the six dots of a Braille character. [AndreBK10a] Andre, Bartley K; Ording, Bas; and Christie, Greg "Activating Virtual Keys of a Touch-Screen Virtual Keyboard", United States Patent 7,844,914, November 30, 2010 virtual/soft keyboard on touchscreen: find centroid of finger touch, and map it to (weighted distance) nearest key. Compare with stroke targeting/ink targeting in PenPoint, Tappert, and Slate PenApps? [Anoto09a] Anoto AB "Streaming Evaluation Kit", Anoto.com, fetched 2009 Anoto digitizer pen with no inboard storage, all data sent in real time to hose computer via Bluetooth. Notes use of unique pattern on each printed page, allowing page to be identified, not just location. [Apple08a] Apple.com "iPhone: Using the Multi-touch Display / Entering Text on the Multi-touch Display Keyboard", support.apple.com, July 10, 2008 iPhone/iPad gestures: flicking and dragging gestures does not select (press-and-hold or press gesture). Tapping halts dynamic zoom. Pinch and zoom gestures. Press to get magnified view of text while editing. Note: does not cover accessibility gestures, such as triple-flick. [Apple09a] Apple.com "iPhone User Guide for iPhone OS 3.1 Software", www.apple.com, 2009 User Guide for iPhone: zoom/pinch, scroll, flick gestures on touchscreen, on-screen (virtual) keyboard: Japanese/Chinese keyboards show alternative characters to pick from: Pinyin and Zhuyin. Writing area for handwriting recognition of Simplified or Traditional Chinese handwriting. VoiceOver accessibility. [Apple09b] Krazit, Tom "Apple awarded key iPhone multitouch patent", CENT News, January 26, 2009 Apple v. Palm, and other vendors: Apple patent with Westerman (of Fingerworks) on multiple iPhone features, not just multitouch touchscreen [Apple09c] Patel, Nilay "Apple vs. Palm: the in-depth analysis", engadget.com, January 28, 2009 Apple v. Palm lawsuit: Palm Pre uses same multi-touch gestures as Apple iPad, article reviews multiple patents in litigation: telephone/address book, screen dimming, etc. [Apple09d] Marco Zehe EDV-Beratung "Erstes Anfühlen von VoiceOver auf dem iPhone", zehe-edv.de, Blog Gedanken und Kommentare rund um Barrierefreiheit in der Softwarebranche Review of VoiceOver screen reader accesibility on iPad, particular note that it is a built-in feature, unlike more expensive screen readers [Apple10a] Apple.com "It's easy to be good with Numbers", www.apple.com, April 2010 iWorks spreadsheet for iPad: refers to swipe and tap gestures [Apple10b] Apple.com "VoiceOver: Für blinde und sehbehinderte Menschen hat Apple eine Lösung in jeden Mac integriert", www.apple.com, October 2010 (in German) Review of VoiceOver screen reader software for MacIntosh with multi-touch trackpad: mentions "rotor" circle/rotational gesture to bring up navigation tools and command input (compare Zoom /spiral on Nokia). Supports up to 32-line braille (braillezeiler) output, 1 on Bluetooth, 31 on USB. [Apple10c] Apple.com "Apple - Accessibility - iPhone - Vision", www.apple.com, fetched 2010 VoiceOver accessibility screen-reader software. Mentions voice prompting on what gesture is appropriate: Rotor gesture with two fingers to change scope of other features: flick to move by pages or lines, etc. Double-tap with two or three fingers. [Apple10d] Apple.com "Apple - Accessibility - VoiceOver - In Depth", www.apple.com, fetched 2010 VoiceOver accessibility screen-reader software. Drag of finger (or two) over item speaks the item. Gestures may be assigned new semantics: preferences may be stored removed on a server. [Apple10e] Zedomax.com "Apple sues HTC Apple slide bar blog posting", zedomax.com, fetched 2010 Web posting on Apple/HTC/Google patent litigation, and blog posting on bounce-back on scroll being a



feature of MacIntosh screen enlargers (accessibility) about 1990. [ArrasK09a] Arras, Kai "Octave/Matlab Tutorial", Social Robotics Lab, Universitatet Freiburg, Germany, October 2009 Matlab/Octave tutorial: Octave is Matlab clone, interactive mathematical graphing and data calculation utility. [ArringtonM08a] Arrington, Michael "Microsoft TouchWall can inexpensively turn any flat surface into a multitouch display", www.crunchgear.com, May 14th, 2008 Microsoft TouchWall, three-laser sensor for multi touch on any flat surface, project display. Described as distinct from Microsoft Surface, which uses cameras to recognize objects placed on it (e.g. a cell phone, Surface reads photos from cell phone). See also CrunchPad about Arrington [AscensionTechnology09a] Ascension Technology Corporation "Ascension Technology Corporation 3D Guidance trakSTAR Installation and Operation Guide, Rev. D", Ascension Technology Corporation, September 22, 2009 3D/6DOF DC magnetic position sensor, compare with Polhemus. Has Wide-Range transmitter (base station) option, senses up to 137 cm from transmitter sensor for high accuracy. [AscensionTechnology10a] Ascension Technology Corporation "Ascension Technology Corporation 3D Guidance trakSTAR2 Installation and Operation Guide, Rev. 2", Ascension Technology Corporation, 2010 3D/6DOF DC magnetic position sensor, compare with Polhemus. [AshbrookD10a] Ashbrook, Daniel and Starner, Thad "MAGIC: A Motion Gesture Design Tool", Proc. CHI 2010, April 10-15, Atlanta. 10 pages Trainable gesture recognition system by example. Includes some automation for false positive and recognition error rates. Input device is three-axis accelerometer, like a Wii. [AshtianiB09a] Astiani, Behrooz and Stuerzlinger, Wolfgang "XNT: Object transformation on multi-touch surfaces", Poster paper, Interactive Tabletops and Surfaces (ACM), 2009 Three-finger translate/scale/pinch/zoom/rotate/move gesture instead of two-finger pinch/zoom: more stable hand posture, improves precision and control. Touchscreen is touch glove with LED light sources on fingertips, detected optically when touch surface with diffuser glass. [Asus10a] Asus "Tablets von Asus und Co.: Hybrid tablet/laptop", speigel.de, fetched 2011, CSS product report Tablet PC / laptop hybrid: tablet PC slate computer fits into top "cover" shell of keyboard. No wired connection to keyboard. [AtkinsonP08a] Atkinson, Paul "A bitter pill to swallow: the rise and fall of the tablet computer", Design Issues: History, Theory, Criticism; MIT Press, Vol 24 No 4, 2008, pp. 3-25 Negative historical review of tablet computer / pen computing, written slightly before Apple iPad, so does not mention iPad/Android tablet success. Characterizes PenPoint as "vaporware", mentions Pen Windows, but no citations to Marlin Eller history (?). Analysis from social construct theory: e.g. "truth" as socially constructed interpretation, technological success as within a socially constructed system. Based on Ph.D. Publication. Mentions Nestor for handwriting recognition (which had no products), but not Pencept or CIC (which did have products). [AtkinsonP08b] Atkinson, Paul "On the Origin of PCs", Submitted for PhD by Publication, U. Huddersfield, April 2008 Summary of publications submitted for PhD by Publication: Computer Memories: The History of Computer Form; The (In)Difference Engine: Explaining the Disappearance of Diversity in the Design of the Personal Computer; The Computer Mouse in the History of Computing: A Bitter Pill to Swallow: The Failure of the Tablet Computer History of computers (personal computers /PCs) from social construction perspective: effect of social drivers (prestige, status, popularization in advertizing and movies) on computer design evolution. [Atmel08a] Atmel Inc. "Amtel's New Family of Touch Screen Solutions Enable Two Touch Gestures for Intuitive user Interfaces", www.atmel.com, press release, October 22, 2008 Amtel: formerly Quantum Research Group. QTouch touchscreen. Capacitive sensing, statement about detecting two touches, refers to multi-touch. Single layer of ITO (front surface?). Discusses cross-talk from adjacent sensors, use strongest signal. Additional material in files: Quantum acquired by Amtel: see Quantum file. [Atmel08b] Atmel Inc. "Touch Sensors Design Guide", www.atmel.com Amtel: formerly Quantum Research Group. QTouch touchscreen. Extensive discussion of functioning of capacitive touch sensors. [Atmel08c] Atmel Inc. "Amtel Touch Library User Guide", www.atmel.com Amtel: formerly Quantum Research Group. QTouch touchscreen software library, includes hooks for software filtering of inputs (not standardized?) [Atmel09a] Atmel Inc. "Atmel QMatrix Touch Technology", www.atmel.com Amtel: formerly Quantum Research Group. QTouch touchscreen. Capacitive sensing, statement about detecting two touches, refers to multi-touch. QMatrix touch sensor IC devices: scan touchscreen keys



consisting of passive matrix of copper or transparent ITO indium in oxide, differential detection to reject capacitive pickup on adjacent keys. States spread spectrum signals to reduce interference. Shows electrode configurations including serpentine and spiral. QMatrix touch sensor IC devices: scan touchscreen keys consisting of passive matrix of copper or transparent ITO indium in oxide, differential detection to reject capacitive pickup on adjacent keys. States spread spectrum signals to reduce interference. [AutioMT09a] Autio, Markku Autlo and Jaervioe, Jami Jarkko Juhani "Graphical Functions by Gestures", United States Patent 7,623,119, November 24, 2009 Gesture on an inactive area (not an icon, link, scrollbar, etc.) is processed as a command if the handwritten character matches character. Compare with PenPoint. [BaileyRPS09a] Bailey, Ralph-Peter Steven "Haptic Input Devices", United States Patent 7,499.021, March 3, 2009 Multi-pod (hexapod/pentapod) linkage mechanism for force feedback using magneto-restrictive fluid brakes. Has squeezable (pressure, but not tip-force) stylus for input. [BarnkowL10a] Barnkow, Lorenz "Eine Multitouch-fähige Küchentheke: Im Kontext des Living Place Hamburg", Hochschule für Angewandte Wissenschaften Hamburg, 2010 Report on multi-touch-enabled kitchen in experimental setting: multi-touch touchscreen / surface using under-the-table optical tracking and internal reflection from finger contact. Cites to NUI Group Community for construction of multi-touch surface. [BarrettG10a] Barrett, Gary L. and Omote, Ryomei "Projected-Capacitance Touch Technology", Information Display, March 2010, pp. 16-21 Overview of capacitive touch tablet technology. "grip supression" for handheld smartphones, similar to palm rejection. Describes true multi-touch as possible in multi-pad self-capacitance sensor and in intersection grid mutual capacitances. [BarrettG10b] Barrett, Gary "Putting the "Touch" in Multi-Touch: An in-depth look at the future of interactivity", www.veritasesvisus.com, Touch Panel, December 2010, pp. 59-61 Brief overview of several multi-touch technologies: describes projected capacitive as multi-touch zones (compare with Kaplow?), multi-touch input seen by applications (in most OSs) as multiple mouse/locator devices. States only three gestures unique to multi-touch (???): pinch, expand, rotate. Asserts that first iPad / iPhone touchscreens were resistive film with "ghosting" for multi-touch. [BarrettG10z] Barrett, Gary L. and Omote, Ryomei "Projected Capacitive Touch Screens", Whitepaper, Touch International Appears to be alternative/revised/version of 2010 SID article by same authors on projective/capacitive touchscreen. One difference is refering to glass vs. plastic substrates, plastic flexible and can be curved. (Microcracks?) [BatesJP09a] Bates, John P.; White, Payton R.; and Takeda, Yutaka "Predictive User Interface", United States Patent 7,498,147, February 3, 2009 Spelling predictor in touchscreen user interface (or icon predictor), highlight "most likely" completions. Shows circular alphabetic arrangement of letters. Compare with IBM Simon, QuikWriting. [BauO10a] Bau, Olivier; Poupyrev, Ivan; Israr, Ali, and Harrison, Chris "TeslaTouch: Electrovibration for Touch Surfaces", Proc. UIST '10, October 3..6, 2010, New York, Comparison vibrotactile (moving parts) haptic feedback/display with electrovibration (modulated electrostatic sheet attracts/repels fingertip, changing fricing). Single sheet of transparent conductor with AC signal applied. Vibration frequencies for sticky/waxy/bumpy/smooth sensations. Sensation is only for moving fingertips in contact: used to simulated multi-touch feedback approximately. [BaudischP08a] Baudisch, Patrick "Lucid touch microsoft research and mitsubishi electric research labs", research.microsoft.com, 2008 Web posting and video (see Video file of LucidTouch) of pseudo-translucency touchscreen user interface: rear-mounted second touchscreen on handheld PDA-type device, shows simulated outline of fingers on front touchscreen display. Multi-finger/multi-touch input, rear touching does not obscure/block view of display. [BaudischP09a] Baudisch, Patrick and Chu, Gerry "Back-of-Device Interaction Allows Creating Very Small Touch Devices", Proc. CHI 2009, April 4..9, 2009, Boston Small touchpad on back of display of cellphone: no fat finger problem, no blocked view (by finger) because can show precise point of touch. Prototype uses (non-transparent) capacitive trackpad. Also two thumb buttons on edges of device. No multitouch by intent. [BeavenDF10a] Beaven, Douglas F.; Ahlgren, Janet; Kelley, Kevin; Palmer, Gorham; and Rudolph, Stuart E. "System for Performing Collaborative Tasks", United States Patent 7,698,160 Graphical user interface allowing users to combine information from multiple internet and other sources into single web page and share information with others. Data from multiple sources can be combined based on keywords, field names, or other attributes present in the data. Patent assigned to Virtual Agility. [BeinhauerW09a] Beinhauer, Wolfgang and Hipp, Cornelia "Using Acoustic Landscapes for the Evaluation of



Multimodal Mobile Applications", Proc. 13th Int'l. Conf. on Human Computer Interaction HCI International 2009, San Diego, CA, USA, July 19-24, 2009, pp. 3-11 Simulated usability test environment for accessibility of mobile computing mobility aids (e.g. sonar mounted on cane or wrist): multi-channel (more than stereo or 5.1) sound simulation [BenyonD10a] Benyon, David "Designing Interactive Systems, Second Edition: A comprehensive guide to HCI and interaction design", Addison Wesley / Pearson Education Limited , www.pearsoned.co.uk, 2010 textbook on interactive user interface design, espcially for group/CSCW and social interaction. Multi-touch and haptics consideration. iTunes as important to iPhone success as multi-touch/gesture user interface. HP iPAQ Pocket computer. Samsun i7410 Sirius projector phone. [BiaforeB10a] Biafore, Bonnie "Microsoft Project 2010: The Missing Manual", O'Reilly, 2010 Microsoft Project collaborative work system: includes CPM critical path method, Gantt charts, resource and schedule allocation views, custom fields. Agile project plans. Customizing ribbon UI. [Billinghurst09a] Billinghurst, Mark and Buxton, William "Gesture based Interaction: Chapter 14 of "Haptic Input" Draft Documents", www.billbuxton.com Gesture defined as hand gesture ("a motion of the body that contain contains information", not as handwriting/shape gesture for pen-computing (cites to Kurtenbach90 for definition). Mentions Theremin as gesture-input device (non-computer). Mentions "put that there" (Bolt 84). Draft of May 27, 2009: File also contains 2002 Draft [BishopT10a] Bishop, Todd "Dead 'Courier' walking: Microsoft gets dual-screen design patent", techflash.com, June 29, 2010 Dual-screen digitizer tablet by Microsoft: 3 Megapixel camera, compared with One Laptop Per Child XO-2 and MoleSkin. additional references on Courier in file. Compare with Refalo? File has additional materials. [BlackS09a] Black, Sue; Boca, Paul P., Bowen, Jonathan P.; Gorman, Jason; Hinchey, Mike "Formal versus Agile: survival of the Fittest?", ICCC Computer, September 2009, pp. 37-ff Review of Agile versus Formal development methods: refers to writing tests before implementation in Agile development. Numerous references, argues that formal proof of correctness and automatic test generation more valuable. [BlevisE08a] Blevis, Eli "Sustainability Implications of Organic User Interface Technologies: An Inky Problem", CACM Vol 51 No 6, June 2008, pp. 56Speculative essay on utility and environmental impact of switching from physical paper to electronic displays: special issue of CACM on tangible user interface interaction technologies. Multi-touch 3D surface interface (e.g. globe/map). [BloombergDS08a] Bloomberg, Dan S. "Unofficial Leptonica v1.68 Documentation", tpgit.github.io/UnOfficialLeptDocs/leptonica, 2004 Comprehensive graphics and image analysis processing library, image segmentation. Includes functions for alpha blending, affine operations (rotate, translate, etc.) [BogueA10a] Bogue, Adam "Segmenting a multi-touch input region by user", United States Patent Application 2010/0085323, August 8, 2010 On a multi-touch tablet/tabletop with multiple users (but not a tablet like DiamondTouch with extra electronics to identify each user), assign touches far apart (in space, mostly) to separate users, assign touches close to another touch (in space and time) to the same user as the first touch. Hardware can stream the assigned touches to different input streams, to maintain some compatibility with existing single-stream input applications. [BonnerM10a] Bonner, Matthew N.; Brudvik, Jeremy; Abowd, Gregory D.; and Edwards, W. Keith "No-Look Notes: Accessible Eyes-Free Multi-touch Text Entry", Proc. 8th Int'. Conf. on Pervasive Computing, Helsinki Finland, May 17-20, 2010 Accessible on-screen keyboard on touchscreen for blind/visually impaired: divide screen into large regions for groups of characters (ABC, DEF), tap on region gives sub-menu of individual characters. Cites to VoiceOver and Quikwriting, includes review of accessibility techniques for touchscreens including multitouch. [BostonGlobe08a] Boston Globe "Tech Development: Taking creepy out of proximity sensing", The Boston Globe, Monday, June 9, 2008, page B6 Electrostatic digitizer proximity sensing in a picture frame: Synaptics.com [BostonGlobe08b] Boston Globe "Tech Development: HP joins the campaign against keyboards", The Boston Globe, Monday, June 16, 2008, page B6 HP TouchSmart display, with integrated touchscreen. Comparison made to One Laptop Per Child PC design with no keyboard, instead second screen which can display a keyboard: compare with Refalo, Microsoft Courier. [BragdonA08a] Bragdon, Andrew "GestureBar: A Training-Free Approach to Disclosing and Teaching Gestures", Dept. of Comp. Sci. report CS-08-06, Brown University, July 2008



Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Microsoft Tablet PC Music Composition tool in 1998(??). [BragdonA08b] Bragdon, Andrew "GestureBar: Making Gestures Browsable, Discoverable, Learnable and TrainingFree", Master's Project, Dept. of Computer Science, Brown University, 2008 Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Microsoft Tablet PC Music Composition tool in 1998(??). [BragdonA09a] Bragdon, Andrew, Zeleznik, Robert; Williamson, Brian; Miller, Timothy; and La Viola, Joseph J. Jr. "GestureBar: Improving the Approachability of Gesture-based Interfaces", Proc. CHI 2009, April 4-9, 2009, Boston Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Kurtenbach Learning-while-Doing, Sketch-based user interfaces / Sketch recognition, Lineogrammer, and a simple crib sheet with tool-tips. Gestures must be discoverable/searchable. [BricklinD09a] Bricklin, Dan "Bricklin On Technology", Wiley Publishing, 2009 (hardcopy book) Collection of Essays and historical notes by Dan Bricklin (Slate Corp. / Lotus 123) inventor of spreadsheet software. Chap 8: Hans On: Tablet and Gestural Computing includes essay from bricklin.com "Tablet Computing Old and New" (earlier "Pen Computing Old and New"). Some essays date to 2000. [Brown09a] Brown University "Brown University - Microsoft Center for Research on Pen-Centric Computing: What is Pen Computing?", graphics.cs.brown.edu, 2009: fetched 2011 Final web page for Microsoft Center for Research on Pen-Centric Computing at Brown University: threeyear funding for research on applications for Microsoft Tablet PC operating system and tablet hardware: ChemPad, MathPad (mathematical notation) Music Notepad, Math Paper, Lineogrammer. Some work listed (SKETCH) pre-dates Tablet PC, dates to 1999. [BrownC10a] Brown, Chris et al "A System LCD with Integrated 3-Dimensional Input Device", SID Digest of Technical Papers, 2010, pp. 453-456 Integrated in-cell LCD display and touchscreen, senses in proximity. Optical sensors in pixels detect infrared light reflected from fingertips etc. [BrownC10b] Brown, Chris; Kida, Kazutoshi; Yamagishi, Shinji; and Kato, Hiromi "In-Cell Capacitance Touch-Panel with Improved Sensitivity", SID Digest of Technical Papers, 2010, pp. 346-349 In-cell LCD touchscreen, capacitive sensing. Stylus or finger depresses conductive surface, changes capacitance to underlying (integrated into LCD) sensing electrode by changing distance to sensor. Second style, two-electrode sensor, approaching conductor changes coupling between electrodes. [BrownSF08a] Brown, Stuart F. "Hands-on computing: Multi-touch screens could improve collaboration without a mouse or keyboard", Scientific American, July 2008, Vol 229 No 1, pp. 64-67 Jeff Han, Perceptive Pixel. Frustrated total internal reflection FTIR. Optical panel is both sensing surface and diffusion screen for rear projection. Also mentions Microsoft Surface, Mitsubishi Diamond Touch, and iPhone. [BruwerFJ08a] Bruwer, Frederick Johannes "Touch Sensor and Location Indicator Circuits", United States Patent 7,329,970, February 12, 2008 Using touch sensors sensing to control a flashlight/light by adding a MMI (man-machine interface: user interface). Touch sensed by microchip that adds intelligent functions to otherwise dumb device: emergency flashing, time schedule, etc. Different function depending on number of button acutuations, activate function on timeout (e.g. 5 seconds). See also Bruwer 2001, 2010. [BruwerFJ10a] Bruwer, Frederick Johannes "Intelligent user interface with touch sensor technology", United States Patent 7,772,781, August 10, 2010 Using touch sensors with proximity sensing to control a flashlight/light by adding a MMI (man-machine interface: user interface). Touch/proximity sensed by microchip that adds intelligent functions to otherwise dumb device: emergency flashing, time schedule, etc. Different actions based on proximity vs. touch, level of proximity e.g. continue operation if user in proximity, activate function (turn on radio display) when hand is near. See also Bruwer 2001. Compare with PenPoint proximity? [BuckleyE09a] Buckley, Edward; Lacoste, Lilian; and Stindt, Dominik "Rear-View Virtual Image Displays", 2009 SID Vehicles and Photons Conference, 16th Annual Symposium on Vehicle Displays Holographic 3-D three dimensional projection on windshield of automobile. see Light Blue Optics file. [BuckleyE09b] Buckley, Edward; Stindt, Dominik; and Isele, Robert "Novel Human-Machine Interface (HMI) Design Enabled by Holographic Laser Projection", 2009 SID Symposium pre-distorted projected image shown on curved surface to appear undistorted Holographic 3-D three dimensional projection on windshield of automobile [BuckleyE10a] Buckley, Edward "A Projector of a Different Color", IEEE Spectrum, May 2010, p. 45 Holographic (portable) projector projects display and simulated/virtual keyboard onto table-top, senses



touchscreen surface by fan of low-power infrared laster light superimposed on projected image and infrared sensors that detect scattering. [BuxtonW09a] Buxton, William "Chapter 14: Gesture Based Interaction / Haptic Input", www.billbuxton.com, dated 18 May, 2009 Book chapter in progress. Gesture recognition used broadly for natural gestures, gestures with the hands in free space. [BuxtonW10a] Buxton, William "A chat with Microsoft Principal Researcher Bill Buxton", The Globe and Mail, Toronto, Canada, August 23, 2010 (theglobeandmail.com) Historical interview with Bill Buxton concerning user interfaces, multi-touch, portable computing. Oscar winner (2003). multi-touch user interface inverted in 1984. Casio wrist-watch with touch-screen in 1984: unistroke single-digits and arithmetic symbols for calculator funcs. IBM Simon in 1993, precursor to iPhone. Non-contact (touch sensors) interfaces in 1982. Apple iPhone developers knew of IBM Simon smart touchscreen phone in 1993, described by Bill Buxton as precursor to iPhone. Online at http://www.theglobeandmail.com/news/technology/personal-tech/controller-freak/a-chat-with-microsoftprincipal-researcher-bill-buxton-part-i/article1679332/ [BuxtonW10b] Buxton, William "Invited Paper: A Touching Story: A Personal Perspective on the History of Touch Interfaces Past and Future", Society for Information Display (SID) Symposium Digest of Technical Papers, May 2010, Vol. 41(1) Session 31, pp. 444-448 Touchscreens, like all new technologies go far back: Plato in 1960s, soft-machines, Casio PF-8000 Data bank, IBM Simon, Multidimensional (X/Y/Z force, X/Y/Z torque) touchscreen . [BuxtonWAS10a] Buxton, William A. S. and SanGiovanni, John "Click-through controller for mobile interaction", United States Patent Application 2010/0275122, October 28, 2010 Transparent/overlay menus and widgets for user interface on a touchscreen: compare with earlier clickthrough work. Cites to ToolGlass, transparent sheets, also to "Chameleon" system detecting position of handheld touchscreen for viewing virtual scene (compare with Tagliozzi video?) [CERNCourier10a] Stumpe, Bent and Sutton, Christine "The first capacitative touch screens at CERN", The CERN Courier, March 31, 2010 Historical article on capacitive touchscreens invented at CERN in the 1970's as simplified/better-organized user interface UI for complex computer system for SPS Super Proton Synchrotron. Capacitor sensors etched into a film of copper on a sheet of glass, each capacitor constructed so that a nearby flat conductor such as surface of a finger increases capacitance by a significant amount. Traces 80 micrometers wide, 80 micrometer spacing, insulating lacquer coating on top. First prototype produced in shop at CERN on flexible transparent Mylar sheet. Gives details of construction using ion sputtering, traces were solderable. Phase-locked oscillator, change in capacitance changed phase not frequency, drift compensation by comparison to reference capacitor in same construction. [CITT08] CITT Center for Instructional Technology and Training, University of Florida "Tablet PC Workshop", www.citt.ufl.edu/training/manuals/TabletPC.pdf, fetched 2008/03/03 Training manual for Tablet PCs, (physical) power button action can be changed [CSAIL08a] CSAIL "MIT CSAIL: Design Rational", rationale.csail.mit.edu, 2008 List of research reports and publications from MIT CSAIL group, mostly relating to sketch recognition [CakmakM10a] Cakmak, Mukerrem; Zhao, Wei; and Yalcin, Baris "Manufacturing of multifunctional electrically conductive-transparent-flexible films", United States Patent Application US 2012/0153236 A1, June 21, 2012 Transparent conductive films for touchscreens, films of conductive fibers. Cites to ITO for transparency and very low resistivity, but is brittle especially on flexible substrates. Compare to Wang/Microtouch. [CannyJ10a] Canny, John and Hartmann, Bjoern "Beyond the Cell Phone: Introduction and Historical Perspective", Seminar Notes, COMPSCI 298-50, Univ. Berkeley, California, September 2010 IBM Simon 1994, Osborne I luggable computer 1981, Grid Compass laptop, PARCTAB hand-held ubiquitous computing unit 1989/1992, Palm PDA with Unistroke Graffiti [CaoX08a] Cao, Xiang; Wilson, Andrew D.; Balakrishnan, Ravin; Hinckley, Ken; and Hudson, Scott E. "ShapeTouch: Leveraging Contact Shape on Interactive Surfaces", Proc. TABLETOP: 2008 IEEE Int'l. Workshop on Horizontal Interactive Human Computer Systems User interface for large touchscreen surfaces (e.g. Microsoft Surface, Rekimoto surface): use area of contact (multi-touch number of fingers, hand area, etc.) to simulate larger or smaller forces on virtual objects. Also simulate pressing, colliding (pushing on virtual edge), friction (require more area contact to slide), flicking and rotate/scale around an anchor point. [CasadesusMasanellR08a] Casadesus-Masanell, Ramon; Boudreau, Kevin; Mitchell, Jordan "Palm (A): The Debate on Licensing Palms OS (1997)", Harvard Business School study N9-708-514, May 20, 2008 Study of internal Palm/US Robotics/3COM debate on whether to license PalmOS to other hardware vendors broadly (as Microsoft) instead of proprietary OS (as Apple). Cites Psion Organiser 1984 as first commercial handheld computing device. GO/Penpoint, mis-states Microsoft pen-computers as using DOS (not Pen



Windows); Momenta, Grid. Describes General Magic Magic Cap as joint project of Apple (?), Sony, Monotorola, others prior to Apple Newton. Palm pursued small third-party developers, large demand. [CasioCalculatorWatch10a] Casio Calculator Watch "Vintage Casio TC-600 Touch Screen Calculator Retro LCD wristwatch", casiocalculatorwatch.net, fetched 2010 Casio wristwatch with unistroke character input on touchscreen, circa 1984 [CechanowiczJ09a] Cechanowicz, Jared and Gutwin, Carl "Augmented Interactions: A Framework for Adding Expressive Power to GUI Widgets", Proc. INTERACT 2009, pp. 878-891 Addition of pressure sensors, Contrast of WIMP user interfaces with other numerous non-shaperecognition-based gestures: shift-click, double-click, timed gestures hover-help and spring-loaded folders. Effort to define general basis for haptic feedback on GUI widgets, rather than ad-hoc development. [CenterForPenCentric08] Center for Research on Pen-Centric Computing, (Microsoft) "What is Pen-Centric Computing?", pen.cs.brown.edu Microsoft funding of graphics research group under Andries van Dam at Brown University. Projects as of 2008 include MathPaper two-dimensional mathematics input and gesture/direct manipulation, Lineogrammer sketch recognition / diagram editing, *Pad SDK ink middleware experimental pen-computing UI kit, ChemPad for teaching chemistry and molecular models, MathPad, Music Notepad with twodimensional gesture input, Fluid Inking for gesture editing of electronic ink (compare with Van Raamsdonk? ), ReMarkable Texts digital note-taking. Devices include pen computers (Tablet PCs), PDAs, electronic whiteboards. [ChangD08a] Chang, Darius "Asus takes user interface to the next level", crave.cnet.net, June 5, 2008 Dual-display computer (compare with Refalo, OLPC One Laptop Per Child) with display on one half, display/touchscreen on other half. Also reads gestures by motion using video camera, optical. [ChangWY09a] Chang, Wen-Yang; Fang, Te-Hua; Yeh, Shao-Hsing; and Lin, Yu-Cheng "Flexible Electronics Sensors for Tactile Multi-Touching", Sensors, Vol. 9, 2008, pp. 1188-1203 Flexible multitouch sensor touchscreen, resistive grid in two layers: avoids shadowing problem by switching individual rows/columns to high/low-impedance, and solving Kirchhof's Laws equations for measurements on all rows and columns. Small spacer dots (compare Elographics?) prevent two layers from touching during bend. Flexibility allows slight deformations, not extreme bending. [ChaudhriI10] Chaudhri, Imruan; Ording, Bas; Anzures, Freey Allen; Van Os, Marcel; Lemay, Stephoen O.; Forstall, Scott; and Christie, Greg "Unlocking a Device by Performing Gestures on an Unlock Image", United States Patent 7,657,849, Feb. 2, 2010 Sliding/swipe slide-to-unlock gesture on Apple iPad and iPhone. Also US 8,046,721. Compare with direct manipulation, virtual devices? [ChenJ10a] Chen, Jian; Eow, Andy; and Ma, Xiaoxu "Simple Techniques for Three-Dimensional Modeling", United States Patent Application 2010/0319100 A1, December 23, 2010 Structure-through-motion modelling of human shape, 3D model using single camera. Customer puts on patterned garment (with targets, may be irregular), multiple views from webcam, system determines 3D shape by analysis of the images. Compare with D. Holz, 2013, etc. [ChenZ09a] Chen, Zhen; Li, Barn-Wan; Wang, Jian; Ma, Wei-Yin "Annotation Management in a Pen-based Computing System", United States Patent 7,555,706, June 30, 2009 pen-computing user interface, annotations can be added to documents. Annotations can be recognized as gestures: recognizing a date annotation bring up a calendar application. Annotations anchored to context/text of document. Compare with Agulnick and Slate annotations and electronic ink indexing, deferred recognition. [ChristianDopplerLaboratory10a] Christian Doppler Laboratory, for Handheld Augmented Reality "History of Mobile Augmented Reality", wordpress.com, fetched November 23, 2010 Timeline of mobile augmented reality and non-mobile augmented reality systems, defined as overlaying computer data on real world. Sutherland head-mounted display with 6-D sensors 1968. Cites Grid Compass 1000 as first clamshell laptop: compare with Data General DG/1? IBM Simon Personal Communicator 1992. Loomis GPS system for visually impaired 1993. [ChuCC09a] Chu, Chi-Chun "Apparatus for setting multi-stage displacement resolution of a mouse", United States Patent 7,532,200, May 12, 2009 Read physical physical switches on side of mouse to get a resolution/delta value to set sensitivity of the mouse. Compare with multi-button stylus? [Cirque09a] Cirque Technology (owned by ALPS) "Capacitive Touch with Near Field Communication", www.cirque.com multi-touch multi-finger gestures on Cirque GlidePoint sensor. Additional materials in file: claim Cirque originated capacitive touchscreens in 1991 [CoE08a] Co, Elise and Pshenkov, Nikita "Emerging Display Technologies for Organic User Interfaces", CACM, June 2008, Vol 51 No 6; pp 45-ff.



Organic user interfaces / tangible user interfaces: E-Ink / electronic ink electrophoretic display technology, electroluminescent displays on clothing fabric. [CockburnA08a] Cockburn, Andy; Karlson, Amy; and Bederson, Benjamin B. "A Review of Overview+Detail, Zooming and Focus+Context Interfaces", ACM Computing Surveys, Vol 41 No 1, Article 2, December 2008, pp 2:1..2:31 Review of of user interface systems that show more and less detail, both graphical zoom (original meaning of focus+context, and in Mac XDock) and semantic zoom. Classifies as zooming; overview plus detail; focus plus context; cue-based to highlight or suppress items selectively. Fisheye zoom (enlargement in graphical context). Mentions pressure/force of touchscreen for zooming (Zliding, etc.). Later version of 2006 technical report, U. Maryland, same authors [CohenP08a] Cohen, Phil; Swindells, Colin; Oviatt, Sharon; and Authur, Alex "A high-performance dual-wizard infrastructure for designing speech, pen, and multimodal interfaces", Proc. ICMI '08, 10th I'tal Conf. on Multimodal Interfaces Wizard-of-oz (two people) simulation of system reacting to group whiteboard utilizing both voice and realtime electronic ink/pen input streams. Compare with Freestyle at Wang. [ConwayJ10a] iPhone Programming: The Big Nerd Ranch Guide "Conway, Joe and Hillegass, Aaron", Big Nerd Ranch Publishing, 2010 (hardcopy book) Development guide for IOS for iPhone (iPad, iPod) touchscreen devices: Simulator, Objective-C, etc. [CoolestGadgets08a] Coolest Gadgets - Apple News "Apple files for 'swipe-gesture' patent application", www.coolest-gadgets.com, Dec. 26, 2008 Patent application publication of Apple filing for 'swipe-gesture': article refers to left, right, up, down gestures across the (simulated) keys of a keypad -- compare with 1980's Japanese calculator with handwriting input on physical keypad? [CrossEM09a] Cross, Elisa M.; Moshrefzadeh, Robert S.; Bottari, Frank J.; Cairns, Darran R.; Chernefsky, Anthony F.; Richter, Paul J. "Capacitive touch screen with conductive polymer", United States Patent 7,477,242, January 13, 2009 (Filed 2002) Conductive polymer for capacitive touch screens: higher resistance than ITO (desirable for for capacitivesheet touchscreens), and not brittle, so may be used on flexible touchscreens. [Crunchpad09] CrunchPad "CrunchPad photos leaked: Arrington builds dream tablet", Jennifer Bergin, blogs.zdnet.com, April 10, 2009 Web-only pen-computing web-tablet with no disk storage. Announced as an "open-source" hardware design, running Ubuntu Linux. File contains additional materials. [CypherA10a] Cypher, Allen "The Evolution of End User Programming", Google Tech Talks, February 1, 2010. Fetched from YouTube.com 2014 History of developments for end-user programming. Says first end-user-programmer 1980 (non-computerperson), MacIntosh. Defines end-user programming as end-user not constrained to an existing application. [Cypress08] Cypress Inc. "TrueTouch touchscreen technology", www.cypress.co, Aug 25. 2008 Multi-touch touchscreen using projected capacitive technology (electrostatic?). Includes comparison chart for touchscreen technologies: Infrared, Surface-Acoustic-Wave (SAW), Surface Capacitance, Resistive [DeNardiA08a] De Nardi, Alessandro "GRAFITI: Gesture Recognition mAnagement Framework for Interactive Tabletop interfaces", Master Degree Thesis, Universita di Pisa, December 2008 Experimental multi-touch system (Rear Diffusion optical detection: rear DI) incorporating Z-order gestures by tunnelling and bubbling (for overlapping objects). Refers to larger target area as "iceberg tips" of object. Gestures are simple, single-stroke, and few (touch, drag, etc. except for double-tap) to avoid recognition ambiguity. Discusses dynamic gestures (e.g. pinch/zoom) which are detected and interpreted progressively, and static gestures (e.g. tap) which have no interpretation phase File contains additional material on open-source code. Not to be confused with Graffiti unistroke handwriting. Contains review of multi-touch hardware and software systems, some reference to Tablet PC but none to handwriting input. Multi-touch Vista, Multitouch.Framework on MacOS/iPad, Sparsh-UI. Gestures recognized by both spatial (shape?) and temporal (motion/movement) aspects of fingers moving over surface (compare with Nokia in-air optical gestures?) Tangible User Interface (TUI)/ Graspable User Interface; lasso/lazo gestures. Defines gesture as including motion. Cooperative gesturing as distinct from multi-user gesture systems for collaboration. Hovering gesture is press-and-hold, not in-air hover: uses two-hand hover on two objects to link them. iPhone generally determines target of gesture on touch-down / hitTest:withEvent / gestureStart, not finger-lift/mouse-up/gestureEnd [DeckerS09] Decker, Susan "Microsoft Wins Reversal of $358 Million Alcatel Award (Update4)", Boomberg News: www.bloomberg.com Sept. 11, 2009 Partial reversal (basis for damages) in Lucent-Microsoft case: only mentions Date-Picker UI, not GO Corporation damages [DegnerBW08a] Degner, Brett William; Staton, Kenneth Lawrence; Hotelling, Steve Porter; and Choi, Shin John



"Multi-touch Sensor Patterson and Stackups", United States Patent Application 2009/0273570, November 5, 2009 Capacitive grid / mutual capacitance touchscreen on flexible substrate, to fit to curved surfaces. Comb-like interdigitated pattern between rows and columns, also three layer with intermediate coupling conductors. Mentions running traces on single layer plus vias. [DenoueL09a] Denoue, Laurent and Foote, Jonathan T. "Force-feedback stylus and applications to freeform ink", United States Patent 7,508,382, March 24, 2009 Stylus with force feedback: magnetic fields acting on metal ball in tip. [DerStandard10a] Der Standard "Toshiba bringt Dual-Screen-Tablet Libretto W100", Der Standard, derstandard.at, June 21, 2010 Dual-screen multitouch portable computer with virtual keyboard, compare with Microsoft Courier and with Refalo, and with proposed design by Negroponte's project [DiamondTouch08a] DiamondTouch, Inc. "DiamondTouch", (various) File on Diamond Touch, spin-off company for DiamondTouch multi-touch digitizer table developed at Mitsubishi, comparable to Microsoft Surface [DignanL10a] Dignan, Larry "Apple's HTC patent suit: Can it derail Google's Android devices?", zdnet.com, March 2, 2010 Apple iPhone patents asserted against HTC/Android and Nokia: antenna design, geographical positioning [DignanL10c] Dignan, Larry "Nokia and Microsoft's Windows Phone 7: It could work, but why stop there?", zdnet.com, December 20, 2010 Nokia sell Windows Hpone 7 instead of Symbian. [DonkerJH09a] Donker, Joao Henrique "The body as a communication medium", 11th Twente Student Conference on IT, Eschede Holland, June 29, 2009 Review of techniques for transmitting signals through human body, some applications. Inter-body and intra-body body/device communications for body sensing networks (BSN). High speed data rate (up to 1000 Kb/s) using radio-frequency signals coupled through human body. Cites to Zimmerman capacitivecoupling signals for smart furniture at MIT. [EInstruction09] eInstruction.com "eInstruction Interwrite DualBoard", eInstruction.com product literature Large whiteboard digitizer using projector, multiple small untethered electromagnetic tablets for other participants in a group: see KayeS. Interwrite DualBoard for Windows and Mac, Interwrite Mobi small wireless digitizer. See also Qomo HitVision RF wireless tablets. [EMSPS.com10a] emsps.com "Old, Used and Out-of-Print SideKick(R) Software", www.emsps.com/olktools/borsk.htm Borland SideKick with phone dialer, PIM: Version 1.0 through 20.0, 97 through 99, for DOS, Windows and MAC [EMSSoftware08] EMS Software "Lotus 1-2-3 Version Information", www.oldsoftwareinfo.com Version information for Lotus 123 versions: www.oldsoftwareinfo.com sells and collects old versions of software in original form. Lotus 123 for DOS only through 2.2X 1991, first Windows version 1.0 for Windows 9991, first MacIntosh version 1.0 1991, OS/2 version 2.0X, 1993. [E-Ink08a] E-Ink Corporation "Wacom and E-Ink Partnership Delivers Pen Input to Electronic Paper Displays", Press Release, Wacom Technology, May 19, 2008 Touchscreen / integrated display and digitizer tablet from Wacom and E-Ink. [E-Ink08b] E-Ink Corporation "E-Paper Soon To Be in Living Color", IEEE Computer, April 2008, p. 20 Editorical comment on E-Ink e-paper displays, handheld devices: color display [EastonSD08a] Easton, Stephoen D. "Attacking Adverse Experts", American Bar Association, 2008 (hardcopy book) Questioning methods and techniques on expert witness testimony and reports [EchtlerF10a] Echtler, Florian; Klinker, Gudrun; and Butz, Andreas "Features, Regions, Gestures: Components of a Generic Gesture Recognition Engine", SIGCHI Workshop: Engineering Patterns for Multi-touch Interfaces, June 20, 2010, Berlin, Germany Recognize multi-touch gestures based on shared features: move/tap/rotate/spin gestures recognized on features motion/blobcount/rotation/scale. Focuses more on direct manipulation gestures. Describes event bubbling through parent windows for gestures. [EchtlerF10b] Echtler, Florian; Pototsch, Thomas; and Klinker, Gudrun "An LED-based Multitouch Sensor for LCD Screens", Proc. ACM TEI 2010, January 2010, Cambridge, Mass, pp. 227-23Multitouch optical touchscreen sensor: infrared LEDs in pairs behind an LCD display -- LCD displays transparent to infrared. One LED used as infrared emitter, other used as infrared detector, for infrared light reflected from objects touching the screen. Has enough proximity sensing to detect hover. Compare Fig. 7 (image of hand) with Rekimoto 2002 paper (electrostatic/capacitive). [Economist08a] Economist Magazine "Touching the Future: Computing: Touch screens are becoming an increasingly popular way to control mobile phone and other devices", The Economist Magazine, September 4, 2008



Article on touch screens, and multi-touch gestures. Cites Elo TouchSystems and Sam Hurst as first touchscreen (resistive tablet) in 1971 (missed earlier references). Quotes Bill Buxton on haptic design. Jeff Han of Perceptive Pixel on pressure-sensitive gestures, push harder to move something "under" another window instead of double-click operation. [EderyYM10a] Edery, Yigal Mordechai; Vered, Nimrod Itzhak; Kroll, David R.; and Touboul, Shlomo "Malicious mobile code runtime monitoring system and methods", United States Patent 7,647,633, Januar 12, 2010 Malware checking on downloaded code (.EXEs, JavaScript, Java applets, etc.) by firewall or filter putting the downloads on a separate machine, examining them or letting them run, then enforcing a security policy or action. Compare with deep packet inspection. [ElanMicroelectronics10] Elan Microelectronic Corporation "Elan Microelectronics Corporation", Elan Microelectronics, product information Electrostatic digitizer / capacitive touchpad vendor [ElektronikIndustrie10a] Elektronik Industrie "Der etwas andere touch", Elektronik Industrie, 10/22/2006, pp. 2223 in German: overview article on capacitive touchscreens using ITO Indium Tin Oxide, based on Quantum Research Group (commercial company) [EliasJG10a] Elias, John Greer; Westerman, Wayne Carl; and Haggerty, Myra Mary "Multi-touch Gesture Dictionary", United States Patent 7,840,912, November 23, 2010 Extension of chorded keyboard (on multi-touch touchpad) to defining chording input as user-defined chord (multi-finger) gestures. Numerous citations to Fingerworks. [EoffBD09a] Eoff, Brian David and Hammond, Tracy "Who Dotted That `i'? : Context Free User Differentiation through Pressure and Tilt Pen Data", Proc. Graphics Interface Conf, May 2009, Kelowna B.C. Canada In multi-touch / multi-pen / multi-user desktop with tablet or touch input, identify user inputs by physical characteristics of writing, rather than constraining users to keep track of which pen to use: pen tilt, pressure (force), and writing speed. tablet-PC lower case, not Tablet PC. Hardware was Wacom Cintiq tablet with tilt and tip force sensing. [EricsonP10a] Ericson, Petter and Lynggaard, Stefan "Methods and Devices Relating To Transfer of Non-Pen Stroke Data", United States patent Application 2010/0073330 A1, March 25, 2010 Anoto forms: Send non-pen data (e.g. bar codes) as if it were pen-stroke data in an unused margin of the printed form. Mentions in passing paper stylus processor/PC contacting look-up server (PLS), and application server based on response from PLS. No mention of encoding out-of-channel data (e.g. voice -see Pen Protocol?) [EsentherA08a] Esenther, Alan "Multi-Touch Gestures for Controlling Synchronized Map Views", Slide presentation, 2008 ESRI International User Conference, ESRI Incorporated, New York Gestures: one finger to select, flick, mouse-operations; two fingers to resize/move/rotate object; five fingers to pan entire map; fist to scroll or tilt. Commercial multi-touch touchscreen tabletops PerceptivePixel, Microsoft surface, TouchTable, DiamondTouch CircleTwelve. For mouse gestures, multiple users handled first-to-touch wins. Middle finger tap for middle mouse button. "Precision-hover" when finger obscures pixel to be selected. [EsslG09a] Essle, Georg; rohs, Michael; Kratz, Sven "Squeezing the Sandwich: A Mobile Pressure-Sensitive Twosided Multi-touch Prototype", Proc. UIST '09, October 2009, Victoria BC, Canada Two-sided (front/back) touchscreen/touchpad interfaces. Prototype built by taping two iPhones together. Citations to Chen, Hybridtouch, Lucid Touch, and Shape touch. [FahraeusC10a] Fahraeus, Christer; Hugosson, Ola; and Ericson, Petter "Device and Method for Recording Handwritten Information", European Patent 1073945 B1, March 24, 2010 Anoto pattern: record handwriting information by matching snapshot images by overlaps / using sequence of images to determine motion/position of handwriting stylus. Filed 1999: WO 1999/060467 [FihnM10a] Fihn, Mark "Touch Panel: Will Apple enable Microsoft's vision for the Tablet PC?", www.veritasetvisus.com, December 2010 Historical review (revised from 2002 and 2008) of Tablet PC and Apple iPad. Tablet PC announced August 1999 by Bill Gates, Prototype demonstrated June 2000. Comparison with success of Apple Newton concept in the form of smart phones. Cites to IBM TransNote and Sony SlimTop Pen Tablet. [FingerSystem10a] Finger System USA "i-Pen Mouse product information", www.fingersystemusa.com, fetched 2010 Optical mouse in a pen-shaped barrel. Newer model includes audio recording: compare with LiveScribe and Plawa. [FlemingN10a] Fleming, Nic "Magnet magic puts phone control in the air", New Scientis, 23 February 2010 Hand-gesture input using hand-held magnet and magnetic compass sensor in an iPhone or portable device [FoleyMJ10a] Foley, Mary Jo "Microsoft's RearType: Physical keys to the iPad, Kindle and Tablet kingdoms?", www.zdnet.com, August 9, 2010



UI alternative to soft/simulated keyboard on touch-screen/pen computer: physical keys for separate hands on back of tablet-type computer. [FrankeK09a] Franke, Katrin "Capturing Reliable Data for Computer-based Forensic Handwriting Analysis II: Penposition Activations", 2009 10th International Conference on Document Analysis and Recognition Pointing out unreliability of data from (electromagnetic) digitizer tablets, even though really only talks about stylus tilt error. One of few papers from handwriting motion analysis that shows insights into some of the problems with digitizer reliability and bad behaviors. Describes morphing of on-line and off-line traces/markings -- compare with Tappert elastic matching? [Fraunhofer08a] Fraunhofer Institut "Computer steuern per Fingerzeig / Gesture-driven computers", Fraunhofer Institute Press Release for CeBIT 2008 exhibition, February 2008 (in German) GestureID hand gestures, demonstrated at CeBIT exhibition for iPoint Presenter. Mentioned as replacement for touch-screen digitizers at infoterminals (kiosks). Uses optical pattern recognition (machine vision) [FriedI08a] Fried, Ina "Microsoft to add multitouch interface to Windows 7", news.cnet.com, May 27, 2008 Windows 7 Vista will include multitouch, compares with Apple iPod user interface. Shown by Steve Ballmer and Bill Gates at trade show. Gestures include pinching (zoom) and two-finger rotating. [FriedI08b] Fried, Ina "Microsoft hopes 'Milan' table PC has magic touch", news.cnet.com, May 29, 2008 Milan code name for "surface computer", Microsoft Surface tabletop PC, using multi-touch input. [FriedI09a] Fried, Ina "Touch in Windows 7: Just for show?", news.cnet.com, July 1, 2009 Touch/multi-touch interface in Windows 7 just for show, because touchscreen digitizer hardware adds $100 to price of PC. N-Trig dual-mode digitizer (electrostatic+electromagnetic?) has funding from Microsoft [FrischM09a] Frisch, Mathias; Heydekorn, Jens; Dachselt, Raimund "Investigating multi-touch and pen gestures for diagram editing on interactive surfaces", Proc. CACM ITS '09, November 2009, Banff, Canada Study of user gestures for sketch editing on electronic whiteboard: bimanual interaction/gestures not common, hypothesis users accustomed to whiteboard/paper drawing with one hand. Two-finger bring together / separate gestures. Two hand gestures for copy (hold original still, drag out with other finger) [FrogPad09a] FrogPad "Mult-Touch FrogPad on Tablet and Mobile", FrogPad.com, uploaded to youtube.com 2009 Single-hand typing keypad layout. Shown in video implemented as movable virtual keyboard on tablet computer (TabletPC?). Audio commentary implies FrogPad has basic multi-touch patents: see file. [Fujitsu10a] Fujitsu America, Inc. "Lifebook Tablet PC. Stay in Touch", Fujitsu America, Inc. Product information on Lifebook T900 Tablet PC, convertible with multi-touch separate display [Fujitsu10b] Fujitsu America, Inc. "CEATEC Fujitsu's dual-screen phone prototype: Video demonstration", Fujitsu Video demonstration at CEATEC 2010 of prototype Fujitsu dual-touchscreen phone, screens can be oriented end-to-end or fold out side-to-side (special hinge): compare with Refalo? [Fujitsu10c] Fujitsu America, Inc. "Fujitsu Shows Off Clamshell Handset With 2 Touch Panels", Fujitsu Fujitsu dual-touchscreen phone, screens can be oriented end-to-end or fold out side-to-side (special hinge). May be combined to be one display, or separate apps (e.g. one keyboard, one application display) compare with Refalo? [Ganapati10] Ganapati, Priya "3-D Tabletop Display Gets Rid of the Glasses", Wired.com, March 25, 2010 Simulated 3-D display: each side of a cube has an LCD display showing a side/top view of something simulated to be inside [GestureTek08a] GestureTek "GestPoint Gesture Recognition for Presentation Systems", www.gesturetek.com Hand-gesture recognition using video input and motion tracking: commercial product. Described as touchfree gesture input utilizing computer vision and body tracking software to convert simple hand movements into direct mouse control. [Gizmodo09a] Gizmodo.com "Que: Plastic Logic's Capacitive-Touch Ebook Reader", Gizmodo.com, November 2009: reporter Wilson Rothman Wacom technology (?) capacitive/electrostatic digitizer in an e-book reader, but says being marketed as a "document reader", not a "book reader". See also file on "Plastic Logic". [Gizmodo10a] Gizmodo.com "First Hands On: Touch Book is Part-Netbook, Part-Tablet", Gizmodo.com, April 13, 2010 Touch Book web tablet / tablet computer: Linux, circular-icon gesture UI (pie gestures?) [GoertzM09a] Goertz, Magnus; Eriksson, Thomas; and Shain, Joseph "Light-based Touch Screen", United States Patent Application 2009/0189878, July 30, 2009 Optical touchscreen using infra-red LED diodes and light detectors. Variations include having LEDs at four corners projecting light in a full plane to the two opposite sides of the LED, and all LEDs on a side being illuminated at once and the detectors on the opposite side read in parallel. States can detect hard and soft touches by detecting position while still above surface and determining vertical height (and velocity) from how much light is reflected from the object/finger above the surface. Prior art discussion mentions resistive and capacitive touchscreen etc. but not optical.



[GoertzM10a] Goertz, Magnus; Eriksson, Thomas; and Shain, Joseph "Optical Touch Systems using Reflected Light", United States Patent Application 2010/0238138, September 23, 2010 Optical touchscreen using infra-red LED diodes and light detectors. Various geometries for reflectors, such as having LEDs at or below surface of device and low-angled mirror on the full side to direct the light across/over the touchscreen surface; LEDs projecting a plane or lines of light to detectors diagonally (rather than directly across); angled upward from points within touchscreen at an angle and detectors also at an angle to detect light reflected diagonally downward from finger or object. One figure showing light propagated across and within touchscreen glass by internal reflection. Assigned to NeoNode: see NeoNode file. [GoldsteinN10a] Goldstein, Neal and Bove, Tony "iPad Application Development for Dummies", Wiley Publishing, 2010 (hardcopy book) Programming tutorial for the iPad/iPhone/iPod. [GonsalvesA09] Gonsalves, Antone "Barnes and Nobel Sued over E-Reader", Information Week, November 3, 2009 E-Reader maker Spring Design (kindle-like product) sues Barnes and Nobel over using features of Alex ereader from confidential product technical discussions [Google08a] Google Code "iphone-haptics", code.google.com/p0/iphone-haptics, 2008 VibeTonz and vibrotactile haptic feedback project for Apple iPhone, Stephen Brewster Tacton project at U. Glasgow [Google08b] Unknown "google chrome", drewp.quickwitretort.com/2008/09/22 Search hits highlighted in Google Chrome scrollbar UI/user-interface: references to work by David McClosy, and Raizlabs ABC and Heatmap [GorbunovA09a] Gorbunov, ArtemCode "Infoscroller by Artem Gorbunov (video)", Posted to flickr.com, 2009 Brief video demonstraction of Infoscroller: "thumb" on slider sized to amount of document visible, thumb shows thumbnail images of section of document as reference. Compare with highlighting on scrollbar. [GreeneK08a] Greene, Kate "What's Next for Computer Interfaces? Touch tricks for small and large displays could be the next big thing", Technology Review, December 11, 2008 Microsoft research Patrick Baudisch adding touch digitizer interaction to back of devices so that display is not obscured by finger or stylus [GreeneK09a] Greene, Kate "A Better, Cheaper Multitouch Interface: A new pressure-sensitive pad could improve large and small touch screens", Technology Review, March 30, 2009 Three-dimensional pressure sensitive touch digitizer: IMPAD Inexpensive Multi-touch Pressure Acquisition Device, using force sensitive resistor (IFSR) ink, Ilya Rosenberg, www.touchco.com [GreeneK09b] Greene, Kate "Touching the Future: Next steps for touch screens", Technology Review, February, 2009 p. 13 Patrick Baudisch of Microsoft: touch pad on back of display, so that small screen remains unobstructed. Also mentions Perceptive Pixel [GreeneK09c] Greene, Kate "Muscle-Bound Computer Interface: Forearm electrodes could enable new forms of hands-free computer interaction", Technology Review, October 28, 2009 electromyography sensors in arm used to recognize hand/arm gestures, such as Air Guitar, etc. Recognition by neural network. [GreenemeierL10a] Greenemeier, Larry "Magic Fingers: Digging into Multi-touch Technology with Both Hands: Perceptive Pixel chief scientist Jefferson Han has big plans for changing how people use computers", Scientific American, March 5, 2010 Interview with Jeff Han on Multitouch technology by perceptive pixel: states that large screen manipulation is the interesting user-interface area, criticizes capacitive sensing multi-touch input as not scalable to large surfaces or use with gloves compared to frustrated internal reflection [GriffinJT09a] Griffin, Jason T.; Holmes, John A.; Lazaridis, Mihal; Little, Herb A. and Major, Harry R. "Hand-held electronic device with a keyboard optimized for use with the thumbs", United States Patent 7,629,964, December 8, 2009 Blackberry keyboard for two-thumb thumb typing: rounded keys, full alphanumeric layout at bottom of display [GroskyWI08a] Grosky, William I.; Westermann, Utz; Zeleznik, Robert; van Dam, Andries; Li, Chuanjun; Tenneson, Dana; Maloney, Christopher; and LaViola, Joseph J. jr. "Applications and Issues in Pen-Centric Computing", IEEE MultiMedia, October 2008, pp. 14-21 WIMP User interfaces predominate, but some applications better with tablet/pen computers: 2D mathematical notation MathPad), chemical structures (ChemPad), Diagramming/sketch input (Lineogrammer). Mentions Wang Freestyle, Pen Windows, GO PenPoint, Anoto photo pen, others. [GrunbergerA10a] Grunberger, Aaron "System Device and Method for Extending a Stroke of a Computer Device", United States Patent 7,808,483, October 5, 2010



Button on mouse to "reverse direction", so you can zag back from the edge of the mouse pad (e.g.) to continue drag motion. Compare with synthetic reality prototype? [GunnHD10a] Gunn, Harold David and Chapman, John "Data entry for personal computing devices", United States Patent 7,681,124, March 16, 2010 Pop-up movable virtual keyboard in the form of a marking menu, using predictive work list and pause and lift (press-and-hold) to re-invoke search for text completion. [HOPL08] HOPL History of Programming Languages "Maths input language", hopl.murdoch.edu.au History of two-dimensional mathematics input languages as programming languages: essentially a short bibliography of papers by Mort. I Bernstein, quotes as saying that real-time / online handwriting recognition is easier than static / OCR character recognition. [HP08a] Hewlett-Packard "HP TouchSmart IQ506/IQ504 PC", (various) File of product information on HP TouchSmart PC, Windows PC with integrated multitouch touchscreen. Used for kiosk applications. [HaagF09a] Haag, Florian "Entwicklunggeines flexiblen Widget-Sets fuer die Stuttgarter Stiftplatte", Diplomarbeit (Master's thesis), Nr. 2893, U. Stuttgart, May 5 2010. Using BrailleDis tactile/haptic display tablet, study of best tactile representations of conventional windows control elements: command buttons (TactonButton), menus, edit boxes, ribbons, sliders, checkboxes, etc. [HamppA08] Hampp, Andrew "Airports, Malls Get More 'Gesture' Ads: Interactive Video-Out-of-Home Ads let Consumers Play", Advertising Age, May 8, 2008 Press article on Reactrix, Monster Media, and GestureTek on gesture input on interactive advertising: footpressure touch pad with foot input [HanJ08a] Han, Jeff Y. "Multi-Touch Sensing through Frustrated Total Internal Reflection (video presentation)", Web posting cs.nyu.edu/~jhan/ftirsense, fetched 2014 See original 2005 paper. Detect multiple finger touches on a rear-projection surface (multi-touch), by reflected light where finger is in contact with surface: same technology used for fingerprint scanners. [Hanselman08] Hanselman, ?? "Introducing RockScroll", Microsoft employee blog posting: www.hanselman.com/blog, October 2008 Scroll bar displays highlighted/small images of text/document to be scrolled: add-on to Microsoft Visual Studio by Rocky Down's [Hardie-BickAR08a] Hardie-Bick, Anthony Richard "Touch Pad", United States Patent 7,411,581, August 12, 2008 Based on 2002 GB filing: Acoustic touchscreen using frequency profiles characteristic of different location (Lamb waves: higher frequencies disperse faster) rather than acoustic delay. Identifies touch events (start of touch), not contact. Requires profiling of construction first, to produce a table? [HarrisonC08a] Harrison, Chris and Hudson, Scott E. "Pseudo-3D video Conferencing with a Generic Webcam", Proc. 10th IEEE Int'l Symposium on Multimedia, Berkeley, CA, Dec 15-17, 2008. pp. 236-241 Simulated 3-D video using flat display: perform trapezoidal scaling of image as viewer moves left and right (and up and down?) as detected by Webcam camera. See also matching video file. [HarrisonC08b] Harrison, Chris and Hudson, Scott E. "Scratch Input: Creating Large, Inexpensive, Unpowered and Mobile Finger Input Surfaces", Proc. UIST 08, October 19-22, 2008, Monterey, CA, pp 205-208 Gesture recognition not using a digitizer, but the acoustic sound of scratching when a finger/fingernail moves on a surface. Cannot detect spatial location, but recognizes multi-part (multi-stroke) gestures such as X, O-with-slash, lowercase-I-with-dot, Y, double tap, single tap. Cites to earlier work on passive acoustic sensing (knocks) and frustrated-internal-reflection digitizer tablets [HarrisonC09a] Harrison, Chris and Hudson, Scott E. "Abracadabra: Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices", UIST '09, October 4-7, 20009, Victoria BC, Canada Proximity-sensing electromagnetic digitizer for finger input on small display, using small magnet mounted on fingertip. Compare with ferrite-rod digitizer technologies? Can detect clicking "below" a virtual surface by reversal of magnetic field, can detect distance/proximity by strength of magnetic field sensed. Describes polar gestures (circular motion around a very small display) [HarrisonC10a] Harrison, Chris; Tan, Desney; and Morris, Dan "Skinput: Appropriating the Body as an Input Surface", CHI 2010, April 10-15, 2010, Atlanta, Georgia Surface (?) acoustic wave design of touch input device comprising an arm band and propagating acoustic signals through human skin: a wearable SAW touch input digitizer. Also refers to distinctive sounds made when tapping on parts of body: arm, palm fingers and thumb. File contains additional materials [HarrisonC10b] Harrison, Chris; Tan, Desney; and Morris, Dan "Search by Sketch", www.chrisharrison.net Project description: Compare with earlier work on sketch recognition: use uses paint interface (not draw/sketch) to draw color patterns, system matches by dominant color and rough orientation, providing results in real time while user is "sketching". See also video file. [HarrisonC10c] Harrison, Chris and Hudson, Scott E. "Minput: Enabling Interaction on Small Mobile Devices with High-Precision, Low-Cost, Multipoint Optical Tracking", CHI 2010, April 10-15, 2010, Atlanta GA



Optical-mouse type input built into portable device (basically on back of display). Thus display is not occluded by finger or touching, no digitizer on display. Gesture recognition for flick, twist, curly-cue gesture. Can also scan a larger surface, thus can do scroll-bar gestures. File contains additional references. [HarrisonP08a] Harrison, Phil "Hand-held Device with Touchscreen and Digital Tactile Pixels", United States Patent Application 2008/0150911, June 26, 2008 Haptic/tactile vibration pixels around edges of touchscreen PDA computer [HartmannC08a] Hartmann, Christoph "Microsoft Visual Programming Language: End-User Perspektive", Monograph, www.acidum.de, Aug 11, 2008 (in German) Microsoft Robotics Studio 2006, Microsoft Visual Programming Language: Review of visual programming languages and environments: cites to Sketchpad 1963, GRAIL 1967, AMBIT/G 1968, ThingLab and other early systems. [HaywardP08a] Hayward, Vincent; Alarcon, Ramon; Rosenberg, Louis B. "Haptic Pads for use with User-Interface Devices", United States Patent 7,336,266, February 26, 2008 Add a haptic output actuator to an existing input devices. Generic to type of input device, and to type of haptic actuator. [HelftM09a] Helft, Miguel "For the Blind, Technology Does What a Guide Dog Can't", The New York Times, January 4, 2009 T.V. Raman, visually impaired researcher at Google, accessibility technology for low-vision on touchscreen cell phones (e.g. Apple iPhone) using Gestures [HembrowD10a] Hembrow, David "EO Personal Communicators EO440 / EO880 running GO PenPoint", www.hembrow.eu, fetched 2010 Personal notes on EO/PenPoint in the UK, including examples of documentation and product. [HewlettPackard08] Hewlett-Packard "HP Pavilion TouchSmart tx2z series", www.shopping.hp.com/series/category/notebooks/tx2z_series TouchSmart 2x2z ultraportable Tablet PC: multi-touch input, implies support for multi-touch gestures on laptop running Windows Vista [HillisWD10a] Hillis, W. Daniel and Ferren, Bran "Touch driven method and apparatus to integrate and display multiple image layers forming alternate depictions of same subject matter", United States Patent 7,724,242, May 25, 2010 Tabletop computer system: slide/zoom/pinch gestures include inertia, may show additional areas of an image or of a second image when moved. Long list of references: See also TouchTable (assignee), appliedminds.com (IP shop). Railing around table to keep users from resting hand on tabletop touchscreen. [HinckleyK09a] Hinckley, Ken; Dixon, Morgan; Sarin, Raman; Guimbretiere, Francois; and Balakrishnan, Ravin "Codex: A Dual Screen Tablet Computer", Proc CHI 2009, April 4-9, 2009, Boston, MA Dual-screen dual-display/dual-touchscreen tablet, 4x6 inch. Compare with Courier and Refalo. Software based on InkSeine note-taking with "floating tool ring" (marking-menu?). Multiple references to other current dual screen computers. Taxonomy of dual-screen "postures": landscape/portrait, book/lectern/laptop, corner-to-corner/face-to-face, battleship (opposite directions). Dual displays can be detached. Collaborative use. Dual-display use includes webpage next to notebook page. [HinckleyK09b] Hinckley, Ken; Bathiche, Steven N.; Vong, William; Cauthorn, James H.; Ladas, Corey M.; and Fry, Kenneth V. "Method for Displaying Information Responsive to Sensing a Physical Presents Proximate to a Computer Input Device", United States Patent 7,602,283, October 13, 2009: also 7,602,382 Provide feedback using proximity data on a touchscreen digitizer. Compare with GO operating system, PenWindows. [HinckleyK10a] Hinckley, Ken; Pahud, Michel; and Buxton, Bill "Direct Display Interaction via Simultaneous Pen + Multi-Touch Input", Society for Information Display (SID) Symposium Digest of technical Papers, May 2010, Vol 41(1), Session 38, pp. 537-540 Using pen and touch gestures simultaneously, users perform different actions with the two hands -dominant hand holds pen/stylus for writing. Cites to earlier work on two-handed/multi-touch user interfaces. [HinckleyK10b] Hinckley, Ken; Yatani, Koji; Pahud, Michel; Coddington, Nicole; Rodenhouse, Jenny; Wilson, Andy; Benko, Hrvoje; and Buxton, Bill "Pen + Touch = New Tools", Proc. UIST '10, October 3..6, 2010, New York, pp. 27-36 Combining stylus/pen input and multi-touch gestures on tabletop: e.g. while writing (electronic ink) with pen on two pages of virt zooms back to full virtual desktop view. [HirschM09a] Hirsch, Matthew; Lanman, Douglass; Holtzman, Henry; and Raskar, Ramesh "BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Fields", ACM Trans of Graphics, Vol 28 No 5, December 2009 Bidirectional LCD sensor/display: LCD display with 3D optical touchscreen and proximity (3D) sensing using synthetic apertures onto optical sensors behind each LCD pixel: no change to LCD. Permits 3D user input



manipulation, proximity changes zoom. [HoTK09a] Ho, Tsz-Kin; Lee, Chung-Yung; Tseng, Man-Chun; and Kwok, Hoi-Sing "Simple Single-Layer MultiTouch Projected Capacitive Touch Panel", SID 09 Symposium Digest of Technical Papers, pp. 447..540, Proc. of SID Symposium, 2009 Single layer multi-touch ITO capactitive touchscreen / touch panel sensor. Rows of separate sensing electrodes, each with own trace to side of substrate. Alternating rows separated by single sensing electrode trace across entire surface. [HolmanD08] Holman, David and Vertegaal, Roel "Organic User Interfaces: Designing Computers in Any Way, Shape, Or Form. Displays on real-world objects allow more realistic user interfaces", CACM, June 2008, Vol 51 No 6; pp 48-ff. Organic user interfaces / tangible user interfaces: deformable user-input devices, such as foldable paper which is also a display, and responds to folding. Shows multiple-touch, multiple-user actions on a globe model, cites to Wellner. [HolzC10a] Holz, Christian and Baudisch, Patrick "The generalized perceived input point model and how to double touch accuracy by extracting fingerprints", Proc. CHI 2010 Experimental system using large-area fingerprint scanner as touchscreen: Address fat finger problem (inaccuracy of fine touch with broad fingertip) by recognizing fingerprint to identify user, and adjusting for typical offset/posture shown by that user. Ascribes error to users having different interpretations of touch / touch location. [HookJ09a] Hook, Jonathan; Taylor, Stuart; Butler, Alex; Villar, Nicolas; and Izadi, Shahram "A Reconfigurable Ferromagnetic Input Device", Proc. UIST '09, October 4-7, 2009, Victoria, BC, Canada, pp. 51=54 X/Y grid matrix of electromagnetic sensing coils used to detect ferrous / ferromagnetic objects: objects need not be tagged, if ferrous. Placing flat ferrofluid bladder on array is touch/force/pressure touchpad surface, interpolated to get higher location resolution. One use is sensing of actual objects: e.g. physical slider with ferrous handle. [HornBL10a] Horn, Bruce L. "Computer System for Automatic Organization, Indexing and Viewing of Information from Multiple Sources", United States Patent 7,840,619, November 23, 2010 Displaying lists or tables of objects (e.g. .JPGs, .MPG3, etc.) automatically sorted by different attributes, and sub-trees (e.g. folders) can be expanded or contracted in the list. Refers to sticky paths for references to objects. [HorneckerE09a] Hornecker, Eva "Tangible interaction", Interaction-Design.org: http://www.interactiondesign.org/encyclopedia/tangible_interaction.html, 2009 Essay defining tangible interaction as distinct from direct manipulation in user interfaces: physical objects interact with users under computer control: numerous references to background of the art. See also augmented reality. [HorowitzBT08a] Horowitz, Brian T. "Synaptics touch pad tech gains momentum", blog.internetnews.com/bhorowitz, June 26, 2008 Scrolling momentum / dynamic scrolling / inertial scrolling by Synaptics in a touchpad digitizer, compared to trackball that keeps spinning after initial push [HoshiT09a] Hoshi, Takayuki; Takahashi, Masafumi; Nakatsuma, Kei; and Shinoda, Hiroyuki "Touchable Holography", Proc. SIGGRAPH 2009, New Orleans, August 3..7, 2009 Prototype ultrasound tactile display with 3D three dimensional finger sensor. Display is array of 324 ultrasound transducers, phase and amplitude control at 40kHz, produces single pressure focal point 20 mm diameter, AM modulation for vibration up to 1 kHz. Sensation compared to feeling a rain drop strike hand. Does not discuss hand blocking/obscuring either sensor or display. Touch sensor is IR reflective tape on fingertip with WII infra-red sensor. [HotellingS09a] Hotelling, Steve Porter; Zhong, John Z. "Single-layer touch-sensitive display", United States Patent Application 2009/0091551, April 9, 2009 Single-layer multi-touch/Multipoint mutual capacitive touchscreen (not really a display). Compare with Cypress? [HotellingS09b] Hotelling, Steven P. "Method and Apparatus to Reject Accidental Contact on Touchpad", United States Patent 7,561,146, July 14, 2009 proximity-sensing hand/palm rest near keyboard: if detect user's palm/hand near keyboard, filter out touch input (palm rejection). [HotellingS10a] Hotelling, Steve; Strickon, Joshua A.; and Huppi, Brian Q. "Multipoint Touchscreen", United States Patent 7,663,607, February 16, 2010 Multi-touch/Multipoint mutual capacitive touchscreen: compare to Fingerworks and Rekimoto? [HotellingS10b] Hotelling, Steve; Ligtenberg, Chris; Kerr, Duncan; Andre, Bartley K.; Strickon, Joshua A.; Huppi, Brian Q.; Chaudhri, Imran; Christie, Greg; and Ording, Bas "Wide Touchpad on a Portable Computer", United States Patent 7,834,855, November 16, 2010



Touchpad the full width of a laptop computer, instead of just a touchpad in the middle. Allows user to move hand less away (horizontally) from keyboard position to use touchpad. Palm discrimination by sensing fingers over keyboard, ignore touch input on touchpad (using separate proximity sensor, or visual sensor above keyboard on top edge of display screen). [HuppiB09a] Huppi, Brian; Fadell; Anthony M.; Barrentine, Derek; and Freeman, Daniel "Automated Response to and Sensing of User Activity in Portable Devices", United States Patent 7,633,076 Motion and/or position sensor in portable cell phone, PDA or hand-held unit to detect user activity: motion / orientation sensor on handheld tablet device? iPhone/iPad [IOGEAR08a] IOGEAR "IOGEAR Digital Scribe", iogear.com Acoustic/sonic digitizer, single acoustic receiver that mounts on corner or top of paper (or other surface) and transmitter stylus/pen with optional ink cartridge. GPEN100C, Applications include note taking (Microsoft OneNote, IRISnotes, etc.) and handwriting recognition (MyScript). Device may also be used as stylus or in mouse-input mode with side button on stylus barrel. File contains additional references [IPad10] iPad "iPad file information", Misc Miscellaneous iPad information [IRIS10a] I.R.I.S. "Readiris 12 Pro", Irislink.com, fetched 2010 OCR software for IRISpen and other scanner devices. IRISnotes software for IOGEAR acoustic stylus digitizer tablet: compared with products requiring special paper (Anoto) and optical digitizer. See also IOGEAR file. [IRexTechnologies09] iRex Technologies BV "iRex Technologies Turns the Page on eReaders with new 8.1-inch Consumer Device", iRex Technologies BV, Press Release Sep. 23, 2009 iRex Technologies e-book reader: marketed with additional applications (compare with PenPoint from GO): uses Wacom capacitive/electrostatic touch digitizer tablet component. Additional materials in file. [IgarashiT09a] Igarashi, Takeo and Igarashi, Yuki "Implementing As-Rigid-As-Possible Shape Manipulation and Surface Flattening", Journal of Graphics, GPU and Game Tools, Vol 14 No 1, pp. 17..30, June 2009 On multi-touch touschscreen UI, user can streatch and distort figures (animations), object is triangular mesh, minimized aggregate distortion of mesh elements. Extension to 3D three-dimensional graphics. [Immersion08] Immersion Corporation "MicroScribe Systems: Solutions for Inspection, Reverse-engineering, and 3D Digitizing", www.immersion.com Mechanical digitizers using multiple-link mechanical arms, articulated-arm digitizers and portable CMMs. Intended for three-dimensional / 3D digitizing of engineering objects. Applications also include medical and surgical simulators : digitizers with haptic feedback under the product name CyberForce. Several citations to studies of touchscreens with and without tactile/haptic feedback, and VibeTonz auditory/haptic components for hand-held devices (cell phones, PDAs, etc.) [Immersion10a] Immersion Corporation "IMMR Haptic Links", www.immersion.com Listing of papers (including research papers) from immersion.com for haptic/feed-back [Immersion10b] Immersion Corporation "The Value of Haptics: A summary of published findings on the value of haptic feedback in human-computer interaction", Immersion Corporation, San Jose, California, 2010 Haptic: Tactile Feedback, Multi-modal feedback, non-visual interaction. General explanation of haptic terms. [ImpedovoD08a] Impedovo, Donato and Pirlo, Giuseppe "Automatic Signature Verification: The State of the Art", IEEE Trans. on Systems, Man, and Cybernetics, Vol 38 No 5, September 2008, pp. 609-35 Signature authentification / signature recognition as biometric identification: notes that paper documents are non "de-materializaing", authentication of paper documents still needed. [InPlay08a] InPlay Technologies "InPlay's State-of-the-Art Technology Produces Industry's First Intelligent Pen for Tablet PCs", www.inplaytechnologies.com, 2007/2008 Press releases and technology descriptions for electromagnetic tablet digitizer for Tablet PCs. Follow Microsoft model of active/passive classifications, passive is front-mounted resistive touchscreen, active is rear-mounted electromagnetic. Grid-powered and battery-powered pen/stylus. Multiple pens/stylus identifiable by ID code: (MagicPoint Digitizer writing pen) pens for colors, or identifying multiple users (also multi-touch?). InPlay: Digitizer Technologies for Mobile Computing Applications InPlay: Active Pen and Digitizer Human Input Devices Digital (InPlay) vs. Analog (Others) InPlay: MP800 Overview: tablet/MagicPoint pen/stylus, custom interfacing chip set for electromagnetic digiti9zers. [Independent09a] The Independent, United Kingdom "Why touch screens push our buttons", The Independent, 9 September 2009 Mostly interview with Chris Harrison and Bill Buxton regarding multi-touch input and touchscreens: mentions Casio touch-screen AT-550 wristwatch in 1980, Lemur touch-surface music controller in 2003 (see reference) quotes Bill Buxton on touchscreens being unusable by blind/visually impaired (compare with iPhone VoiceOver?) Chris Harrison latex touch-surface covering. [InformationDisplay10a] SID Society for Information Display "The Best of Times For Touch: Touch Technology



Issue", SID Information display, www.informationdisplay.org, March 2010, Vol 26 No 3 SID special issue on touch technology (rather than displays), especially multi-touch digitizers. Compare with 1990's article in Information Display by Ward/Tappert? Advertisement: IRTouch multi-touch is ready for Windows 7 Editorial: The Limitless Horizon for Touch Article: LCD In-Cell Touch Article: ProjectedCapacitive Touch Technology Article: The State of the Touch-Screen Market in 2010 Article: Touch Screens and Touch Surfaces are Enriched by Haptic Force-Feedback: inertial actuation, piezo actuation, (electrostatic) surface actuation, lateral actuation, (contracting) electro-active polymer actuation, bending wave, capacitive electrosensor interface (CEI) in nerve endings Journal of the SID Article: Beneath the Surface (Microsoft Surface) Journal of the SID Article: Taking Touch to New Frontiers: Why It Makes Sense and How to Make It Happen News: FlatFrog Laboratories optics-based multi-touch kits; touchco flexible touchscreens [IraniP10a] Irani, Pourang; Shi, Kang; Cechanowicz, Jare; and Subramian, Sriram "Pressure Augumented Mouse", United States Published Patent Application 20100127983, May 27, 2010 Mouse with additional pressure sensors: references contrast with WIMP interfaces [IshiiH08a] Ishii, Hiroshi "The Tangible User Interface and Its Evolution", CACM, June 2008, Vol 51 No 6, pp. 3236 Special issue on organic/graspable/haptic/tangible user interfaces. Deformable/rollable input devices, such as recognizing shadows of physical stick figures or clay models. [IwamotoT08a] Iwamoto, Takayuki; Tatezono, Mari; and Shinoda, Hiroyuki "Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound", Proc. EuroHaptics 2008, pp. 504..513 Prototype tactile feedback for 3D three dimensional display by sound pressure on skin (of hand/fingers) from hexagonal/annular array of ultrasound/ultrasonic transducers. Resolution about 20mm -- comparable to width of fingertip. Compare with UltraHaptics. [IwasakiK09a] Iwasaki, Ken; Miyaki, Takashi; and Rekimoto, Jun "Expressive Typing: A New Way to Sense Typing Pressure and Its Applications", Proc. CHI 2009, April 4-9, 2009, Boston MA. Measure touch-typing pressure (actally force -- no, actually acelleration) using built-in exceleropeter in MacBook portable computer. Applications for pressure form mode switch, emphasis/Bolding in messaging, biometric authentication using typing motion pattern. Compare to application of stylus/finger pressure/force in touchscreen/tablet applications. [IzadiS08a] Izadi, Shahram; Hodges, Steve; Taylor, Stuart; Rosdenfeld, Dan; Willar, Nicolas; Butler, Alex; and Westhues, Jonathan "Going Beyond the Display: A Surface Technology with an Electronically Switchable Diffuser", Proc. UIST '08, October 19-22, 2008, Monterey, California pp. 169-278 SecondLight: Combined two-image display and multi-touch sensor: surface has cells (like LCD) electronically switchable between diffusing and transparent state: in transparent state, image from display behind is visible, and an optical camera behind the display can see objects above the surface. In diffusing state, rear display not visible, but reflects second image projected from above. At the same time, can sense touching fingers on surface from reflected light by FTIR frustrated total internal reflection. Mentions measuring area of contact of fingertip as indirect measure of pressure/force, and major and minor axis of oval to determine rotational orientation/angle of finger tip. [IzadiS09a] Izadi, Shahram; Hodges, Steve; Butler, Alex; West, Darren; Rrustemi, Alban; Molloy, Mike; and Buxton, William "ThinSight: a Thing Form-factor Interactive Surface Technology", CACM, December 20009 Vol 52 NO 12, pp. 90..99 Version of 2007 and 2008 papers: Multi-touch optical touchscreen: mount array of infra-red transmitters and receivers (coarse) behind and LCD, detect reflection of infra-red reflective fingers and objects through LCD. Bi-cubic interpolation to interpolate to higher accuracy for "center" of a finger touchpoint. Cites to Rekimoto. [JainAK08a] Jain, Anil K.; Flynn, Patrick; and Ross, Arun A. "Handbook of Biometrics", Springer-Verlag, 2008 (hardcopy book) Handbook/dictionary reference work on biometric technologies for authentication of identity. Section on spoofing of biometric input, including spoofing of fingerprints back to 1920's. [Janam08a] Janam Technologgies LLC "Janam XP30 User Guide", Janam Technologies LLC, 2008 Janam XP30 touchscreen/digitizer smart phone: Uses Graffiti 2 gestures: backspace, shift, menu (diagonal) Shortcut (bottom curlique, cut (pig-tail), paste gesture, copy (down-left), undo [JarrettRJ09a] Jarrett, Robert J.; Duncan, Richard J.; Ribas-Ribikauskas, Emily K. "Scrolling Web Pages using Direct Interaction", United States Patent 7,519,920, April 14, 2009 Scroll/drag a web page using a stylus on a touchscreen: when move stylus to perimeter, page will autoscroll with simulated momentum. [JayantC10a] Jayant, Chandrika; Acuario, Christine; Johnson, William A.; Hollier, Janet; and Ladner, Richard E. "VBraille: Haptic Braille Perception using a Touch-screen and Vibration on Mobile Phones", Proc. ASSETS '10, October 25..27, 2010 Orlando FL



Simple haptic text display for deaf-blind: smartphone touchscreen divided into six parts to mimic Braille dots, vibrates when user's finger is over a part corresponding to a raised Braille dot. [JazzMutant09a] JazzMutant.com "Lemur: Schliessen Sie Ihren Lemur an", www.jazzmutant.com MIDI controller/GUI software toolkit for touch gestures specific to controlling music and audio systems. multi-touch. [JazzMutant09b] JazzMutant.com "Lemur V2.0 User Guide", www.jazzmutant.com MIDI controller/GUI software toolkit for touch gestures specific to controlling music and audio systems. multi-touch. [JazzMutant09c] JazzMutant.com "Behind the Lemur", www.jazzmutant.com multi-touch digitizer capable of handling an unlimited number of fingers at once: genesis of the device was that the developers designed a user interface assuming such a device was available. multi-touch. [JobsSP09a] Jobs, Steve P. et al "Touch Screen Device, Method, and Graphical User Interface for Determining Commands by applying Heuristics", United States Patent 7,479,949 B2, Jan 20, 2009 Gesture (multi-touch gesture) recognition by heuristics, Numerous other iPad/iPhone gesture patents incorporated by reference. Specific to finger contacts / touchscreens, not other stylus digitizer input. Very large spec (362 pages), shows slide-to-unlock. [JoguetP10a] Joguet, Pascal and Largillier, Guillaume "Multipoint touch sensor with active matrix", United States Patent Application 2010/0066686 A1, March 18, 2010 Includes description of JazzMutant/Stantum multi-touch touchscreen, overcomes masking/shadowing effect of simple switch matrix designs by using active matrix switching array (similar to active matrix LCD circuitry) to measure each coordinate in grid of perpendicular wires separately. Also refers to using conductive rubber or piezo switch elements at nodes. See also 2007 SID article by Largillier [JonesR09a] Jones, Randy; Driessen, Peter; Schloss, Andrew; and Tzanetakis, George "A Force-Sensitive Surface for Intimate Control", Proc. New Interfaces for Musical Expression NIME09, June 3-6, 2009, Pittsburgh, USA Madronalabs.com Madrona Soundplane: force/pressure sensing X/Y touchsurface, intended as controller for electronic musical instruments. Multi-touch sensing. Cites to earlier devices. "Control intimacy" -- tightlycoupled haptic feedback? Capacitive grid force-sensitive touchscreen tablet: X and Y antennae separated by a rubber/elastomeric sheet, pressing brings wires closer together for stronger capacitive coupling. Cites to Radio Drum from 1980's. Used as musical instrument interface. [JonesRE08a] Jones, Randall Evan "Intimate Control for Physical Modeling Synthesis", Master's Thesis, Dept. of Comp. Sci, U. of Victoria, BC, Canada Multi-dimensional force sensor: multi-touch sensor used to sense dynamic pressure image over the touchpad surface. Grid of X and Y conductors, forming capacitive-sensor nodes, separated by layer of rubber dielectric. Dynamic calibration to compensate for mechanical hysteresis of rubber sheet (persistent pressure indication after touch has ended). Interpolation of values plus centroid detection, correlating to centroids detected in previous sample (tracking touches). [KaneSK08a] Kane, Shaun K.; Bigham, Jeffrey P. and Wobbrock, Jacob O. "Slide rule: Making mobile touch screens accessible to blind people using multi-touch interaction techniques", Proc. ASSETS '08, October 13-15 2008, Halifax, Nova Scotia, pp. 73i80 Accessibility multitouch touchscreen gestures for visually impaired: compare with Apple VoiceOver gestures? Second-finger tap (see earlier touchscreen gestures w/o multi-touch), "L" gestures, one-finger scan/scroll, flicks. No citation to VoiceOver. [KarlsonAK08a] Karlson, Amy K. and Bederson, Benjamin B. "One-Handed Touchscreen Input for Legacy Applications", Proc. CHI 2008 April 5-10, 2008, Florence Italy, pp. 1399-1408 For one-handed PDA use, user defines a ThumbSpace -- the area reachable by the thumb while holding the PDA in one hand. Show thumbnail (sic) image of whole screen in ThumbSpace for user to pick from on whole touchscreen. Refers to whole screen touching as DirectTouch. [KentJC08a] Kent, Joel C.; Aroyan, James L.; Gao, Ting; Tanaka, Yoshikazu; and Scharff, Daniel H. "Touchscreen for detectig multiple touches", United States Patent Application 2008/0266266 A1, October 30, 2008 Optical (interrupted light beam) touchscreen, detects multiple touches unambiguously by projecting additional diagonal beams across surface: diagonal beams produced by same narrow-angle infrared LEDs by means of beam splitters. [KimSC08a] Kim, Seung-Chan; Kim, Chong Hui; Yang, Tae-Hon; Yang, Gi-Hun; Kang, Sung-Chul; and Kwon, Dong-Soo "SaLT: Small and Lightweight Tactile Display using Ultrasonic Actuators", Proc. 17th IEEE Int'l. Symposium on Robot and Human Interactive Communication, TU München, Munich, Germany, August 1-3, 2008, pp 430..435 Small curved pin-cushion unit for tactile displays -- actually for wearables? Simulates texture both by static pins (4x4 grid) and by vibration (up to 20 Hz). Compare with HyperBraille / BrailleDis 7200. [KimSC09a] Kim, Seung-Chan; Kim, Chong Hui; Yang, Tae-Hon; Han, Byung-Kil; Kang, Sung-Chul; and Kwon, Dong-Soo "Small and Lightweight Tactile Display (SaLT) and Its Application", Proc. 3rd Joint Eurohaptics Conf.



and Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Salt Lake City, March 18..20, 2009 Small curved pin-based unit for tactile displays -- wearables, tactile texture display on mouse. Simulates texture both by static pins (4x4 grid) and by vibration (up to 20 Hz). Compare with HyperBraille / BrailleDis 7200. [KlimovsV09a] Klimovs, Vjaceslavs and Pisarenko, Sergejs "Towards enhancing perceived performance through adoption of proposed benchmarking techniques", Report 2009-029, University of Gothenburg, Germany, 2009 Report on perceived versus actual performance in user interactions: mentions "kinetic scrolling" and "twoway scrolling". Compare with Apple slider/slide-to-unlock? [KolokowskyS10a] Kolokowsky, Steve and Davis, Trevor "Understanding key specs in reliable capacitive touchscreens", EE Times-Asia / eetasia.com, 2011 Cypress Semiconductor white paper on specifications for capacitive touchscreen using XY grid for multitouch digitizers. Accuracy, finger separation (minimum discernable distance), response time, refresh rate (a.k.a. report rate or data rate in samples/sec), power consumption. Compare with Ward paper on digitizer technology: paper does not mention stylus tilt, registration, noise, fixed or variable error, etc. [KrempM08a] Kremp, Matthias "Microsoft bittet zu Tisch", Spiegel news magazine (in German), January 10, 2008. www.spiegel.de Report from Consumer Electronics Show on Microsoft digital table / surface table touch digitizer, with direct manipulation of photographs. References to iPhone interface. Digitizer touch sensing limit is 52 touch points: four players, 10 fingers plus 3 playing figures each. Compare with Wang Freestyle for graphical manipulation? [KrempM08b] Kremp, Matthias "Windows 7 bekommt iPhone-Interface", Spiegel news magazine (in German), May 28,2008. www.spiegel.de Report on Microsoft Windows 7 touch-UI interface. Also mentions digital table / surface table touch digitizer, with direct manipulation of photographs. Points out awkward positioning of touch screens at a desk, compares with keyboardless one-laptop-per-child touch system. [KrempM10a] Kremp, Matthias "Nokias letztes Aufgebot: Smartphone N8", Spiegel news magazine (in German), October 20, 2010 www.spiegel.de Nokia N8 cellphone with touch-screen interface, runs videos (Linux OS?) and GPS Ovi Maps no charge. [KroekerKL09] Kroeker, Kirk L. "Electronic Paper's Next Chapter", CACM, Vol 52. No 11, November 2009, pp. 1517 Overview report on E-Ink, Plastic Logic, and other vendors of electronic paper and flexible displays. E-Ink acquired by Prime View International of Taiwan, which manufactures the Kindle. Mentions color displays: no information on tablets or digitizers. Electrophoresis, electrophoretics, electrofluidic technologies. [KruegerMW09] Krueger, Myron W. "Myron W. Krueger", en.wikipedia.org Mentions computer input of two-arm gesture holding arms out to sides, recognized optically [KurtenbachG09a] Kurtenbach, G. "Demo of Marking Menus Versus Linear Menus", YouTube.com, posted 2009 Demo of marking menus versus linear menus: date of video probably earlier than posting date. [LapstunP09a] Lapstun, Paul; Bertok, Attila; King, Tobin Allen; Underwood, Matthew John; and Silverbrook, Kia "Collapsible force sensor coupling", United States Patent 7,523,672, April 28, 2009 Netpage pen/stylus: piezo-electric force sensor in stylus, with spring to absorb shock and protect sensor if hit hard. Netpage is an optical position-sensor stylus using camera and infra-red illuminator/LED and a dotlike encoding of position. Cites to Sekendur. Compare with Anoto? [LeeJH10a] Lee, Jae-hoon "Audio/video Device Having a volume Control Function for an External Audio Reproduction Unit by Using Volume Control Buttons of a Remote Controller and Volume Control Method Therefor", United States Patent 7,672,470 B2, March 2, 2010 Volume control on (on touchscreen) when an external amplifier/speakers (headset) are plugged in. Related to LabView? [LeeJK10a] Lee, Jong-Kwon; Kim, Sang-Soo; Park, Yong-In; Kim, Chang-Dong; and Hwang, Yong-Kee "In-cell adaptive touch technology for a flexible e-paper display", Solid-State Electronics, Vol 56 (2011), pp. 159-162, available online 18 November 2010 Light-sensitive touch panel with display, photo-sensor arrays integrated into cells of electronic ink paper display. Capacitive coupling to finger to avoid shadowing of light. [LiKA08a] Li, Kevin A.; Baudisch, Patrick; and Hinckley, Ken "BlindSight Eyes-Free Access to Mobile Phones", Proc. CHI 2008, April 5..10, 2008, Florence Italy, pp. 1389..1398 Accesibility adaptation of smartphone with physical keypad for the blind: larger buttons, wider spacing, static mapping of menus to keys. No citation to VoiceOver. Electronic file includes presentation slides. [LiY08a] Li, Yang; Klemmer, Scott; and Landay, James A. "Tools for Rapidly Prototyping Mobile Interactions", Chapter XXI in Handbook of Research on User Interface Design and Evaluation for Mobile Technology, National Research Council of Canada, Joanna Lumsden Ed., 2008, pp. 330-345



Prototyping UI design using UIMS with storyboarding, either with paper or with software tools. Prototype with Toshiba Tablet PC (Microsoft) and HP iPAQ Pocket PC with interactive maps. [LightBlueOptics10a] Light Blue Optics "Light Speed, Light Touch, Light work", www.lightblueoptics.com, fetched 2010 Holographic projector and infrared sensor for creating a multi-touch "touch screen" on any surface on which an image can be projected. [LinJC08a] Lin, Jao-Ching; Huang, Shyh-In; Chu, Lin-Abel; and Shen, Chung-Yi "Method of scrolling window screen by means of controlling electronic device", United States Patent 7,319,457, January 15, 2008 Scrolling using multiple taps in scrolling press zones: more taps, faster scrolling, also tap to stop scrolling. [LindemannP10a] Lindemann, Patrick "A Short Report on Multi-touch User Interfaces", Dept. of Media Informatics, University of Munich, Germany: wiki.medien.ifi.lmu.de, Fetched 2010 Review of multi-touch gestures. Cites to 2003 for flick gestures (with hand or finger), but does not mention PenPoint/GO. Cites Buxton on multi-key/n-key rollover on a keyboard in 1970's being a form of multitouch. [LinuxForDevices10] LinuxForDevices.com "Multi-touch displays support all ten fingers at once", www.LinuxForDevices.com, 2010-02-18 multi-touch digitizer capable of ten fingers concurrent/simultaneous input. PMatrix from Stantum. Resistive touchscreen display. Quotes Jim Meador at Stantum regarding non-proprietary nature of Apple iPad pinch gesture. Compare with JazzMutant. [Livescribe08a] Science Daily "Smart Pens Help Blind See", www.sciencedaily.com, May 9, 2008 Voice/handwriting recording using Anoto digital pen and digitizing (tablet) paper. Includes synchronized voice recording. Article / press-release attempts to make accessibility connection to being annotation device for the visually impaired. [Livescribe08b] LiveScribe, Inc. "Livescribe: Never Miss A Word (Smartpen)", www.livescribe.com Voice/handwriting recording using Anoto digital pen and digitizing (tablet) paper. Includes synchronized voice recording built into pen. Touch previous mark, pen plans recorded audio recorded at that time of writing: example is write "define" followed by word, get definition. Uses Anoto technology, see also EchoPen. [Livescribe08c] Various: Naone, Erica "Livescribe Pulse: Handwrtten notes get digital audio support / Create Talking Dots Shapes and Labels", www.livescribe.com Collection of articles on Livescribe / Anoto pen. Tear-down photographs, applications e.g. talking dots aid for blind, labeling of physical objects. [Livescribe10a] Fahlberg, Tim "Livescribe Smartpen: How (and why) to Enlarge audio Dots (+How to Make them Findable for those with Vision Loss]", www.edlivescribe.com, December 8, 2010 Use Anoto Pen (Livescribe Pulse or Echopen) and printed labels to make audio notes that can be played back by visually impaired/blind user or motor impaired by touching sensor to printed dot. File contains additional references, such as LiveScribe/Anoto Post-it notes. UI for creating notes still requires use of printed paper with control functions. Compare synchronized voice and handwriting of Freestyle with LiveScribe Anoto pen. [LjLies10a] LjLies (YouTube) "Video demonstration: Casio PF-8000 Calculator, circa 1984", YouTube.com, uploaded 2010 Video demonstration of Casio PF-8000 calculator, using keys as digitizer input for recognizing handwritten numerics and math operator symbols. [LucentMicrosoft08a] Associated Press: Jessica Mintz "Microsoft to Appeal $367M Patent Ruling", Associated Press, April 4, 2008 Microsoft found to infringe GO patent on using gestures on Tablet PC. Other patents in case concern video compression, forms input, and color management. [LucentMicrosoft08b] LucentMiccrosoft "Exhibits: Lucent v. Microsoft technical references", Lucent v. Microsoft Exhibits (technical references) in Lucent v. Microsoft in San Diego: long list. [LucentMicrosoft08c] Lucent v. Microsoft "Lucent v. Microsoft", Press announcement file Press articles on Lucent v. Microsoft patent case concerning United States Patent 5,544,295 on tablet PC and gesture-based user interface [LuckieD10a] Luckie, D. "Newton Hall of Fame", www.msu.edu/~luckie/halloffame.htm Biographical information of major personalities in Apple Newton development team: Steve Capps (Finder), Walter Smith (NewtonScript, a.k.a. WallyScript), John Sculley, Michael Tchao ("Apple had been working for years on slate computers that read handwriting"), Stepan Pachkov, Larry Yaeger (Cursive Recognizer) See also 2002 version [LuckieD10b] Luckie, D. "Newton Gallery", www.msu.edu/~luckie/newtgal.htm Detailed specs on Apple Newton MessagePads, several models,: OS 2.0, 2.0 1.3, and 1.0 released 1993. Associated with Steve Capps.



[MSDN10a] Microsoft MSDN Block "The Old New Thing: If Windows 3.11 required a 32-bit processor why was it called a 16-bit operating system: Windows 3.11 Standard vs. Enhanced Mode", blogs.msdn.com, 17 May 2010 Architectural overview of Windows 3.11 operating systems: Windows 3.11 standard mode -- 16-bit protected mode / Enhanced Mode: 32-bit virtual machine manager in 32-bit protected mode, running standard mode windows and DOS as virtual machines. Enhanced mode as new operation system with preemptive multi-tasking and virtual memory. Win32S application ran in 32-bit mode. [MacKenzieIS10a] MacKenzie, I. Scott and Felzer, Torsten "SAK: Scanning Ambiguous Keyboard for Efficient OneKey Text Entry", ACM Trans. on Computer-Human Interaction, Vol 17 no 3, Article 11, July 2010 Scanning / virtual keyboard for accessibility for physically impaired: single key or other input, display scans through character, user selects character with key when the character comes up. One reference to touchscreens. [MackeyBL08a] Mackey, Bob Lee "One layer capacitive sensing apparatus having varying width sensing elements", United States Patent 7,382,139, June 3, 2008 Capacitive/electrostatic tablet, single layer, with three sets of (horizontal) conductors with wavy widths. Which conductor(s) give coupling give Y oridinate, degree of coupling between adjacent conductors (of the three) gives X ordinate. Alternative "fishbone" pattern has cross hatches of different widths. Compare with charge-ratio tablets? [MaddalozzoJ10a] Maddalozzo, John Jr.; BcBrearty, Gerald Francis; and Shieh, Johnny Meng-Han "Method and system for touch screen keyboard and display space sharing", United States Patent 7,768,501, August 3, 2010 Virtual keyboard (transparent/translucent: or may display image of touchpad, or other image) comes up on touchscreen when proximity of user's hands is detected, removed when hands no longer near surface. [Makowski08a] Makowski, Dirk "Windows für Pen Computing 1.0", www.winhistory.de, fetched September 6, 2008 Screen shots of Windows for Pen Computing 1.0 on Windows 3.11 from 1992, German-language version. All applications (but not OS) were pen-enabled by means of pen tool "Stiftpalette". Notebook / Notizbuch application allowed free-hand electronic ink note-taking. Gestures/Gesting include check-mark to edit text, similar to PenPoint OS. Comparison with Windows for Pen Computing 2.0 on Windows 95, which is described as more of an add-on and less integrated. Shows multiple gestures accepted in text editor: check mark to edit text, space, new line, cut (pigtail), undo (loop), backspace. [Makowski08b] Makowski, Dirk "Windows XP Tablet PC Edition (Codename "Lonestar"); 2005 edition", www.winhistory.de, fetched September 6, 2008 Screen shots of Windows XP Tablet PC edition, German-language version. Lonestar released in 2002. [Makowski08c] Makowski, Dirk "Windows Vista", www.winhistory.de, fetched September 6, 2008 Screen shots of Windows Vista, German-language version, 2007. Includes Tablet PC functionality. [Makowski10a] Makowski, Dirk "Windows für Pen Computing 1.0", www.winhistory.de, 2010 Revised version of 2008 posting on Windows for Pen computing 1.0 on Windows 3.11 from 1992. Includes video (see file) demonstrationg PenWindows 1.0. [MarriotG09] Marriot, Greg; Bar-Nahum, Guy; and Bollinger, Steve "Touch Pad for Handheld Device", United States Patent 7,495,659, February 24, 2009 Detection of events (gestures?) on a touch-pad and filtering the native coordinates based on those events. Compare with Microtouch, and with Pencept and CIC recognition Macros? [McDermidWJ09a] McDermid, William J. "Touch Sensor Pad User Input Device", United States Patent Application 2009/0009482 A1, January 8, 2009 Touch typing on a multi-touch touchscreen keyboard: user first touches to establish home row position (e.g. eight fingers gesture), virtual on-screen keyboard displayed as visual feedback/confirmation at that position, user may then type at positions relative to established home row position: compare to Westerman? [Measurand08a] Measurand Inc. "Measurand Sampling shapes with ShapeTape: Use enough sensors to do the job!", www.measurand.com, 2008 Flexible tape sensor using optical fibers to detect bend and twist configuration of the tape. Discussion of limits to resolution and detecting due to spacing of points of sensing along the tape: bending or dent between two sensing points not detected. [MicroTouch08a] 3M Touch Systems (Microtouch) "MicroTouch Capacitive TouchSensor System", 3m Touch Systems, 2008 Capacitive touchscreen/digitizor with actuator for haptic feedback, can be mounted to frame/carrier/display for ClearTek II touch sensor touchscreen [MicroTouch08b] 3M Touch Systems (Microtouch) "MicroTouch TouchPen System for ClearTek II Touch Screens", 3m Touch Systems, 2008 Capacitive touchscreen/digitizor that senses either a stylus or a finger touch -- but not both, stylus/pen takes precedence -- stated as palm-rejection. multi-touch.



[MicroTouch09a] 3M Touch Systems (Microtouch) "MicroTouch Software MT7 Gesture Application Programming Interface API", 3m Touch Systems, 2009 Microtouch: Software toolkit MT7 for gesture recognition. Refers to "Slow Draw Anomalies" with jagged electronic ink (polling input of touch coordinates on tablet?) Stroke matches (elastic matching?), Strokes and Stroke data objects. Electronic file includes MicroTouch Capacitive TouchSense system with haptic transducer touch feedback for Gaming, 3M MicroTouch Software MT7 User Guide. [Microsoft08a] Microsoft Corporation "Integrating Application Gestures", msdn.microsoft.com, ms704819 Windows Vista / Tablet PC: defines gesture as "a glyph that defines the shape traced by the gesture", plus a hotpoint [Microsoft08b] Microsoft Corporation "Designing for Direct Manipulation", msdn.microsoft.com, ms698539 Tutorial for Tablet PC developers on basic physical differences between stylus/pen interaction with a digitizer, and operation with a mouse. Does not refer to details of mouse/stylus message difference in programming. Tablet resolution in Tablet PC is normalized in software to 0.01 mm. Size of targets, involuntary hand movements, hand/stylus obscuring view of display, hover/proximity feedback with digitizer tablet. [Microsoft08c] Microsoft Corporation "SystemGesture Enumeration", www.microsoft.com List of System Gestures for Microsoft Tablet PC/Vista. Lists Tap and DoubleTap as gestures that work for both system and application. Description for Tap refers to choosing a command from the menu or toolbar, take action if command is chosen. [Microsoft08d] Microsoft Corporation "Microsoft Tablet PC - Glossary", msdn.microsoft.com, ms840456 Definitions of Application Gesture, Gesture, Application Gesture, Glyph (as ink data). Refers to "Microsoft gesture recognizer" as engine that specifically recognizes ink iput as either system or application commands. "Object recognizer" for recognizing non-gesture, non-handwriting shapes such as musical notation, math notation (i.e. sketch recognition would fall into this). Stroke based on pen down/up. ISF Ink serialized format (similar to JOT?) [Microsoft08e] Microsoft Corporation "Application Gestures and Semantic Behavior", msdn.microsoft.com, MSDN, fetched February 21 2008 Vista/Tablet PC SDK documentation from MSDN: defined gestures include scratch-out/erase, Triangle/insert, Check-off, Curlicue/fixed (compare with proofreader's marks), Double-Curlicue/copy, double-circle/paste, left-semicircle/undo, right-semicircle/redo, Caret gesture/paste, Inverted-caret/insert, flick left / backspace, flick right / space (compare with GO flick gestures), left/right/up/down corners with directional gestures. See also 2003 version of same reference. [Microsoft08f] Microsoft Corporation "Announcing the Ultra-Mobile PC", msdn.microsoft.com General announcement of Ultra-Mobile PC (UMPC) running Windows XP Tablet PC Operating system. See also 2006 version. Ultra-Mobile PC Development Quick Start Guide. Keyboard optional, may include GPS, webcam, fingerprint reader, digital TV tuner. Microsoft Touch Pack for Ultra-Mobile PC UMPC. Brilliant Black skin with large buttons, DialKeys on-screen keyboard (for thumbs) from Fortune Fountain Ltd. UMPC display Emulator. [Microsoft10a] Microsoft Corporation "Touch Gestures", Windows Mobile 6.5, Touch Gestures, dated 4/8/2010 Tap gesture of finger represents left click of a mouse. Mouse events WM_MOUSEMOVE and WM_LBUTTONDOWN events not coordinated with GID_SELECT message for gesture. Double tap for left double click of mouse, using GID_DOUBLESELECT gesture message. Hold (press-and-hold/dwell) generates separate GID_HOLD message, used to represent right click of a mouse, GID_END messaged at end of hold time threshold. Flick gesture GID_SCROLL message to scroll, GID_PAN gesture to pan (can be after hold gesture) [Microsoft10b] Microsoft Corporation "Microsoft Project 2010 Product Guide: Project Standard and Project Professional", Microsoft Corp., 2010 Microsoft Project collaborative work system: includes CPM critical path method, Gantt charts, resource and schedule views. Agile project plans. [Microsoft10c] Microsoft "A history of Windows", windows.microsoft,com, archived 2010 Corporate history page on Microsoft Windows. Interface Manager code name, announced as operating system 1983, released as Windows 1.0 1985. Windows 1.0 in 256Kb Ram. Windows 2.0/2.11 1987 Control Panel. Windows 3.0 operating system. Windows NT new operating system 1993. 32-bit. Mentions nightly build/stress tests in development. Very brief mention of Tablet PC, none of Pen Windows: Windows Touch on Windows 7. [MilekicS09a] Milekic, Slavoljub "Using gaze interactions to interact with a display", United States Patent 7,561,143, July 14, 2009 Using a gaze tracker / visual input to simulate mouse events: no specific reference to gestures, but to graphical user interface [MilekicS09b] Milekic, Slavoljub "Action Affection and Control: Interface guidelines for complex visual content",



Museums and the Web, April 15..19, 2009, Indianapolis Review of design principles for intuitive user-friendly interfaces: primary mode of affective communication between humans is visual, not speech. [MilekicS10a] Milekic, Slavoljub "Curriculum Vitae: Slavko Melekic", www.uars.edu/faculty/smilekic/cv.html, Feb 27 2010 Educational uses of touch technology and human/computer design: Please Touch Museum (touch-screen user interface) at Phoenix Art Museum and Cleveland Museum of Art, Dallas Museum of Art: KiddyFace digital environment. [MistreeBFT10a] Mistree, Behram F.T. and Paradiso, Joseph A. "ChainMail: A Configurable Multimodal Lining to Enable Sensate Surfaces and Interactive Objects", Proc. ACM TEI 2010, January 2010, Cambridge Massachusetts Small (1" square) electronic boards with sensors, can be attached to cover any surface. Compare with electronic artificial skin, Stiehl "Huggable" (cited), touchscreen electronic skins (Rekimoto cited). Combines light, whisker touch, audio/sound, and pressure sensors: note that most pressure stimuli also produce an incidental audio sound. [MistryP08a] Mistry, Pranav and Maes, Pattie "Quickies: Intelligent Sticky Notes", Proc. 4th Int'l Conf. on Intelligent Environments, IE08, Seattle, 2008 Electronic ink application using small outboard (sonic?) digitizer with a PC, and paper with embedded RFID tag. Lets writing/ink be tied to the piece of paper, similar capability to Anoto technology, however RFID can be used to locate the paper (which may be tucked into a book or folder) using RFID technology, retrieve specific notes. Demonstration includes used of deferred recognition to index/search notes. [MistryP08b] Mistry, Pranav and Maes, Pattie "Intelligent Sticky Notes that can be Searched, Located and can Send Reminders and Messages", IUT08, January 13-16, 2008, Maspolomas, Gran Canaria, Spain Electronic ink application using small outboard (sonic?) digitizer with a PC, and paper with embedded RFID tag. Lets writing/ink be tied to the piece of paper, similar capability to Anoto technology, however RFID can be used to locate the paper (which may be tucked into a book or folder) using RFID technology, retrieve specific notes. Demonstration includes used of deferred recognition to index/search notes. [MistryP10a] Mistry, Pranav "SixthSense: Integrating Information with the Real World", www.pranavistry.com/projects/sixthsense, copyright 2010 Display image can be projected onto any surface, including a hand: tips of fingers (more than one) identified by colored bands on fingertips allow fingers to be detected optically with respect to image, forming an optical digitizer touch surface. [MiyakiT09a] Miyaki, Takashi and Rekimoto, Jun "GraspZoom: zooming and scrolling control model for singlehanded mobile interaction", Proc. MobileHCI09, September 1-18, 2009, Bonn Germany thumb-gesture input combining motion and pressure/force sensing: light touch is move, push-touch is zoom: refers to apple Pinch and Flick gestures for zooming and scrolling, comments that Flick may require multiple flick gestures for long list. Suggests using contact area of fingertip as substitute for pressure/force sensitive digitizer touchscreen. [MobilePoint08a] MobilePoint "MobilePoint: Advance the Sale", MobilePoint Corporation, 2008 Sales brochure for MobilePoint QuikQuote real-time quotation application running on Tablet PC: pencomputing-based sales/form application. See also PenPoint. [MoscovichT09a] Moscovich, Tomer "Contact Area Interaction with Sliding Widgets", Proc. UIST '09, October 4-7, 2009, Victoria, B.C., Canada Addresses targeting accuracy on touchscreen by selecting all widgets/objects under area of touch (since finger touch contacts an area, not a single pixel), and user moving to activate a particular function: Sliding widget, multi-function widget. Gives example of moving down a scroll list for "History", then flicking a stroke left or right to move forward in history. Click sound on activation (audible highlight) and also visual highlight. Compare with Apple slide-to-unlock? [MoskowitzSA08a] Moskowitz, Scott A. and Berry, Michael W. "Method and Device for Monitoring an Analyzing Signals", United States Patent 7,346,472, March 18, 2008 Digital watermarking, using abstract (watermark) of data object as ID of the object in a database. Abstract based specifically on perceptual qualities of the object. [MoskowitzSA10a] Moskowitz, Scott A. and Berry, Michael "Method and Device for Monitoring and Analyzing Signals", United States Patent 7,660,700, February 9, 2010 Cites to 7,346,472. Digital watermarking of audio/visual signals. Compare abstract/analysis of a given signal with stored abstracts of reference signals to find a match. [MotionComputing08a] Motion Computing, Inc. "Use Shorthand to Create a Gesture", http://www.motioncomputing.com/resources/Shorthand_Gesture.pdf Motioncomputing Tablet PC: Definition of single-letter "gestures" using side-button on stylus: appears functionally identical to "recognition macros" of Pencept PenPad 320 series product circa 1985, and similar feature from CIC HandWriter from about the same time. Press side barrel button on stylus, write one letter



or symbol, release side button. Pre-defined keyboard input is invoked. LE 1700. [MotionComputing08b] Motion Computing, Inc. "Motion moves in on other markets, unveils new product", Austin Business Journal, March 7, 2008 and other press reports Motion Computing F5 Tablet PC, XFR D630 : Ruggedized Tablet PC. Also competing ruggedized toughertablet market from Samsung, Sony, Dell Latitude XT convertible tablet. Mentions patent infringement lawsuit from Typhoon Touch. [MotionComputing08c] Motion Computing "Motion LE1700 Tablet PC User's Guide: Windows Vista", Motion Computing, 2007 Tablet PC User's guide: mostly hardware features including SIM/phone card slot. Press-and-hold / tap or touch and drag touchscreen, or use Function button and tap. [MotionComputing09a] Motion Computing, Inc. "J3400 Tablet PC Benutzerhandbuch (J3400 Tablet PC User Guide)", Motion Computing, 2009 User guide for Tablet PC running Windows 7 (not Windows Tablet PC/XP). For Arbeiten mit dem Stift (Stylus) has long-hold and barrel button for emulating right-mouse-click. Schreibblock (input panel) has both handwriting are input and simulated/virtual on-screen keyboard. Option to set barrel button to be erase (compare with Freestyle pen) instead of right-mouse-button click simulation. [Moto.com10a] labs.moto.com "Blog Archive: DIY Touchscreen Analysis: touchscreen stress test", http://labs.moto.com/diy-touchscreen-analysis Blog posting of testing of touchscreens and multi-touch surfaces: mentions erratic behavior at light touch, edge problems. Features bad behaviors/errors for which Moto touchscreens tend to do well. compare with other papers on digitizer effects and user Interface by Phillips and Ward. [Motorola10a] Motorola "Droid 2 Global User Guide", Motorola, 2010 User guide for Droid 2 Android touchscreen cell phone / smart phone with stylus and finger gestures: touch (tap), touch & hold, hold followed by drag, scrolling. Slide out physical keyboard, no handwriting recognition. [Motorola10b] Motorola "Webtop Application Overview", developer.motorola.com, 2010 Webtop: Desktop extension to Android PDAs and Smartphones: when using external keyboard and monitor (Lapdock station), displays Firefox 3 window executes Android applications within browser: use smartphone as portable PC with external display/keyboard. Filemanager for accessing internal files of smartphone. Electronic file contains additional materials. [MotusGames08] Motus Games "Motus Darwin product information", http://www.motusgames.com WII-like game controller using accelerometers and magnetic north, rather than infra-red optical digitizer for sensing hand gestures and position. Compare to accelerometer pen from Hew Crane at SRI. [Mozilla08] mozilla.org "FireGestures 1.1.2 von Gomita", http://addons.mozilla.org/de/firefox/addon/6366 Mouse gestures for Mozilla / Firefox browser, Thunderbird Mail, and Chatzilla. Gesture recognition apparently based on changes in direction. Certain gestures involve right mouse button, mouse wheel. [Mueller-TomfeldeC10a] Mueller-Tomfelde, Christian, (Ed.) "Tabletops - Horizontal Interactive Displays", Springer Verlag, ISBN 978-1-84996-112-7, 2010 Anthology of tabletop/touchscreen displays, technologies, applications. Sections Under Tabletops, On and Above Tabletops, Around and Beyond Tabletops (3D) [MuirNC10a] Muir, Nancy C. "Microsoft Project 2010 for Dummies", Wiley Publishing, Inc., 2010 Microsoft Project project planning tutorial [MultiTouchTechnology08a] multitouchtechnology.com "Multi-Touch Technology", www.multitouchtechnology.com CNN using multi-touch screens from 2006 TED conference. Apple MacBook with Multi-Touch touchpad using two-finger gestures similar to Apple iPod. [MultiTouchTechnology08b] multitouchtechnology.com "Multi-Touch Technology: Firefox 3.1 Adds Multi-Touch Support", www.multitouchtechnology.com Several announcements of multi-touch gestures in Apple computer products, touchpads, gestures supported in Firefox 3.1: pinch, two-finger scroll by page or window, swipe/flick, mentions fewer number of gestures in Safari web browser. [MultiTouchVista08a] codeplex.com "Multi-Touch Vista: TouchablePanel code example", multitouchvista.codeplex.com, fetched Feb 2011, dated 2008 Example WPF code for multi-touch demo on Windows Vista using Multitouch.Framework.WPF.Controls.Windows from schema.multitouch.com [MurphyC10a] Murphy, Chris "7 Tips for Using the iPad in Business", Information Week, November 1, 2010, page 26 Review of use of iPad touchscreen tablet at Mercedez-Benz Financial: customers sign agreements using iPad, states that development was less difficult than iPhone because display size allowed application to be built as more ordinary web/browser application. [MurphyG10a] Murphy, Glenn "The Chromium Projects: Tablet", dev.chromium.org, fetched 2010



Proposed tablet/gesture UI user interface for a Chrome OS tablet UI: mentions dwell/hover gesture for contextual actions [MurraySmithR08a] Murray-Smith, Roderick; Williamson, John; Huges, Stephen; and Quaade, Torben "Stane: Synthesized Surfaces for Tactile Input", Proc. CHI 2008 - Multitouch and Surface Computing, April 2009, Florence Italy pp 1299-1302 Haptic/Tactile input device: Hand-held object with various ribbed textures on the surface and piezo microphone inside: rubbing or stroking a finger across the surface causes vibrations characteristic of the ribbed texture. Control of zoom, volume, position by stroking different seconds. Additional materials in file: "M" gesture, double-tap with finger gesture to turn UI on/off (for carrying pen computer so that cursor does not move) [MyllymakiS10a] Myllymaki, S.; Huttunen, A.; Palakuru, V.; and Jantunen, H. "Capacitive Recognition of the Uer's Hand Grip Position in Mobile Handsets", Progress in Electromagnetics Research B, Vol 22, pp. 203..220, 2010 Capacitive method for measuring/detecting the position of the user's hand holding a mobile phone. Combines two sensors: one using the antenna, the other a dual-electrode (transcapacitance). The more of the user's hand is around the cel phone, the more the capacitance change. Purpose is to transmit stronger signal when user is covering most of the antenna, but otherwise spare the battery. [NUIGroup09a] NUI Group Authors "Multi-Touch Technologies: Community Book, 1st ed", available at nuigroup.com, May 2009 Textbook/tutorial from open-source hardware project for multi-touch (optical: rear diffusion) hardware. Discusses open-source Gesture widget and software libraries, such as X12, GDML Gesture Definition Markup Language, NUIFrame, Graffiti (not the Palm unistroke library), AME Arts, Media and Engineering Patterns Library, other gesture frameworks. [N-Trig09a] N-Trig Corporation "DuoSense Technology", www.n-trig.com July 2009 N-trig digitizer tablet, senses stylus and touch/multi-touch, electrostatic (capacitive) stylus, mentions unintentional touch / hand-rejection Additional materials in file: "M" gesture, double-tap with finger gesture to turn UI on/off (for carrying pen computer so that cursor does not move) [NaoneE10a] Naone, Erica "New Computer Interface goes beyond Just Touch", Technology Review, April 12, 2010 Multi-modal (haptic?) interface by Ken Hinckley at Microsoft, using both pen and hand. One example is using hand to hold a virtual object stationary on an active surface, and a pen/stylus to tear of a virtual page. [NaoneE10b] Naone, Erica "Redesigning the Web for Touch Screens", Technology Review, May 04, 2010 Re-designing of web pages for touch input, specifically for the iPad. Cites to Internation children's Digital Library re-design, multi-lingual (pictographic/iconic?) user interfaces was particularly amenable to touch input. [NelsonF10a] Nelson, Fritz "Review: Dan Bricklin's Note Taker HD puts the "Pad" in iPad", Information Week, July 6, 2010 Notetaker application for apple iPad. Appears similar to DayTimer product by Slate in the early 1990s for PenPoint and for Pen Windows: follows paradigm of ink-as-datatype, editing of electronic ink, as in van Raamsdonk [NelsonF10b] Nelson, Fritz "iPhone 4 Teardown Telegraphs Future iPad Gyro", Information Week, June 24, 2010 iPhone 4 cell phone / tablet contains gyroscope in addition to accelerometer, identifies as InvenSens ITG3200 or STMicro L3G4200D three-axis MEMS gyro. [NetworkSignatures10a] NetworkSignatures.com "Intermediate Network Authentication", www.networksignatures.com Materials relating to two-factor authentication products (exchange of PK signatures) for iPhones, Javaenabled phones and Blackberry mobile devices, based on patent 5,511,122, licensed from Naval Research Laboratory. [NetworkWorld09a] NetworkWorld.com "Researchers develop 3-D squeezable input device", www.networkworld.com, Dec. 01, 2009 Suma, by Cambridge Consultants, squeezable mouse-like input device for three-dimensional input. Article states light-sensitive sensors for pressure/touch, ability to control two points of view simultaneously with one hand. [NetworkWorld09b] NetworkWorld.com "Nanotouch technology shrinks touchscreen displays", www.networkworld.com, April 7, 2009 Pseudo-transparency: on small screen, touch screen is on rear of display so that finger does not occlude view of display -- show simulated finger on display when user touches back of display for input. Earlier technology was called Lucid Touch. [NeuroScript09] NeuroScript "Testing Digitizer / Tablet Linearity", http://www.neuroscript.net software package for measuring performance of tablet digitizers: manual procedures for testing linearly by using diagonal lines, but does not mention velocity-induced errors (compare with Carau). Also Testing



digitizer/Mouse: resolution, etc. [NeuroScript10] NeuroScript "MoveAlyzeR Tutorial", http://www.neuroscript.net software package for measuring performance of tablet digitizers: missing coordinates, missing points, nonlinearity. Part of offering to for measurements for neuromotor analysis of handwriting motion. [NewScientist08] NewScientist.com news service "Phones with feeling are more useful", www.newscientist.com, 24 March 2008 VibeTonz vibrotactile haptic/vibration feedback for touch interfaces and touchscreens: VibeTonz is a hardware actuator component for PDAs and devices similar to Apple iPhone [NewScientist08b] NewScientist.com news service "Fat fingers no problem with 'see-through' touchscreen", www.newscientist.com, 18 December 2008 Fat fingers / touchscreen accuracy: Place touch pad on back side of hand-held touch-screen device, so that finger tip does not obstruct/obscure view of display and where user is touching [NewScientist09a] NewScientist.com news service "'Fat fingers' can become dainty for touch screens", www.newscientist.com, 24 November 2009 Improve accuracy of touch-screen input by sensing orientation of finger: experimental system to demonstrate used fingerprint scanner to determine orientation of finger [NewScientist10a] NewScientist.com news service "Putting the touch into touchscreens", www.newscientist.com, 26 April 2010 Review of haptics technologies for touchscreens: vibration to give the illusion that a button protrudes from the service, lateral vibration to simulate the haptic sensation of a sharp edge, high frequency vibrations to simulate a slippery surface, force-feedback systems [NewScientist10b] NewScientist.com news service "World's first ice touchscreen virtually burns", www.newscientist.com, November 18 2010 Nokia demonstration system of turning blocks of ice into translucent optical touch screen (based on reflected light where hands come into contact), displaying virtual flames where touched. [NewScientist10c] NewScientist.com news service "Extreme PowerPoint places you in 3D light show", www.newscientist.com, October 4, 2010 Microsoft LightSpace prototype: 3D optical/Kinect sensing of hand gestures including picking up objects for interactive slide show presentations on multiple was. Mentions human factors concerns (presentation technology is distracting to audience). Mentions multi-touch. [NextWindow09] NextWindow Inc. "NetWindow Optical Touch Screens", www.nextwindow.com, July 2009 Infrared optical digitizer multi-touch flat-panel displays: notes that no optical coating is required on display. Sensors mount at corners of front surface of rectangular glass assembly. [NguyenM10a] Nguyen, Mai; Wang, Xin; Ta, Thanh; Lao, Guillermo; and Chen, Eddie J. "System and Method for Managing Transfer of Rights Using Shared State Variables", United States Patent 7,774,280, August 10, 2010 DRM usage rights management, rights transfered from a rights supplier to a rights consumer, with some rights derived from a meta-right / meta-right is the right to receive certain rights [Nokia08a] Nokia "Nokia N810 Internet Tablet", www.nokia.com product information Dedicated pen computer with MP3 and auto map software, open-source (Linux). Touchscreen digitizer. [Nokia08b] Nokia "First glimpse inside Nokia S60 Touch. Going beyond Multi-touch", www.unwiredview.com, January 8, 2008 Review of S60 No-touch / touchless electrostatic touch UI device from Nokia. Allows in-air threedimensional 3D gestures. Compared to "Minority Report" film. Due to patent conflicts with Apple MultiTouch (see Fingerworks), based on Active Matrix LCD with integrated optical touch screen. Gestures include clockwise and counter-clockwise circles, X mark for delete, check mark, lasso / enclosed curve, two finger pinch and expand for zoom. [Nokia08c] Nokia "Nokia creates a real Minority Report", phonenews.blogs.sapo.pt, January 9, 2008 Review of Nokia publication on in-air gestures, compared with film "Minority Report": clockwise and counter-clockwise circular rotations, pinch and zoom, stationary pointing, X, check mark, enclosed curve / lasso to select. See also Nokia S60 [Nokia09a] Nokia "Nokia N900 touchscreen smartphone", Nokia (physical device) Nokia touchscreen smartphone, GPS, Symbian operating system [Nokia10c] Nokia "Nokia N8-00 User Guide", Nokia.com, fetched Dec 2010 Nokia N8 touchscreen cellphone with network applications. Physical slide switch to unlock. Includes haptic (vibration) feedback as confirmation on some touch inputs. [Nokia10d] Nokia "Nokia, Microsoft", (various) Folder with information on Nokia and Microsoft Windows 7 phone OS [Nokia10e] Miller, Matthew "Smartphones and Cell Phones -- The Nokia NS: uncompromising specifications and functions in the smartphone world", CBS web site, December 14, 2010 Nokia Smartphone, USB peripheral port (not just data port), FM radio, video, stereo microphones



[Nokia10f] Nokia "Nokia 5230 touchscreen smartphone", Nokia (physical device) Nokia touchscreen smartphone, GPS, Symbian operating system [Nokia10g] Nokia "Nokia N8-00 touchscreen smartphone", Nokia (physical device) Nokia touchscreen smartphone, GPS, Symbian operating system [Nortd.com09a] nortd.com "TouchKit: open source multitouch development kit", labs.nortd.com, fetched 2009 Open source development system (hardware and software) for multi-touch technology, includes projection of images on reflective surface. Also mentions CUBIT multitouch system for rapid implementation of multitouch projects: ofxTouch API for C++ OpenFrameworks on various platforms. File contains additional materials: Zaagtech multi-touch kit [NovaMobility08] Nova Mobility Systems "Nova Mobility Rugged Tablet PC and Handheld Computer Systems", www.novamobility.com NMS-5000, NMS-SUNPAD, Tablet PC, Windows Mobile, Windows CD, Windows XP Embedded. touch-screen based pen computers and PDA's. Licensee of Typhoon Technologies. Market is industrial uses, data collection. Integrated bar code, camera, mag stripe reader. Touch screen with palm rejection -- digitizer technology not stated, says resistive touch. Keyboard usable while user wearing heavy gloves. Compare with products by Leo Shpiz for military market. [ONeilSoftware10a] O'Neil Software, Inc. ">Barcodes, A Timeline / The barcode time line", O'Neil Software Inc, 2010 History of barcode patterns: circular bar-code invented 1948 Silver and Woodland. Barcode to track physical items / invoices 1981. [OSNews10a] Holwerda, Thom "Apple Slapped with Multitouch Patent Lawsuit", OSNews.com, March 31, 2010 Patent 5,825,352 (see) asserted by Elan Microelectronics, 1996 concerning multitouch, including determination of local maxima for fingers. Compare with Rekimoto? [ObradovichML08a] Obradovich, Michael L. "GPS Publication Application Server", United States Patent 7,343,165, March 11, 2008 Server receives GPS and other data from a personal device, displays location on a web homepage accessible to selected users. [ObradovichML09a] Obradovich, Michael L.; Pirtle, John D.; Schebesch, Steven W. "Technique for Effective Navigation based on User Preferences", United States Patent 7,522,992, April 21, 2009 Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, based on preferences in a user profile. [ObradovichML09b] Obradovich, Michael L.; Pirtle, John D.; Schebesch, Steven W. "Technique for Effective Navigation based on User Preferences", United States Patent 7,593,812, September 22, 2009 Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, receives a user touch input, on a point of interest, establishes communications to it. [ObradovichML10a] Obradovich, Michael L.; Pirtle, John D.; Schebesch, Steven W. "Technique for Effective Navigation based on User Preferences", United States Patent 7,650,234 January 19, 2010 Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, computes a route based on traffic and/or weather. [ObradovichML10b] Obradovich, Michael L.; Pirtle, John D.; Schebesch, Steven W. "Personal Communication System to Send and Receive Voice Data Positioning Information", United States Patent 7,702,455, April 20, 2010 for a hand-held device with a touchscreen, server requests traffic data, and computes a route to avoid traffic problems. [ObradovichML10c] Obradovich, Michael L.; Pirtle, John D.; Schebesch, Steven W. "Technique for Effective Navigation based on User Preferences", United States Patent 7,739,039, June 15, 2010 Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back route with areas to avoid and points of interest. [OdomG08a] Odom, Gary "Tool Group Manipulations", United States Patent 7,363,592 B1, April 22, 2008 Tool bar re-configures automatically, changing/re-arranging toolbar menu items depending on what tool user has selected and is using most frequently: file contains press reports on patent assertion against Microsoft. See also www.patenthawk.com (filed under Odom): patenthawk.com [OdomG08b] Odom, Gary "User Selection of Computer Login", United States Patent 7,350,079, March 25, 2008 computer login using user's choice of multiple input devices: mouse, keyboard etc. and multiple types of input: e.g. actual password, keystroke timing, etc. [OdomG09a] Odom, Gary "About Patent Hawk", www.patenhawk.com, fetched 2009 Patent search firm (Gary Odom), see notes on Microsoft, Gary Odom correspondence [OdomG09b] Odom, Gary "Patent Hawk press file", News reports, mostly 2009 Reports and comentary Gary Odom / Patent Hawk patent infringement lawsuit vs. Microsoft, Patent Hawk firm had prior working relationship with Microsoft. "Did Patent Hawk Violate Contracts In Suing Microsoft" [OdomG10a] Odom, Gary "User-Selectable Signatures", United States Patent 7,725,725, May 25, 2010



computer login using user's choice of multiple input devices: mouse, keyboard etc. and multiple types of input: e.g. actual password, keystroke timing, etc. [OrdingB08a] Ording, Bas "List Scrolling and Document Translation, Scaling, and Rotation on a touch-screen display", United States Patent 7,469,381, December 23, 2008 User-interface on touchscreen (or near-touch) for document dragging: when part of document is dragged, expose another part of the document: also describes "bounce" at end of scroll: Compare with Alternate Reality Kit? Twist gesture to rotate an object or screen. Re-exam document also in file. Bounce-list at end of scrolling on Apple iPod/iPad [OrdingB10a] Ording, Bas; Forstall, Scott; Christie, Greg; Lemay, Stephen O. and Chaudhri, Imran "Continuous Scrolling List with Acceleration", United States Patent 7,786,975, August 31, 2010 Scrolling/flick gesture: faster gesture results in faster scrolling: flick-to-scroll. [OrdingB10b] Ording, Bas; Forstall, Scott; Christie, Greg; Lemay, Stephen O. and Chaudhri, Imran "Portable Electronic Devices with Multi-Touch Input", United States Patent 7,812,826, October 12, 2010 multi-touch user interfaces: multi-touch gestures used to adjust parameters e.g. rotate a virtual knob. [OrdingB10c] Ording, Bas; Forstall, Scott; Christie, Greg "Methods for Determining a Cursor Position from a Finger Contact with a Touch Screen Display", United States Patent 7,843,427, November 30, 2010 On touch-screen user interface, target a touch to the nearest valid UI object based on position of centroid of touch: example is small keyboard. Compare with Ink targeting, gravity? Compare with Penpoint and W4PC targeting on ink. [Orgacon09a] Agfa/Orgacon "Indium Tin Oxide and Transparent Conductor Products", Orgacon.com, 2009 ITO and organic transparent conductive inks for screenprinting: intended for touchscreen electrode market [OuyangTY09a] Ouyang, Tom Y. and Davis, Randall "A Visual Approach to Sketched Symbol Recognition", Proc 2009 IJCAI, pp 1463-1468, Pasadena CA On-line/dynamic character recognition, but using visual features (OCR-like?) rather than dynamic or timesequence recognition. Mentions problems of overlapping/touching sketch symbols and gestures. States that recognizer is rotationally invariant (so what about M and N, E and 3, S and Z, etc.?) Does not describe recognizing handwriting: electrical symbols, powerpoint symbols, etc. File contains additional references for sketch recognition for mechanical-engineering, to 2003 [OuyangTY09b] Ouyang, Tom Y. and Davis, Randall "Visual Recognition of Sketched Symbols", IUI 2009 Workshop on Sketch Recognition, Feb. 8 2009, Sanibel Island, FL OCR-type recognition, mentions down-sampling of images prior to classifier. See other Ouyang papers. States superior to Graffiti, which requires constrained writing. (Unistroke) [OuyangTY09c] Ouyang, Tom Y. and Davis, Randall "Learning from Neighboring Strokes: Combining Appearance and Context for Multi-Domain Sketch Recognition", Advances in Neural Information Processing, 2009, pp. 14011405 Recognition of clusters of squiggles and angles in a sketch as an indication that the strokes should be parsed together in one object [OuyangTY10a] Hardesty, Larry "Sketch -interpreting software", M.I.T. news, Feburary 19, 2010 Trainable sketch-recognition system, using both dynamic (time sequence and stroke direction) information and static visual appearance (but not from scanner). Example is molecular-sketch and simple circuitdiagram systems. But states that input to existing system only done for molecular sketch -- no actual recognition, only prettifying? [OxfordEnglishDictionary10a] Oxford English Dictionary "Oxford English Dictionary", www.oed.com, fetched 2010 Definitions of select terms. gesture (with motion), pointer (computer address; screen symbol), stylus (pointer or finger) [Palm08a] Unknown "Die Palm Pilot History", Posted on scribd.com and handheld.vn, 2008 Compendium of over 100 handheld devices up to 2008 running Palm OS 5.0 on PalmPilot type models, with hardware specifications [Palm09a] Palm "Palm Pre Benutzerhandbuch", Palm Inc, 2009 User guide (in German) for Palm PRE hand-held touchscreen PDA [ParkesA08] Parkes, Amanda; Poupyrev, Ivan; and Isshi, Hiroshi "Designing Kinetic Interactions for Organic User Interfaces", CACM, June 2008, Vol 51 No 6; pp 58-ff. Organic user interfaces / tangible user interfaces: combination of physical input and haptic/physical feedback, not specific to gesture input. Reference to three-dimensional mechanical digitizers. [PatelN08] Patel, Nilay "MIT reinvents the Post-It note ... with Post-It notes", www.endgadget.com, May 1, 2008 press reports on Quickie handwriting notes using what appears to be the Anoto pen, with an RFID tag (?) embedded in the paper. Also Girton Labs Ixp-Note, post-it note sized small digitizer, form factor like a postit note. Prototype of DPF711K, touch-screen small size with a post-it note sized display from Audiovox, along with Audiovox Digital Message Center. Sony E- Series display/touchscreen digitizer for taking notes. [PatelPredd09] Patel-Predd, Rachel "The Trouble With Touch Screens", IEEE Spectrum, January 2009, pp 11-12



touchscreens and digitizers using indium tin oxide ITO: indium is scarce resource, alternatives are antimony tin oxide, carbon nanotube, flexible polymer PEDOT [PeltonenP08a] Peltonen, Peter; Kurvinen, Esko; Salovaara, Antti; Jacucci, Giulio; Ilmonen, Tommi; Evans, John; Oulasvirta, Antti; and Saarikko, Petri ""It's Mine, Don't Touch!": Interactions at a Large Multi-Touch Display in a City Centre", Proc. CHI 2008, April 5-10, 2008, Florence, Italy, pp. 1285-1294 CityWall public multi-touch display in Helsinki: social behaviors among users coordinate use. [Peregrine09a] ThePeregrine.com "Peregrine wires you into a keyboard: we go hands-in", ThePeregrine.com, fetched 2010 Electronic gesture data glove, approximately 30 gestures. "Hot-key touch points": Linear (1D position) touch sensors in glove fingers, palm, thumb for particular hand gestures with fingers, not 3D proximity sensing. Combination of touches mapped to keyboard entry for faster gaming. [Peregrine10a] ThePeregrine.com "The Peregrine Gesture Glove / gaming data glove -- Not your daddy's power glove", ThePeregrine.com, fetched 2010 Electronic gesture data glove (touch gestures of hand only), approximately 30 gestures, indented for gaming market. File contains additional materials [Perttula08] Pertulla, Drew and McClosky, David "skim text file reader", bigasterisk.com/projects/skim Text file document read with scrollbar highlighting parts of text in the scrollbar [PhuongHM10a] Phuong, Hoang Minh; Dzung, Nguyen Tien; and Hoang, Thang Manh "Implementation of Webcam-based Touchscreen", Proc. 3rd Intl. Conf. on Comunications and Electronics ICCE Aug 11-13, 2010, pp. 201-206 compare with Smart Desk / Weller: determine "touchscreen" position using webcam and edge detection to determine location of fingertip. [Physorg08] PhysOrg.com "Smart desks make sci-fi a reality in the classroom", www.physorg.com Durham University in UK, Dr. Liz Burd, Director of Active Learning in Computing: experimental program using multi-touch desk surfaces in education: SynergyNet project of the Technology-enhanced Learning Research Group [Piquepaille08] Piquepaille, Roland "Multi-touch smart desks in the classroom", zdnet.com, Sept. 17, 2008 Durham University UK experimental SynergyNet project using multi-touch tablet desks for gradeschool education [PlasticLogic09a] PlasticLogic.com "Plastic Logic Reader", Plastic Logic.com, 2009 ff File on Plastic Logic Reader, electronic ink display with touch-sensitive touchscreen interface: swipe gesture to turn pages. [PlatzerA10a] Platzer, Andrew and Herz, Scott "Application Programming Interface for Scrolling Operations", United States Patent 7,844,915, November 30, 2010 API to set parameters for scroll bounce [Plawa10a] Plava Feinwerktechnik GmbH "Scrivo.1 product information", Plawa.com, fetched 2008, 2010 Optical mouse in a pen-shaped barrel. Newer model includes audio recording: compare with LiveScribe and FingerSystem. Finger Note software: ink (?) annotation of screen captures for note taking -- Compare with Wang Freestyle. User manuals in English and German. [PluchinoA09a] Pluchino, A; Rapisarda, A.; Garofalo, C; "The Peter Principle revisited: a computational study", Presentation: Econophysics Colloquium 2009, Centro Ettore Maiorana Erice, Italy, Oct. 25-31, 2009 Game-theory Mathematical model of overcoming Peter Principle (individuals are promoted until they reach a position they are not competent at, and then stay there: thus organizations tend to have managers who are not competent at their job) by promoting people at random. Cites also to 2010 paper with same title, Physica A 389 (2010) 467 [PocketCalculatorShow10a] PocketCalculatorShow "Seiki Computer Watch Fun", PocketCalculatorShow.com, fetched 2010 Variety of Seiko computer/calculator watches. RC-20, circa 1985, buttonless touchscreen input with 42x32 pixel display, graphical icons [PolssonK09] Polsson, Ken "Chronology of Personal Computers", Posting at www.islandnet.com Timelines of history of personal computers, back to 1947. Windows for Pen Computing and GO/PenPoint both announced January 1991. [PositiveSystems10a] Positive Systems Inc. "POSey POS Point Of Sale product information", www.pos-ey.com fetched 2010 POS system for DOS/Windows95 and later using modified MicroTouch unMouse touch screen to replace mouse. File contains additional information. [PoupyrevI08a] Poupyrev, Ivan "What is Gummi? Gummi: A bendable computer (video presentation)", http://www.sonycsl.co.jp/person/poup/projects/gummi.html fetched 2008 Bendable computer: transparent flexible 2D touchscreen sensors combined with flexible organic display. Bending use for up/down zoom on map application, etc.



[Poyner10a] Poyner, Rick "Wintab Backgrounder", Wacom Engineering www.wacomeng.com, fetched 2011 Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages. Also exists in version from 1993 (original), later Wacom version (after LCS/Shumer lawsuit) with minor updates [PrescherD10a] Prescher, Denise; Weber, Gerhard; and Spindler, Martin "A Tactile Windowing System for Blind Users", Proc. ASSETS '10, ACM, October 25-27, 2010, Orlando FL, pp. 91-98 BWS Braille Window System using BrailleDis 9000 120x60 pin-matrix display. Display divided into Braille Window portions: Window title region, view type region, tactile scrollbar, Braille text output, command output, etc. Shift/function key to enable multitouch gestures on touch-sensitive display, so as not to confuse with touch reading of display. Circular zoom gestures, pinch/spread used for dividing regions. Minimap "world view". Cites to problems of screen readers only reading top-level Window, confusions of pop-up windows. [PrimeSense10a] PrimseSense Ltd. "PrimeSense Natural iteration", www.primesense.com, 2010 WII-like 3D gesture sensor: combines structured (patterened) IR light source with IR video sensors for depth sensing, 4 external audio sources with 2D (stereo) microphone sensors, color video. [ProbertML10a] Probert, Matt and Probert, Leela "Research Results for 'SideKick'", www.probertencyclopaedia.com, fetched 2010 Borland SideKick phone dialer dials a phone number found anywhere on the DOS screen: it didn't need to be in SideKick mode [Qomo09] Qomo HiteVision LLC "Qomo HiteVision QIT 30 product literature", Qomo HiteVision LLC, www.qomo.com Untethered electromagnetic digitizer, multiple tablets may be used wirelessly to annotate on a common shared whiteboard display [QuinnM08] Quinn, Michelle "Women put a finger on an iPhone problems", Los Angeles Times, June 16, 2008 Apple iPhone touch-sensitive digitizer does not work for women with long fingernails, or with gloves. Described tongue-in-check as accessibility problem. [Radivojevic08] Radivojevic, Zoran; Zou, Yanming; Wang, Kong Qiao; Takala, Roope Tapio; Lantz, Vuokko Tuulikki; Lehtiniemi, Reijo; Rantala, Jukka Ilmari; and Vatanparast, Ramin "Apparatus, Methods and Computer Program Products Providing Finger-Based and Hand-Based Gesture Commands for Portable Electronic Device Applications", United States Patent Application US 2008/0005703 A1, January 3, 2008 Nokia: In-air finger and hand gestures in front of a PDA or cell phone: describes a sonic/sonar digitizer for detecting hand gestures close to the device, but also mentions electrostatic/capacitive and optical hardware, generally relates to use of in-air hand and finger gestures, not the hardware. Mentioned in other references as multi-touch. [RantalaJ09a] Rantala, Jussi; Raisamo, Roope; Lylykangas, Jani; Surakka, Veikko; Raisamo, Jukka; Salminen, Katri; Pakkanen, Toni; and Hippula, Arto "Methods for Presenting Braille Characters on a Mobile Device with a Touchscreen and Tactile Feedback", IEEE Trans. on Haptics, Vol 2 No 1, Jan 2009, pp 28-39 Additional materials in file. Vibrating touch screen puts Braille at the fingertips: rhythmic haptic encoding of braille characters similar to Morse code; haptic/tactile display of braille on a touchscreen/multi-touch surface. Spatial encoding of simulated braille as stylus moved on touchscreen. Rhythm method worked better. Prototype implemented on Nokia 770 Internet Tablet with added piezoelectric actuator. [RapidRepair10a] RapidRepair.com "iPad Repair Guide: Stay tuned for a shocking tear down of the new Apple iPad", www.rapidrepair.com, fetched 2010 Disassembly of Apple iPad: shows separate front-mounted digitizer with transparent screen. [RayJ10a] Ray, Jim "SAGE: Cold-War Forerunner to the Information Age", Available on eskimo.com SAGE system: information on light gun [RekimotoJ08a] Rekimoto, Jun "Organic Interaction Technologies: From Stone to Skin", CACM, June 2008, Vol 51 No 6, pp. 38-44 Special issue on organic/graspable/haptic/tangible user interfaces. Deformable/rollable input devices, as SmartSkin, PreSense (pressure-sensing touchscreen surface -- see elastomeric touchscreens?). Examples include HoloWall -- compare with Wellner? [RekimotoJ08b] Rekimoto, Jun "BrightShadow: Shadow Sensing with Synchronous Illuminations for Robust Gesture Recognition", Proc. CHI 2008, Works in Progress, april 5-10, 2008, Florence Italy, pp. 2769-2744 Optical 3D finger tracking system (multi-touch), uses shadows from synchronized multiple light sources (overhead above tablet) at known locations, shadows on known flat surface. High-speed video camera 200 fps. Away from tablet: subtract known background from image, look for bright spots of fingertips. Compare with SmartDesk.



[RekimotoJ09a] Rekimoto, Jun "SensableRays: Opto-Haptic Substitution for Touch-Enhanced Interactive Spaces", Proc. CHI '09, April 4-9, 2009, Boston, MA, pp. 2519-2528 "passive" 3D/2D tactile feedback, without direct position sensing. Instead, just an optical receiver on a finger-mounted sensor/vibrator that picks up time-modulated light (or infrared) projected by rays on a location on a surface, and vibrates in response to the light beam projected to that particular spot. 3D by using two light sources. [ReynaA09] Reyna, April S. "Multi-touch Technology", www.wiziq.com, June 6 2009 Internet posting of review of multi-touch technology: Perceptive Pixel, Apple, Microsoft Surface (infrared), Apple iPhone [RichterH10a] Richter, Hendrik; Ecker, Ronald; Deisler, Christopher; and Butz, Andreas "HapTouch and the 2+1 State Model: Potentials of Haptic Feedback on Touch Based In-Vehicle Information Systems", Proc. 2nd Int'l Conf. on Automotive User Interfaces and Interactive Vehicular Application, AutomotiveUI 2010, Nove 11-12 2010, Pittsburgh, PA. pp 72-79 Tactile/Haptic feedback on a touchscreen in an Automotive map user-interface: also tracks finger in proximity, or when not touching firmly. Application of Buxton 3-state model: Out-Of-Range on proximity, active tracking (a.k.a. passive tracking, and pressed. [RichterPJ08a] Richter, Paul J.; Cairnes, Darran R.; and Bottari, Frank J. "Antiglare Coating and Particles", United States Patent Application 2008/0014340 A1, January 17, 2008 Antiglare coating for touchscreens: silica particles in inorganic polymer matrix. [RicksTK10a] Ricks, Theodore K; Burberry, Mitchell S.; and Johnson, David M. "Touch Input Device with Display Front", United States Patent Application 2010/0020039 A1, January 28, 2010 Using flexible display (polymer-dispersed imaging layer), put display in front of pressure-sensitive touchscreen (spacing dots, resistive), so that touchscreen does not reduce visibility of display. Says resistive-film touchscreens with spacer dots have 75% transparency, ITO capacitive typically 90%. Touchscreen may be opaque. [RobbinsDC08a] Robbins, Daniel C; Sarin, Raman K;, Horviz, Eric J.; and Cutrell, Edward B. "Advanced Navigation Techniques for Portable Devices", United States Patent 7,327,349, February 5, 2008 Browsing or scrolling a document (such as a map) using a virtual semi-transparent overlay over the image of the document: like moving a magnifying lens? [RohrabaughGB08a] Rohrabaugh, Gary B. and Sherman, Scott A. "Scalable Display of Internet Content on Mobile Devices", United States Patent 7,461,353, December 2, 2008 Convert HTML layout to resolution-independent vector display-list form, for better rendering on small (handheld/smartphone) displays. [RosenbergI09a] Rosenberg, Ilya; Grau, Alexander; Hendee, Charles; Awad, Nadim; and Perlin, Ken "IMPAD - An Inexpensive Multi-Touch Pressure Acquisition Device", Proc. CHI 2009, April 4-9, 2009, Boston, Mass. pp. 32173222 Force/pressure sensitive touchpad/tablet using elastomeric material (conductive rubber: force decreases resistance) between perpendicular matrix layers of conductor lines. Effect is grid of force/pressure FSR force-sensitive-resister points whose sensing areas overlap and fall off with distance. Interpolation to get higher resolution of single points, or detect multi-touch points. [RosenbergI09b] Rosenberg, Ilya; Perlin, Ken; Hendee, Charles; Grau, Alex; and Awad, Nadim "The UnMousePad - The Future of Touch Sensing", Proc. SigGraph '09, August 3..7, 2009, New Orleans Multi-touch touchpad using force variable resistors with interpolation (IFSR) to get position resolution finer than minimum discrimination distance for multiple touches (like human skin). Described as touch imaging technology. [RosenbergI09c] Rosenberg, Ilya and Perlin, Ken "The UnMousePad - An Interpolating Multi-Touch Force-Sensing Input Pad", ACM Trans. on Graphics, vol 26 no 3, August 2009, pp 65:1..65:9 Multi-touch touchpad using force variable resistors with interpolation (IFSR) to get position resolution finer than minimum discrimination distance for multiple touches (like human skin). Described as touch imaging technology. Distinguishes true pressure (force) sensing from contact area of iPhone touchscreen. Pressure (force) readily used to adjust multiple slider values, translate/rotate/scale in 3D three dimensions. Automatic/adaptive trade-off of resolution vs. scan rate by using every other grid line: sleep mode at very low power consumption. Two layer grid with FSR force-sensing resistant paint layer between: correction matrix for point slightly off exact crossing point of layers. [RosenbergLB09a] Rosenberg, Louis B. and Riegel, James R. "Haptic Feedback For Touchpads and Other Touch Controls", United States Patent 7,592,999, September 22, 2009 (See also 8,031,181, 8,059,105) Haptic force feedback transducer/actuator mounted on back surface of touch surface or tablet, which is mounted on springs to allow motion. [RothV10a] Roth, Volker; Schmidt, Philipp; and Gueldenring, Benjamin "The IR Ring: Authenticating Users' Touches on a Multi-Touch Display", Proc. UIST '10, Oct. 3..6, 2010, New York



Infrared optical multi-touch touchtable using infra-red IR LEDs, low resolution camera (base on Community Core Vision Framework). Infrared-emitter signet ring generates detectable pseudo-random signal, can be use to identify user's touchs, nearby touches assume to be from that user's hand (multi-user). User identification described as security feature on multi-user touch tables. Describes fingerprint authentication as easily copied, especially without "livelihood" detection. [RoudautA09a] Roudaut, Anne; Lecolinet, Eric; and Guiard, Yves "MicroRolls: Expanding Touch-Screen Input Vocabulary by Distinguishing Rolls vs. Slides of the Thumb", Proc. CHI 2009, April 4-9 2009, Boston, Massachusetts, pp. 927-936 Gesture based on rolling of thumb instead of sliding of thumb. Six microroll gestures (four directions, two circular) four slide gestures (direction) four swipe gestures (direction) two diagonal rubbing (rubout) gestures. Instead of marking menus, etc. [RoudautA10a] Roudaut, A.; Baudisch, P.; Holz, C.; and Becker T. "The 15 Euro designer multitouch pad:", Hasso Plattner Institut: Seminar at ITS '10, materials available at www.anneroudaut.fr/fun/acrylicpad.html How to construct a multitouch FTIR optical digitizer using easily-available parts at home in your spare time. Acrylic sheets with webcam camera underneath. Recommends spraying surface with silicone lubricant to get better detection (better optical contact and reflection). Includes video showing both operation and construction. [RutschmannD09a] Rutschmann, Dirk and Josten, Marcus "Method and System for Detecting the ThreeDimensional Shape of an Object", United States Patent 7,489,813, February 10, 2009 Single camera, determine shape of an an object form multiple views. Object has photogrammetrically evaluatable marks (targets). Uses marks on floor (surface) at known locations, object at known position, so that no scale reference object needed. Example is special pants/trousers/clothing with marks on human body. Compare to Chen 2009 and structure-through-motion. [SafferD08] Saffer, Dan "Interactive Gestures: Designing Gestural Interfaces", O'Reily Press, 2008 General tutorial book on gestural interfaces including motion gestures such as Wii, hand-gestures, eyewinks, body motions, multi-touch from Fingerworks and iPhone. Mentions "the Clapper" as a gestural interface with indirect manipulation. First multi-touch device in 1982. Refers to gestures as motions of the human body. [SaitoS10a] Saito, Shiori "History of Touch INterface", Alberta College of Art and Design, CIRO Project, June 10, 2010: available at www.cirolab.org/blog/wpcontent/uploads/2010/06/History-of-Touch-Interface.pdf Somewhat sketchy history of touchscreen devices, does mention Mehta 1982 optical touchscreen (multitouch), Rekimoto multi-touch. [Samsung10a] Samsung "Galaxy Tab Tablet Mobile Product Information", www.samsungmobile.com, Dec 2010 Samsung Galaxy Tab mobile tablet device, includes Flash, android operating system. File contains additional material. [SchmidtD10a] Schmidt, Dominik; Chehimi, Fadi; Rukzio, Enrico; and Gellersen, Hans "PhoneTouch: A Technique for Direct Phone Interaction on Surfaces", Proc UIST '10, October 3-6, New York City, USA, pp. 13-16 Use input of smart phone such as iPhone (inertial input, keys, etc.) long with input from touch surface (simulated with video detection) as user input. [SchoeningJ09a] Schoening, Johannes; Krueger, Antonio; and Olivier, Patrick "Multi-touch is dead; Long live multi-touch", Proc. CHI 2009, April 2009, Boston Opinion piece on multi-touch having a 25 years history before Jeff Han and the iPhone from Apple, many designs not taking into account 25 years of HCI lessons learned (esp. Buxton) [Schweibenz09] Schweibenz, Eric. W. "eInstruction Corporation Files New 337 Complaint Regarding Certain Collaborative System Products", Press Release, LexisNexis, July 13, 2009 eInstruction whiteboard system using wireless tablets, versus importer Qomo Hitevision of wireless tablets, contributory infringement [SchweitzerB10a] Schweizer, Bobby "How You Should Gauge the Quality of a Touchscreen", www.tested.com, November 12, 2010 overview article on touchscreen products with short notes on multi-touch history, apparently taken from www.buxton.com. States that "flick" gesture first implemented in 1999: compare with PenPoint flick gestures in 1991? First multi-touch touchscreen by Boie at Bell Labs 1985, using capacitive array of touch sensors over CRT display. IBM Watson smartphone 1992. [SchwesigC08] Schwesig, Carsten "What makes an Interface feel Organic?", CACM, June 2008, Vol 51 No 6; pp 67-ff. Organic user interfaces / tangible user interfaces: Gummi flexible display, can be bent as a form of gesture input (for zoom) combined with touchpad / touchscreen sensing. Bending force determines how fast it zooms. [ScottJ10a] Scott, James; Izadi, Shahram; Rezai, Leila Sadat; Ruszkowski, Dominika; Bi, Xiaojun; Balakrishnan, Ravin "RearType: Text Entry Using Keys on the Back of a Device", Proc. MobileHCI 2010, September 7-10, 2010,



Lisbon. User interface with physical keyboard on rear of hand-held (two hands) display: compare with LucidTouch [SensAble09a] Sensable Technologies "Haptic Devices and Toolkits for Haptic Application Development", www.sensable.com Product information on hand input devices with haptic force feedback: mechanical linkages. Force/touch effects include constant/gravity, viscosity, 3D friction, spring, friction, stiffness, damping, front/back faces of virtual objects. [SensAble09b] Sensable Technologies "FreeForm Modeling / FreeForm Modeling Pluss: 3D Touch digitales Modellieren für Cad Cam", www.sensable.com 3D modeling software for displaying 3D models using Phantom 6DOF haptic feedback electromechanical stylus. Electronic file contains additional materials. [ShannonMM08] Shannon, Meg McGinity "This Menu has changed: Innovative interfaces will give wireless the golden touch", CACM, May 2008, Vol 51 No 5, pp 19-21 Essay on multi-touch interfaces, gesture with iPhone. Jeff Han cited as multi-touch. GestureTek video gesture recognition -- GestureTek in business since late 1980s. [ShenE09a] Shen, Ehr-li (Early); Tsai, Sung-sheng (Daniel); Chu, Hao-Hua "Double-side Multi-touch Input for Mobile Devices", CHI 2009, April 2009, Boston MA, pp. 4339-4344 Multitouch touchpanel on back/rear of PDA/iphone so that display is not obscured. Multi-touch gestures (e.g. two-finger "grab"), fingertip positions displayed with small cursors on front display. Also double-side input multi-touch gestures on both sides at same time. [ShermanE09a] Sherman, Erik "Can Apple's MultiTouch Patent Withstand Scrutiny - or Challenge?", BNET Technology Insights, Feb 6th 2009: www.bnet.com Review of published patent application relating to iPhone/Fingerworks touch gestures [ShinodaH10a] Shinoda, Hiroyuki; Chigusa, Hiromasa; and Makino, Yasutoshi "Flexible Tactile Sensor Skin Using Wireless Sensor Elements Coupled with 2D Microwaves", Jnl. of Robotics and Mechatronics, Vol 22 No 6, 2010, pp. 784-789 Prototype multitouch tactile sensor skin, rectangular matrix of (distinct) RFID sensor elements in flexible insulating material between two conductive sheets, RFID sensors powered by microwaves. When enough force applied to sensor elements, mechanical switch on RFID closes, RFID sends signal (in the RFID way). Because RFID elements not affixed to sheets, no points of stress to break. [ShirleyP09a] Shirley, Peter and Marschner, Steve (eds.) "Fundamentals of Computer Graphics, 3rd Edition", CRC Press, 2009 Somewhat more advanced textbook topics on computer graphics: alpha mask/channel == transparency mask (image blending). Textures, ray-tracing, etc. but not bitblt or simple masking. [Shneiderman10a] Shneiderman, Ben and Plaisant, Catherine "Designing the User Interface: Strategies for Effective Human-Computer Interaction, 5th Edition", Addison-Wesley, publishers 2010 (hardcopy book) University textbook on computer-human interaction - Earlier editions 1987, 1998. Discussion of direct manipulation, with Microsoft Office as an example -- direct manipulation because you have WYSIWYG behavior in the document, not because it is a tangible user interface. One chapter on input devices talks about pointing/digitizer input, Graffiti, some use of gestures. Earlier editions go back to 1986. [SilberschatzA09a] Silberschatz, Avi; Calvin, Peter Baer; and Gage, Greg "Operating System Concepts, 8th Ed.", John Wiley and Sons, Inc., 2009 Upper-level course textbook on operating systems design and architecture: Unix (Mach, UNIX BSD), Windows (XT/2000,64). Historical notes on Multics protection domains / ring system, ACLs and capabilities; demand paging 1960 on B5500 and Atlas. [SimaSystems08a] SiMa Systems, Inc. "SiMa Systems Product Development Platform (PDP) for multi-touch/dual force (MT/DF) touchscreen sensor.", SiMa Systems, Inc., March 2008 pressure/force-sensitive (?) resistive multi-touch touch sensor: Multiple Simultaneous Touch Activation, user presses harder to activate, stylus recognized by smaller contact area. microTILE electronics for reference design. Electronic file contains additional information. [SlatterDN10a] Slatter, David Neil "Digital Pen and Paper System", United States Patent 7,756,364, July 13, 2010 Application of Anoto-like optical pen with paper pattern (cites to Anoto): processor in stylus does handwriting recognition, shows text in small display in pen barrel. Includes use of stylus orientation/angle. [SmartTech09a] SMART Technologies "Specifications SMART Table Model 230i", www.smarttech.com Multi-touch tablet smart table, rear-camera with rear projection: DViT digital vision touch. Includes SDK toolkit for Windows. [SmartTech10a] SMART Technologies "SMART Technologies Inc. PROSPECTUS", www.smarttech.com SMART board interactive whiteboard digitizers, with digital ink, multi-touch: Prospectus for public offerin, educational applications [Sobell08a] Sobell, Mark G. "A Practical Guide to Red Hat Linux, Fourth Edition.", www.sobell.com/RH4 (author



publication) Definition Input Focus. The input focus specifies the window that receives keyboard input. [Spiegel09a] Der Spiegel "Apples neue Maus hat keine Tasten", Der Spiegel, Germany: October 20, 2009 multi-touch mouse has no wheels or buttons: uses a touch surface. [Spiegel10a] Der Spiegel "Warum viele Blinde das iPhone lieben", Der Spiegel, Germany: October 17, 2010 Review of VoiceOver for iPhone: refers to gesture input on touchscreens has being the solution for visuallyimpaired/blind computer access [Spiegel10b] Der Spiegel "CeBIT vor zehn Jahren: iPad-Opa und Bluetooth-Waschmaschinen", Der Spiegel, Germany: Feb 28, 2010 Mentions skeye.pad webpad tablet computer at CeBIT 2000 about the time of the Tablet PC introduction by Microsoft: Wireless connection, Webpanel, touchscreen interface with browser. [Stantum09a] Stantum.com "PMatrix: The unparalleled Multi-Touch detection firmware", www.stantum.com Formerly known as JazzMutant / Lemur. Resistive-grid multi-touch detection using X/Y addressable nodes in a grid pattern, notes that it works with any touch stylus, unlike capacitive digitizer. Includes gesturerecognition software for pinch, twist, rotate, close, zoom gestures. Mentions bounce on scroll action, similar to iPhone/iPad? [Stantum09b] Stantum.com "Unlimited multi-touch", www.stantum.com Resistive-grid multi-touch detection using X/Y addressable nodes in a grid pattern, notes that it works with any touch stylus, unlike capacitive digitizer. [Stantum09c] Stantum.com "Stantum adds Finger-Pressure Detection to Its Patented PMatrix Multi-Touch Detection Platform", www.stantum.com pressure/force-sensing multi-touch resistive touchscreen digitizer. [Stantum10a] Stantum.com "Stantum Unlimited Multitouch: How it Works", www.stantum.com Capacitive multi-touch that can detect both fingers and stylus (passive) at same time: apparently by size of contact/detection area? [StefikMJ09a] Stefik, Mark J. and Pirolli, Peter L. "System for Controlling the Distribution and Use of Digital Works", United States Patent 7,523,072, April 21, 2009 DRM usage rights management, usage rights control what rendering/use is granted to a document or object, discete DRM control for discrete parts of document/object, DRM rights stored separately from managed object. Compare with Benson certificates? [StewartWM09a] Stewart, Walter Mason; Carrera, Marcelo; and Hook, Robert G. "E-Mail Virus Protection System and Method", United States Patent 7,506,155, Marcy 17, 2009 Filter Email for virus detection by executing program on another computer to check before sending to email recipient: send recipient converted file (e.g. PDF) instead. Virus is assumed to be executable code. [StuerzlingerW08a] Stuerzlinger, Wolfgang and Wingrave, Chadwick A. "The Value of Constraints for 3D User Interfaces", Proc. Virtual Realities: Dagstuhl Seminar 2008, Uni. Trier, Germany, pp. 203..224 Review of 3D input and 3D display in consumer-level systems: Perspective and occlusion more important than stereoscopic display. 2D touchscreens better than existing 3D input devices, mostly because of accuracy and precision of the device. Additional references on 3D input with similar devices. [SturmI09a] Sturm, Irene; Schiewe, Maria; Köhlmann, Wiebke; and Jürgensen, Helmut "Communicating_through_Gestures_without_Visual_Feedback", Proc. ACM PETRA '09, June 09..13, 2009, Corfu, Greece Presentation of ideas for non-visual (audio, tactile) feedback and touchscreen gestures using BrailleDis 9000 tactile display, and generally for a user interface for the blind with tactile graphical output. BrailleDis (HyperBraille) returns touch-intensity data from a coordinate matrix (capacitive?). [SuehleR10a] Suehle, Ruth "Five open source alternatives to the iPad", opensource.com, 4 Apr 2010 web tablets, open source: Neofonie WePad with Android OS, TouchBook, iFreeTablet, MSI Taiwanese tablet with Android, Asus / Asusteck Eee Pad, rumored Google Tablet. [SutantoH10a] Sutanto, Herry; Spencer, Richard L; Landstad, Todd M.; Bhattacharyay, Subha; Wang, Haiyong; and Jones, F. David "Controlling Multiple Map application Operations with a Single Gesture", United States Patent 7,752,555, July 6, 2010 Using feature points/characteristics of a gesture in carrying out the operation of the gesture: characterizes this as performing multiple operations. Examples include endpoints and apex point of a checkmark. Compare with using single or multiple hotpoints of a gesture? [Synaptics08a] Synaptics Inc. "Synaptics RMI3 Interfacing Guide", Synaptics Inc. 2007/2008 Synaptics capacitive-grid touchpad hardware controllers: Devices support MultiFinger (multi-touch) detection (report count of fingers), Enhanced gestures (multi-finger pinch and zoom). [Synaptics08b] Synaptics Inc. "Touch Sensor features gester recognition technology", ThomasNet news: April 28 2008 Synaptics transparend capacitive-grid touchpad hardware controllers: Devices support MultiFinger (multi-



touch) detection (report count of fingers), Gesture recognition in controller (compare with Pencept recognition macros?). Multi-finger pinch and zoom). [Synaptics09a] Ingram, Shawn "Synaptics adds multi-finger gestures to non-Apple TouchPads", Gatetell.com, January 7, 2009 Synaptics: Multi-touch/multi-finger gestures in touchpad driver for multi-touch touchpad. Compare with Pencept recognition macros? [Synaptics10a] Synaptics Inc. "Synaptics RMI4 Interfacing Guide", Synaptics Inc. 2010 Synaptics capacitive-grid touchpad hardware controllers: Devices support NumberOfFingers detection (report count of fingers) with additional position registers for each finger position, TouchShapes contoured special areas on tablet. [Synaptics10b] Purdy, Kevin "Synaptics: Get Mac-Like Scrolling and Gestures on a Windows Laptop", lifehacker.com, March 16, 2010 Synaptics driver/utility for Windows to enable multi-touch touchpad gestures in driver. One example: two fingers indidates scrolling operation of middle wheel on Microsoft mouse. [TIPB10a] TIPB.com "New Gestures in iPhone SDK 3.2 for iPad", tipb.com, March 10, 2010 Additional gestures for iPhone: Pinch-in/out/mix, Rotate-(anti)clockwise, 3Tap, etc. [Tabletized09] www.tabletized.com "Journey to the beginning of Tablets. The Apple Newton.", www.tabletized.com, fetched 2010 History of Newton development: Walter Smith developer of NewtonScript object-oriented programming language. Original price for Newton Toolkit was $1000. Developers of Newton Toolkit: Norberto Menendez, Ben Sharpe, Peter Potrebic. Appears to be copy of Wikipedia article on Newton (platform). [TanenbaumAS09a] Tanenbaum, Andrew S. "Modern Operating Systems, 3rd Ed.", Prentice Hall, 2006 Upper-level textbook operating system design, case studies of Linux and Windows Vista (7). ACLs access control lists stored by row, or by column. Computer security also discusses logic bombs, insider attacks, buffer overflow attacks, format string, return to libc, malware (ransomware) etc. [TappertCC10a] Tappert, Charles C. "Example of Historical Research: Palm v. Xerox Patent Infringement Lawsuit", Course presentation, Seidenberg School of CSIS, Pace University, NY, January 1, 2010 Personal record of expert-witness analysis concerning Xerox Unistroke handwriting recognition patent in Palm. v Vexor patent suit. Prior-art research lead to limiting of claims, and thus to a settlement. Interesting record of how prior-art research is done. [TaubDM09a] Taub, Daniel Matthew "The BoPen: A Tangible Pointer tracked in Six Degrees of Freedom", Master's Thesis, EECS/MIT, August 21, 2009 Light-pen-like device using projected barcode pattern, projector uses switchable diffuse/clear surface that can project onto the surface and also see objects above the surface. Credits Doug Englebart with first GUI graphical user interface. Cites to this bibliography and to this collection by name "which made my related works section both more interesting and immensely more time-consuming", with thanks to Jean Renard Ward [TechHotSpot10a] Unknown "Pen Computing", www.iturls.com/English/TechHotspot/TH_pen.asp Compendium of URL links to pen computing, lists this reference. Terminology: uses Diagram Recognition for Sketch Recognition, includes several Chinese commercial handwriting recognition / pen-computing links, multi-touch, early penpoint, etc. [TechNet10a] TechNetYes.blogspot.com "The Windows PC ClickPad finally improved? Synaptics ClickPad IS Series 3 preview", technetyes.blogspot.com/2010/11/endgadget-news_19.html, November 2010 Synaptics ClickPad 3 touchpad with ACM accidental contact mitigation. Shows configuration for optional multifinger (multi-touch) gesture: two-finger scrolling, pinch zoom, rotating, three-finger press / flick, four=-finger flick, scrolling (at edge?), tapping, clicking (separate?), etc. [TechnologyReview08a] Technology Review "Mouse for 3-D Navigation; Do-It-Yourself Multitouch Display", TechnologyRview, July/August 2008, page 30 SpaceNavigator 3-D mouse, www.3dconnexion.com -- six-degrees-of-freedom joystick-like input device. nortd.com TouchKit -- open-source multitouch table input tablet, apparently optical [TheEngineer08a] The Engineer "Magic touch", www.theengineer.co.uk, July 28, 2008 Review of some touch-screen and haptic technologies (seems to confuse "haptic" with "touch-input"). Roope Takala, Haptikos project at Nokia. Christophe Ramstein, Immersion corporation. Gary Todd, Virtual Veins. Mark Williams, Warwick Manufacturing Group (WMG), UK. iPhone. [ThreeDconnexion08] 3dConnexion "Perfekter Begleiter für Globe Strotter: 3Dconnexion stellt SpaceNavigator für Notebooks vor", www.3dconnexion.de Press release for SpaceNavigator, three-dimensional mouse pointing device, using force/pressure for up and down, also rotation. [ToepkeMG08a] Topeke, Michael G.; Blum, Jeffrey R.; and Parker, Kathryn L. "Soft Input Panel System and Method", United States Paten 7,411,582, August 12, 2008



Virtual Keyboard and other devices simulating actual devices by generating system messages same as drivers (e.g. keyboard driver) for physical input devices. SIP Soft Input Panel. Compare with PenCept, The Guide (Wang), PenWindows, other systems with simulated keyboards, including Ken from Wang, Freestyle, Doster/Oed [TouchInternational09a] Touch International "Touch International 57-B-5-12 Surface Capacitive Controller Specification Guide", Touch International Document 6500292 Rev 1.1, 2009 Five-wire surface capacitive touchscreen: shows wire connection diagram for five-wire interface. For Tablet PCs (Microsoft) [TouchUserInterface09a] TouchUserInterface.com "How the Stantum's Resistive Multi-Touch Screen Works", TouchUserInterface.com, July 18, 2009 Stantum multi-touch resistive grid touchscreen, description from 2007 patent application. Resistive grid with spacer dots, other rows and columns are set to high-impedance state. [TouchUserInterface10a] TouchUserInterface.com "Yet Another Thing and Flexible Resistive Multi Touch - SiMa Systems", TouchUserInterface.com, February 11, 2010 SiMa Systems multi-touch resistive grid touchscreen, MT/MF multi-touch multi-force. Includes "Dynamic home row" virtual keyboard/typing. Comparison with Stantum PMatrix, Peratech QTC quantum tunnelling composite and FSR force sensing resistor. [Touchco09a] TouchCo "Touchco IFSR Multi-touch Resistive Sensors", www.touchco.com Commercial company with touch/pressure sensitive digitizer using IFSR Force-sensitive resistor ink. Electronic file contains additional information on multi-touch resistive technology. Shows heat-map style image of hand on tablet surface, using actual force sensing / pressure sensing. [Touchco10a] TouchCo "Touch User Interface - Touch Screen and Multi Touch", www.touchco.com Detailed description of IFSR (ink force-sensing resistor) multi-touch touchscreen: Stantum / Touchco [Touchtisch08] Touchtisch Projekt "Touchtisch Ausgabe 01: Erste Erfolge", (in German) Touchtisch Projekt, Brauneggerstrasse 55, Konstanz, Germany, touchtisch.wordpress.com Student project on touch-sensitive interface, using in part a Wii hardware digitizer and Flash (software) on a MAC. [TurnbullJ09a] Turnbull, James; Lieverdink, Peter; and Matatek, Dennis "Pro Linux System Administration", apress.com, 2009 Linux administration manual: covers syntax and RegEx expressions in Linus ACLs access control lists. [TyphoonTouch09] Typhoon Touch Technologies "Typhoon Touch Technologies, press information", www.typhoontouchtechn.com Licensing company for two patents (see CloughWA) relating to application generators for tablet/pen computers. Company originally formed to distribute MP3 files. Press reports on settlements with Electrovaya, Motion Computing. See also Nova Mobility (product partner). EDGAR report states that original purpose of company was to sell MP3s on-line, acquired patents from Nova Mobility Systems. See file folder for patent information. [UllrichCJ09a] Ullrich, Christopher J.; Steger, Ryan; and Gomez, Daniel "Method and apparatus for multi-touch tactile touch panel actuator mechanisms", United States Patent Application 2009/0003238, January 1, 2009 Haptic/tactile display: grid of independent tactile cells (under multi-touch or single-touch touchscreen). Cells can have any kind of haptic/tactile output: piezoelectric, micro-electrio-mechanical element, thermal fluid pocket, laminar flow modulation, etc. Compare to Braille displays such as BrailleDis? [UyM09a] Uy, Michael "Integrated Sensing Display", United States Patent 7,535,468, May 19, 2009 Optical sensors (e.g. CCD imaging) integrated into display panel (e.g. LED). Applications include two-way video with user eye contact directed in same direction as camera, versus separate webcam camera mounted on top. Compare to in-cell optical technologies for touchscreens. [ValePO08a] Vale, Peter O. "Browser Navigation for Devices with a Limited Input System", United States Patent 7,454,718, November 18, 2008 Automatic user interface behavior of scroll, move to next field/element, or enter data using cursor keys (forward,down, etc.) for small display with small keyboard (smartphone) [VanMensvoortK09a] van Mensvoort, Koert Martinus "What You See is What You Feel", Eindhoven University of Technology, 2009, ISBN: 978-90-386-1672-8 PH.D. Thesis on haptic touch user interfaces. Force-feedback, mechanically simulated or optically simulated (via cursor displacements) or via animation of stickiness. Changing cursor cited to 1988 for WIMP interfaces. Dynamic cursor invented 1968 by Engelbart. References to force-feedback back to 1990. [VertegaalR08a] Vertegaal, Roel and Poupyrev, Ivan "Organic User Interfaces", CACM, June 2008, Vol 51 No 6, pp. 26-30 Introduction to special articles on organic/graspable/haptic user interfaces. Credits Alan Kaye (indirectly) for inspiring the Tablet PC. Deformable/rollable input devices, such as paper that can be bend or folded. [VilppulaM09] Vilppula, Matti; Mattila, Arto; and Niemi, Markku "Method and Device for Position Determination",



United States Patent 7,558,696 Handheld mobile (cellphone) device using GPS and other means for determining geographic position, API picks available means/method that meets accuracy requirements specified via the API [VoelkelT08a] Völkel, Thorsten; Weber, Gerhard; and Baumann, Ulrich "Tactile Graphics Revised: The Novel BrailleDis 9000 Pin-Matrix Device with Multitouch Input", Proc. ICCHP '08, 11th Int'l. Conf. on Computers Helping People with Special Needs, pp. 835-842 BrailleDis 9000 (BrailleDis 7200) tactile display for visually impaired. Pin-Matrix output, multi-touch input (capacitive?). Display pixels are braille dots using piezo-electric actuators. [VogelD10a] Vogel, Daniel "Direct Hand Input and Hand Occlusion", Ph.D. Dissertation, U. Toronto Dept. of Computer Science, 2010 Hand occlusion for pen input: points out problems of that pen / handwriting / gesture recognition is used with legacy/existing software. Pen input faster than pointing with a mouse -- compare with observations on Windows for Pen Computing and GO/PenPoint in early 1990's. [VogelD10b] Vogel, Daniel and Balakrishnan, Ravin "Direct Pen Interaction With a Conventional Graphical User Interface", Human Computing Iteration, 2010, Vol 25, pp. 324-388 Test of Tablet PC user interface (Microsoft Windows) with conventional mouse UI. Problems are poor precision with tapping and dragging, hand occlusion, hand instability, gorilla-arm fatigue, hand fatigue, difficulty moving to keyboard, difficulty with small targets/menus/icons, and lack of additional mouse buttons or roller. Based on Vogel Ph.D. Thesis. Various citations to Ward/Philipps 1987 paper. [VuurpijlL10a] Vuurpijl, Louis; Niels, Ralph; Willems, Don; van Gerven, Marcel "Design Issues for Pen-Centric Interactive Maps", Interactive Collaborative Information Systems, SCI 28l, pp273..297, Springer-Verlag (partial copy) Design study of pen-computing gestures, marking in interactive mapping system: iconic gestures and semantics specific to mapping (road blocked, paramedic symbol, etc.) [Walkenbach08a] Walkenbach, John "Lotus 1-2-3 Release History", j-walk.com/ss/123/index.htm History of Lotus 123 Release dates: First Windows (non-DOS) version in 1991, Lotus 1.0 for Windows. Via Voice speech voice recognition added in 1998. [Walkenbach08b] Walkenbach, John "Excel 2007: Where It Came From", "Excel 2007 Power Programming with VBA", Wiley.com: j-walk.com, Chapter 1 Excerpt Release and development history of Excel spreadsheet program: cites provenance from VisiCalc available at www.bricklin.com; Lotus 1-2-3 [WalkerG09a] Walker, Geoff "LCD In-Cell Touch", Tutorial Session, NextWindow Optical Touch, SID November 17, 2009 Tutorial on integrated in-cell LCD display/touchscreen technologies: Light-sensing, capacitive-sensing, switch-sensing (resistive/voltage). Optical: problems if black display / no ambient light. States Multi-touch from 1982, pinching gestures from 1983. Asserts capacitive cannot sense stylus (?). [WalkerG09b] Walker, Geoff "touch Technologies tutorial", Tutorial Session, NextWindow Optical Touch, SID October, 2009 Tutorial on touchscreen technologies: covers multi-touch, force sensing (Vissumo). Vissumo force sensors at edges of display, not at corners. Software solution to "occlusion" problem for multi-touch. [WalkerG10a] Walker, Geoff "Touch Technologies Tutorial", walkermobile.com, last update of April, 2010 Tutorial on touchscreen technologies. Haptic feedback. Distinguishes Touch Screens (passive) from Active Digitizers. Flexible displays. Non-multi-touch: analog resistive, Surface Acoustic Wave (SAW) Surface Capacitive (electrostatic), Traditional infrared (IR), EMR Electromagnetic Resonance, APR Acoustic Pulse Recognition, DST Dispersive Signal Technology, Force sensing (Vissumo). Multi-touch: Projected capacitive, LCD In-Cell (Optical, Switch, Capacitive), Optical, digital resistive, Waveguide infrared (RPO), Vision-base optical (e.g. includes FIR/FTIR). Figures on major vendors, market share. Alternative materials to ITO. No treatment of electromagnetic, acoustic/sonar, or Anoto pattern, as in Feb 2010 version? Covers Wacom EMR, RPO Waveguide infrared optical: waveguide layer on top of display. Circular polarizers to reduce visual reflections. Formulae for anti-glare non-smooth hardcoat surface coatings, anti-smudge hydrophobic coating, contrast ratio, anti-Newton's-rings, [WalkerG10b] Walker, Geoff and Gregg, Abbie "Fundamentals of Touch Technologies and Applications", Tutorial session, SemiCon West, July 14, 2010 Excerpt of Tutorial on touchscreen technologies. Continuous ITO film/sheet: analog resistive, surface capacitive. Multi-Touch. Touch technologies with patterned ITO: analog multi-touch resistive, projected capacitive, embedded on-cell and in-cell. Manufacturing processes e.g. photolithography Without ITO: Acoustic Pulse Recognition / APR; Dispersive Signal Technology / DST; Materials and Process 5: Etch; Surface Acoustic Wave / SAW; Waveguide Infrared / RPO; Optical; Force Sensing; Vision-Based; Electromagnetic resonance (EMR) multitouch tablets; ITO replacement materials (redacted): sub-micron metal mesh, silver (Ag) nanowires, carbon nanotubes, magnesium hydroxide, conductive polymer No



treatment of electrostatic stylus, acoustic/sonar, or Anoto pattern? [WalkerG10c] Walker, Geoff and Fihn, Mark "Frontline Technology: LCD In-Cell Touch", Information Display Vol 3 No 10, March 2010, pp. 8-14 Touchscreens with touch sensor physically integrated into display: Cites to 2007 SIGGRAPH publication. InCell generally projected capacitive (X/Y grid matrix), analog resistive, or digital resistive: light/optical sensors in display. One use is touchpad on laptop (not touch display) with displayable icons in touchpad. [WallDAE08a] Wall, David A.E. "System and method for processing digital documents utilizing secure communications over a network", United States Patent 7,360,079, April 15, 2008 Central server distributes "digitally signed" copies of electronic document to multiple receivers, signature as specified by sender, but trust relationships are only between sender and server, and receiver(s) and server. Compare with CertCo digital signatures? [WangF09a] Wang, Feng and Ren, Xiangshi "Empirical Evaluation for Finger Input Properties In Multi-Touch Interaction", Proc. CHI '09, April 7, 2009, Boston MA 1063-1072, Session "Tabletop Gestures" Determine shape of contact of fingertips on table-top digitizer (FTIR optical). Vertical contact and horizontal contact touch gestures, determine direction of finger from oval shape, gestures using intersection of two pointing fingers to get active point, rocking gestures. Use of pointing to reduce occlusion problem, directional sector (radial) menus. [WardM10a] Ward, Mark "Sensors turn skin into gadget control pad", BBC News, March 26, 2010 Skinput: acoustic digitizer mounted on arm: must learn/be trained to recognized characteristic sound of tapping on different parts of the skin surface. [WareWH08a] Ware, Willis H. "RAND and the Information Evolution: A History in Essays and Vignettes", RAND Corporation corporate publication series, 2008 Personal history of RAND Corporation. States that RAND tablet was first operational digitizing tablet, 10inch flat surface with free-pen stylus. Later printed-circuit version was 36 inches by 36 inches. Supported by JOHNNIAC project, also Videographic System to integrate characters and graphics (video) in same display, used for (electronic ink / gesture) annotation of maps with text. Saved user actions to reconstruct image rather than saving bitmap. Dictionary (stroke order) look-up to recognize handwritten Chinese characters. BIOMOD graphical user interface to construct biological models. First version of RAND tablet was electromechanical (reversed flatbed plotter), then woven grid of Formex wires with distinct signal on each wire: 0.1" resolution. Final version used (new at the time) PCD with 0.01" resolution, still unique signal on every conductor. Notes that users quickly adjusted to separate tablet and display. 100 data points/second, with filtering, stroke identified (handwriting recognition) of features relative positions of strokes, stroke directions, corners, end-point location, and context (1 vs I? 0 vs O?) History of GRAIL project (later BIOMOD and CLINFO). GRAIL was automatic recognition of gestures and characters for system flowcharts. BIOMOD graphical language for construction system models. [WeisbergDE08a] Weisberg, David E. "The Engineering Design Revolution: The People, Companies and Computer Systems That Changed Forever the Practice of Engineering", David E. Weisberg, 2008. Available at www.cadhistory.com History of CAD/CAM history. Chapter 7: Applicon. Gesture UI of Applicon CAD/CAM system known as pattern recognition command entry/ Tablet Symbol Recognition / sketching a pattern with a stylus. User could define new patterns/gestures. Circle/lasso gesture to select and zoom, dot-horizontal and dot-vertical for dimension lines. Gestures more clumsy with a mouse when implemented on Macintosh as MacBRAVO!. Ken Ledeen (Ledeen Recognizer) at ComputerVision in 1985. [WestermanW08a] Westerman, Wayne Carl; Lamireaux, Henri; and Dreisbach, Matthew Evan "Swipe Gestures for Touch Screen Keyboards", United States Patent Application Publication 2008/0316183 A1, December 25, 2008 Swipe gestures by moving finger across keys on (simulated) keyboard touch-screen: compare with 1980's Japanese calculator in collection? [WestermanW08b] Westerman, Wayne and Elias, John G. "Method and apparatus for integrating manual input", United States Patent 7,339,580, March 4, 2008 Broad description of Fingerworks/Apple electrostatic multi-touch touchscreen sensing and multi-touch gestures: claims are to pinch gestures for cut/paste. [WestlyE10a] Westly, Erica "A Layman's Guide to the Tablet PCs of 2010", www.fastcompany.com, January 14, 2010 Review of "slab-form" tablet PCs, including Android and rumored apple iSlate. Lenovo IdeaPad U1 running Linux with multitouch tablet as display portion of Windows PC, Plastic Logic e-reader with multi-touch capacitive digitizer and annotation, dual-screen courier tablet (Microsoft?) [WhiteJ09a] White, Jeremy "Multi-touch Interfaces and Map Navigation", Master's Thesis, (Cartography and GIS), U. Wisconsin/Madison, 2009 Physical design of large rear-projection multi-touch interactive surface (FTIR Frustrated Total Internal Reflection: see Han). Contains review of usability of multi-touch gestures over mouse/keyboard. Describes



algorithm for finding "center" of finger-touch blob for pointing. [WiViK09a] Prentke Romich Company, prentrom.com "WiViK On-screen Keyboard Version 3 / Microsoft Windows", Prentke Romich Company / Bloorview Kids Rehab, 2009 On-screen visual/virtual keyboard for accessibility: abbreviation expansion, word prediction from adaptable dictionary, expanding quadrant selection (compare with Marking menus). Click selection, dwell selection (using both a time and a distance threshold), scanning selection (hit switch when desired item is highlighted in a sequential scan of items). [WigdorD09a] Wigdor, Daniel; Leigh, Darren; Forlines, Clifton; Shipman, Samuel; Barnwell, John; Balakrishnan, Ravin; Shen, Cia "Under the Table Interaction", Proc. UIST '06, October 15-18, 2006, Montreux, Switzerland, pp. 256-268 Tabletop touchpad with multi-touch sensors on both top and bottom side of table. Under-the-table can use first-contact selection rather than lift-off, since cursor fully visible, not obscured by finger/hand. Bimanual/two hands: Under-the-table can also be used at same physical position as hand on top table, but for second display, or on same display. Also useful for simultaneous-input two-person games (e.g. Diplomacy), since hands/moves not visible under table. [Wikipedia08a] en.wikipedia.org "Mouse gesture", en.wikipedia.org/wiki/Gesture-based_interface General reference on mouse gestures: history first mouse gesture (not using a pre-defined shape) was "drag" by Apple to replace a dedicated "move" button on the mouse. Cites lack of feedback to show that the gesture was recognized correctly (showing actual gesture: compare with GO/Agulnick), and visibility to make gestures easy to learn. Cites to Buxton for awkwardness of mouse as a gesture device compared to stylus and tablet digitizer. [Wikipedia08b] en.wikipedia.org "Post-WIMP", en.wikipedia.org/wiki/Post-WIMP Post-WIMP: defines as paper sheet user interface, contrasted to WIMP Windows Icons, Menus, Pointing device user interface. See Andries van Dam [Wikipedia08c] en.wikipedia.org "Scrollbar", en.wikipedia.org/wiki/Scrollbar Mentions google chrome browser scroll bar which places highlights/marks in trough of vertical scroll bar to indicate places where term was found that user was looking for in a web page [Wikipedia08d] en.wikipedia.org "Wang Laboratories", en.wikipedia.org/wiki/Wang_Laboratories, fetched 2008 Wang Laboratories Freestyle electronic ink/voice whiteboarding and communication desktop system using Hitachi digitizer: has link to Freestyle demonstration video [Wikipedia08e] en.wikipedia.org "WIMP (Computing)", en.wikipedia.org, fetched 2008 WIMP user interface: window, icon, menu, pointing device: mentions post-WIMP (Andries van Dam) [Wikipedia10a] en.wikipedia.org "Handwriting movement analysis", en.wikipedia.org/wiki/Handwriting_movement_analysis Cites to Teulings and Jean Renard Ward. Mentions early history of pen-input devices and handwriting tablets: Scripture 1895 devices for storage of pen positions, McAllister 1900 [Wikipedia10b] en.wikipedia.org "Xcode", en.wikipedia.org, fetched 2010 Xcode IDE and development tools superceded Project Builder IDE, which was derived from NeXT IDE and tools. [Wikipedia10c] en.wikipedia.org "SideKick", en.wikipedia.org, fetched 2010 Borland SideKick PIM / Personal Information Manager for DOS: included calendar, text editor, calculator, and address book / phone dialer. Over one million copies sold in first three years (starting 1983). Later development done by Starfish Internet Sidekick. [Wikipedia10d] en.wikipedia.org "Ken Knowlton", en.wikipedia.org, fetched 2010 Biography on Ken Knowlton: cites to patent 3,879,722 on automatically relabeled keys/button (soft button lables) by projection, graphical flip book from Freestyle at Wan, character/handwriting recognition with Jourjine [Wikipedia10e] en.wikipedia.org "Multi-Touch", en.wikipedia.org, fetched 2010 Review article on Multi-Touch: states that 1982 movie "Tron" shows multi-touch -- compare with Tog's movie? [Wikipedia10f] de.wikipedia.org "Gestenerkennung", de.wikipedia.org, fetched 2010 Wikipedia: Gesture recognition. Describes gesture as motion of human body, does not mention pen or tablet gestures. Also on www.spiegel.de [Wikipedia10g] en.wikipedia.org "Gesture", en.wikipedia.org, fetched 2010 Wikipedia: Gesture. Describes gesture as visible body actions communicate particular messages. [Wikipedia10h] en.wikipedia.org "Gesture Recognition", en.wikipedia.org, fetched 2010 Wikipedia: Defines gesture recognition as recognizing human (body motion) gestures, not PenPoint-type gestures. [Wikipedia10i] en.wikipedia.org "Pointing Device Gesture", en.wikipedia.org, fetched 2010 Wikipedia: Gesture for pen computing. States that "drag" was first gesture on apple Macintosh, involving



motion while holding down mouse button. [Wikipedia10j] en.wikipedia.org "Pen computing", en.wikipedia.org, fetched 2010 Wikipedia: Pen computing -- contains (lengthy) historical notes going back to 19th century. direct manipulation/ handwriting recognition / gesture recognition. Absolute/relative pointing/locator input. [Wikipedia10k] en.wikipedia.org "Pie menu", en.wikipedia.org, fetched 2010 Wikipedia: Pie menu. Cites early references to pie menus to 1970s, as a gesture gestural user interface (similar to "drag" gesture?), makes assumptions about heirarchical menus and ability to chain multiple levels of menus. [Wikipedia10l] en.wikipedia.org "History of Microsoft Windows", en.wikipedia.org, fetched 2010 Wikipedia: Historical timeline of Microsoft Windows operating system. DOS addressing modes replaced with Windows 95. No mention of Windows for Pen Computing, or of Tablet PC. Comparison with OS/2. [Wikipedia10m] en.wikipedia.org "Windows CE 3.0", en.wikipedia.org, fetched 2010 Wikipedia: Windows CE 3.0 operating system ca. 2000 for palm-top pen computers, touchscreen digitizer. Reference has no information on gesture input. See also Windows [WillN10a] Will, Niko "On-Line Handschriftenerkennung (OLCR) der Deutschen Einheitskurzschrift", Master's thesis, Hochschule Ravensburg-Weingarten, Germany, 21 December 2010 German short-hand handwriting character recognition: segmentation based on histrogram, strokes, loopdetection. Uses chain codes. Shows stenopad: interactive ink-editing GUI on Microsoft INK API. [WilsonAD10a] Wilson, Andrew D. "Using a depth camera as a touch sensor", Proc. ITS 2010, November 7..10, 2010, Saarbrücken, Germany, pp. 69-72 Optical multitouch on un-instrumented surface/tabletop. Touch detected (approximately -- not perfect) by height above surface -- suggests using hand-posture information to improve. Advantages include ability to recognize movement in air/proximity, gestures in air (e.g. picking up an object and moving it), and know which of multiple touches were from the same user / same hand. [WingfieldN09a] Wingfield, Nick "Notes for Tablet Aware Application Developers", Wall Street Journal, January 12, 2009 Mentions touch gestures on Windows 7/Tablet PC not working well, HP implementing alternatives [WingfieldN09b] Wingfield, Nick "Microsoft Betting Big on 'Touch'", Wall Street Journal, January 12, 2009 Microsoft investing in N-trig Ltd. touchscreen hardware for Windows 7 / laptops in response to Apple iPhone touch-sensing screens on mobile phones. [WithanaA10a] Withana, Anusha; Kondo, Makoto; Makino, Yasutoshi; Kakehi, Gota; Sugimoto, Maki; and Inami, Masahiko "ImpAct: Immersive Haptic Stylus to Enable Direct Touch and Manipulation for Surface Computing", ACM Computers in Entertainment, Vol 8 NO 2, Article 9, December 2010, 16 pages Haptic stylus to give illusion of touching objects inside a flat tablet display: when pressed on screen, stylus shortens (collapsible moving shaft) and projected tip of virtual stylus elongates in displayed image. Described as "direct touch". Note: what type of touchscreen/tablet technology -- proximity sensing? [WitternH09a] Wittern, Hauke "Sketch Recognition", Colloquium of the Hochschule für Angewandte Wissenschaften Hamburg, May 28, 2009 Review of Sketch Recognition, cites to ASSIST system (and video) from M.I.T. and to Sutherland Sketchpad. Research focus is multi-touch input for sketch recognition in programming tools. Additional materials in electronic file [WobbrockJO09a] Wobbrock, Jacob O.; Morris, Meredith Ringel; and Wilson, Andrew D. "User-Defined Gestures for Surface Computing", Proc. CHI 2009, Boston MA, April 4-9, 2009 Design hand/finger gestures on touchscreen touch table by showing use the effect, and then letting user make what they think is an appropriate gesture: i.e. for pan or drag, user placed hand on surface and moved across as direct manipulation. Not so much intended for user-defined gestures, as helping designers pick gesture sets informed by user natural behavior. Refers to surface computing (two authors from Microsoft). Elicited gestures included single select: tap or lasso gesture. Group select: hold and tap (two hand), move: drag or jump gesture (one hand to the other); cut/paste: two separate taps with same hand; rubout gesture. Drags to/from off-screen. Cites to Wellner. [WobbrockJO10a] Wobbrock, Jacob O. and Myers, Brad A. "Using Edges and Corners for Character Input", United States Patent 7,729,542, June 1, 2010 EdgeWrite unistroke/graffiti gesture/character recognition using joystick input, similar to approach for Trackball EdgeWrite. User moves to edge limits when making character, somewhat similar to zone-based recognition. Chart of input alphabet, similar to presentation of Grafitti. [WooYoung09] Woo-Young, Kwak; Seung-Jun, Lee; and So-Young, Park "Scrolling method of mobile terminal", United States Patent Application 20090070711 Scrolling in mobile terminal, with feedback when the scrolling hits the last item of a menu list. Refers to flick gesture input, and to a graphical effect being displayed at the end of scrolling. [WoolfSD08a] Woolf, Susan D.; Baird, Andrew; Jiang, Sheng; Beezer, John L.; and Rubin, Darryl E. "System and



method for annotating an electronic document independently of its content", United States Patent 7,337,389, February 26, 2008 Adding electronic ink annotations for a document (browser) with ink, highlight (translucent electronic ink), erase (of electronic ink annotations): ink annotations stored separately from document. Compare with Wang Freestyle annotations, PenWindows annotation inking layer, van Raamsdonk, DotSpots? [WordNetWeb10a] WordNetWeb "Stroke; Stylus", wordnetweb.princeton.edu, fetched Feb 2011 Definition of stroke Definition of stylus [YangM08a] Yang, Ma; Leedham, Graham; Higgins, Colin; Htwe, Swe Myo "Segmentation and Recognition of Phonetic Features In Handwritten Pitman shorthand", Pattern Recognition, 2008 Paper file may have 2004 version: See also paper file for Higgins [YeeW09a] Yee, Wendy "Potential Limitations of Multi-touch Gesture Vocabulary: Differentiation, Adoption, Fatigue", Proc. 13th Intl. Conf. on Human-Computer Interaction HCI: Part II: Novel Interaction Techniques, pp. 291-300 (partial copy) direct gestures (direct manipulation) contrasted with indirect actions/gestures e.g. multi-finger swipe to clear windows. [YiK10a] Yi, Kwonju; Choi, Changkyu; Suh, Sungjoo; Yoo, ByungIn; Han, Jae-Joon; Park, Dusik; Kim, Changyeong "Novel LCD Display with a Sensible Backlight", SID Digests of Technical Papers, 2010 Prototype for multi-touch LCD touch screen with proximity sensing: an infrared sensor array integrated with pixels, infrared light source into glass, sensors pick up images causes by FTIR frustrated total internal reflections. Proximity sensing by infrared sources above the display surface, sensors pick up reflected infrared light -- no comment on blocking by other fingers? Sensors at finer spacing than the LED display pixels (two between each pair of pixels). Cites to JazzMutant as multitouch: gives no date, says resistive. Cites to DiamondTouch as multi-touch. [Zeleznik08a] Zeleznik, Robert; Bragdon, Andrew; Liu, Chu-Chi; and Forsberg, Andrew "Lineogrammer: Creating Diagrams by Drawing", Proc. UIST 08, pp. 161..170 Prettyfying graphical editor for line drawings: limited use of snap-to grid, heuristics to recognize drawings or text or gestures; double-circle / lasso gesture to select, pigtail lasso to delete (compare Scriboli), a GestureBar, scribble/scratch out, undo gestures similar to W4PC, zig-zag for zoom in / out, polilines, pressand-hold/dwell for panning. Uses timeout for distinguishing characters versus drawing. Supports overdrawing to connect disjoint lines. Vertex dragging. [ZeleznikR10a] Zeleznik, Robert; Bragdon, Andrew; Adeputra, Ferdi; and Ko, Hsu-Sheng "Hands-On Math: A page-based multi touch and pen desktop for technical work and problem solving", Proc. UIST '10, October 3-6, 2010, New York. pp. 17-26 Multi-touch electronic ink / mathematical text editing user interface using both pen and fingers: primarily for students applying algebraic manipulation. Users learn gestures by dynamic example much more readily than by text or verbal description (duh!). Ink/text adjusts for cut/paste/move (Compare with van Raamsdonk?). Two-finger gestures for open space, swap positions, move, select. Gestures include shape of hand/palm on surface, vertical angle orientation of finger (see Rekimoto?). Instead of shape/path gestures, menu under fingertips of non-dominant hand to select tools (see Buxton bi-manual). [ZibregC09a] Zibreg, Christian "Redmond goes after Apple with five touch-enabled wild mice of the future", geek.com, October 6, 2009 Touch-enabled / multi-touch mouse prototypes from Microsoft, contrasted to multi-touch touchscreen of Apple. Touch gestures on mouse -- compare with Apple touch mouse? Technologies include mechanical (two finger cups on actuator arms), optical, capacitive touch, FTIR to detect fingers on surface of mouse, tracking of finger gestures around the mouse not on surface. Includes video demonstraction. [ZibregC09b] Zibreg, Christian "Apple's A-bomb against Palm: A 358 page iPhone patent", www.tgdaily.com, January 27, 2009 Reference to Palm Pre and Apple iPhone and unspecified patent claims concerning separate gesture area from display touchscreen in hand-held device [3MTouch08a] 3M Touch "Dispersive Signal Touch Technology: Technology Profile", 3M Touch, 2008 Acoustic touch digitizer, using bending waves within substrate/glass instead of surface waves SAW, determines center location of smearing/dispersed/reflected acoustic waves. Compare with TouchDynamics / Acoustic Pulse Recognition? biblio07 (Prev page)

Main Page


"You can win all the battles, and still lose the war." -- Samurai saying "Die Zeit ist nur das, was man an der Uhr abliest."



-- Albert Einstein, bekannter Physiker und Geigenspieler





Main Page


[Acer11a] Acer Corp. "Acer ICONIA dual-screen touch PC: product information", Acer.com fetched 2011 Dual-screen folding tablet PC touch computer: compare with Refalo, Sony S2 [AhmadN13a] Ahmed, Nasir; Szymkowiak, Andrea; and Campbell, Paul A. "Keystroke dynamics in the pretouchscreen era", Frontiers in Human Neuroscience Vol 7, December 2013, Article 835 (www.frontiersin.org) overview of keystroke dynamics / typing characteristics for biometric user authentication. Cites to Gaines 1980 article, not to Saltzer 1975: bibliography mostly post-1990, but extensive, includes touchscreen/touch biometrics. [AlexanderJ11a] Alexander, Jason; Marshall, Mark T.; and Subramanian, Sriram "Adding Haptic Feedback to Mobile TV", Proc. CHI 2011, Work-in-Progress presentation; Vancouver BC May 2011 Ultrasonic standing waves used to give haptic feedback (vibrotactive) of touch above touchscreen, acoustic radiation pressure: low-frequency ultrasonic (40 KHz) does not penetrate body, so produces sensation on skin. Using vibration of back side of tablet-sized hand-held video display. Question of *what* feedback to provide left open. Part of UltraHaptics project. [Alps12a] Alps Electric "Touch Input Devices - Electronic Components - Alps Electric", www.alps.com, fetched 2012 Vendor of touchscreen, touchpad, tablet devices. Capacitive and resistive touchscreens, touch sensors/switches, GlideSensor. [AmmDT13a] Amm, David T.; Schlub, Robert W.; Leung, Omar S.; King, Brian M.; Li, Qingxiang; Vazquez, Enrique Ayala; Angulo, Rodney Andres Gomez; Jiang, Yi; and Caballero, Ruben "Electronic devices with capacitive proximity sensors for proximity-based radio-frequency power control", United States Patent 8,432,322, April 30, 2013 Integrate capacitive proximity sensor with a radio-frequency antenna (can be in front of antenna). Used in touchscreen cellphone or PDA to reduce specific absorption rate SAR electromagnetic radiation into users body when detector senses proximity. Compare with J.R. Smith, Roomware, Eastty hand proximity detection? [AmmaC12a] Amma, Christoph; Georgi, Marcus; and Schultz, Tanja "Airwriting: Hands-free Mobile Text Input by



Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors", 2012 16th International Symposium on Wearable Computers, p. 52-59 Recognize handwriting from individual HMM Hidden Markov Models of hand motion gestures using 3D accelerometer and 3D gyroscope affixed to hand. Citations to 3-D gesture recognition and system: compare with Crane accelerometer pen? File contains additional references: "A Glove That Lets You Write in the Air" (MIT Technology Review); "Writing without Keyboard: Handwriting Recognition on the Wrist" KIT Karlsruhe Institute of Technology [AnalogDevices11a] Analog Devices, Inc. "Analog Devices Data Sheet AD7147: CapTouch Programmable Controller for Single-Electrode Capacitance Sensors", Analog Devices www.analog.com, 2011 Capacitive-to-digital A/D for single-electrode capacitive sensors (capacitance to ground). Intended for portable devices. 13 sensor inputs channeled through a switch matrix. 9 msec update rate at 64 (6-bit) resolution, 36 msec at 256 (8-bit) resolution. Low-power mode: up to 800 msec update rate. [AndroidCookbook11a] Cowham, Adrian "Android Recipe for Haptic Feedback", Receipt Haptic Feedback (Recipe 1242, Revision 2945) in Android Cookbook: A Crowd-sourced Cookbook on Writing Great Android Apps: chapter 7, Graphical User Interface Programming example for haptic feedback using Android vibrator actuator (Motorola Xoom has no vibrator) in response to OnTouch events. [Apple12a] Apple, Inc. "iPhone User Guide for iOS 6 Software", Apple, Inc. 2012 VoiceOver accessibility touchscreen user interface for visual impaired/blind: Taps, swipes with multiple fingers/multi-touch up to four, two-finger scrub gesture. Road following on maps using acoustic feedback: pitch rises as farther from route. Two finger "rotator" gesture to select from rotary menu. Keyboard shortcuts with VoiceOver. Touch and hold gesture (press-and-hold). Handwriting recognition for chinese Kanji/Wubihua (stroke) characters. [ArifAS13a] Arif, Ahmed Sabbir and Stuerzlinger, Wolfgang "Evaluation of a New Error Prevention Technique for Mobile Touchscreen Text Entry", Proc. ACM OZCHI '13, November 2013, Adelaide, Australia User pressure/force inferred on touchscreen from time duration (press harder --> press longer), contact area, or larger shift in position when pressing hard. Pressing harder helps with accuracy only for novice users with virtual keyboard. [AshtianiB11a] Ashtiani, Behrooz and Stuerzlinger, Wolfgang "2D Similarity Transformations on Multi-Touch Surfaces", Graphics Interface 2011, May 25-27, St. John's, Newfoundland, CA, pp. 57-64 Multi-touch rotate/translate/scale gestures on touchscreen tabletop work better than separate gestures. Cites to Kruger 2005 paper on single-touch gestures for rotation by touching at corners. [Asus11a] Asus "Eee PC T101MT product information", usa.asus.com, fetched 2011 Eee PC T101MT multi-touch tablet PC. Resistive (?) touchscreen digitizer, netbook size laptop with swivel display to fold into tablet/notebook configuration: compare with earlier HP Tablet PCs running XP/Tablet PC version of Windows. Windows 7 multi-touch software. Also uses stylus. File contains additional materials. [AtkinsonR11a] Ran Atkinson "Ran Atkinson correspondence", ottawa.on.ca/ipsec discussion list, fetched 2011 Discussion concerning Atkinson cryptography patents and IPSEC standards. [Atmel11a] Atmel Corporation "QTAN0079: Buttons, Sliders and Wheels -- Sensor Design Guide", Atmel Corporation, 2011 Design guide to constructing capacitive user interface touch and touchscreen sensors. Self-capacitance and mutual-capacitance. Resistive and spatial interpolation. General tutorial into physics of capacitive touch sensors, mostly zero-dimensional sensors (buttons) and one-dimensional sensors (sliders), and simple two-dimensional sensors (wheels, as in original iPod, and simple X/Y touchscreens). QTouch sensors are self-capacitance, QMatrix sensors by Atmel are mutual capacitance. Capacitive sensing measurement by charge-transfer (time delay to discharge). Nearby LEDs a source of interference/noise, guarding against ESD electrostatic discharge. Shape of ground/shield electrodes particular important for self-capacitance. [BadayeM12a] Badaye, Massoud; Vavaroutsos, Peter G.; Carey, John; and Prendergast, Patrick "Single layer touch sensor", United States Patent Application 2012/0256642 A1, October 11, 2012 Mutual capacitance multi-touch capacitive touchscreen, single layer geometry. Each column conductor (for the full column) connects at a bottom edge of substrate, a vertically-running set of separate pads next to it, each with a thin trace running up to the top edge. At the top edge, there are two layers where a matrix/grid of horizontal traces traces has connections at diagonal points to the pads' vertical traces, giving a logical wiring of all the pads in the same row together. Thus on top surface, only vertical traces. Pads and column are interdigitated (comb shapes), or interdigitated triangles etc. to maximize mutual capacitance. [BahrJ11a] Bahr, Jakob; Peschke, Joshua; Kammer, Dietrich; Franke, Ingmar; Steinhauf, Juliane; Kirchner, Maxi; Schoenefeld, Frank; and Groh, Rainer "SampleSurface: Kollaboratives Musizieren mit Multitouch-Unterstützung", Workshop-Proceedings der Tagung Mensch und Computer 2011, Technische Universität Chemnitz, Germany, pp. 267-271



touchscreen/tabletop user interface for electronic music; audio loops, waveform sampling, audio editing and selection. Hardware is Microsoft Surface SDK. Includes video demonstration showing multi finger gestures, zoom rotate stretch. Compare with Kurtenbach audio interface with marking menus? [BarrettG12a] Barrett, Gary L. "TouchInternational -- Examining Glass Substrates: An Insider's Guide to Stronger and Scratch Resistance Glass", Whitepaper, TouchInternational 2012 Review of high-durability and scratch-resistant glass substrates for touchscreens, including Gorilla Glass chemically strengthened glass, tempered glass. Comment on cracking of iPhone 4 screens. [BeavenDF12a] Beaven, Douglas F. "Processing management information", United States Patent 8,095,413, January 10, 2012 Project and resource management system [BedersonBB11a] Bederson, Benjamin B. "The promise of zoomable user interfaces", Behavior and Information Technology, Vol 30 No 6, Nov 2011, pp. 853-866 Zoomable user interfaces: more information than fits on the screen. Broad review of applicable techniques and considerations. [BenkoH12a] Benko, Hrvoje; Jota, Ricardo; and Wilson, Andrew D. "MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop", Proc. CHI '12, May 5-10, 2012, Austin TX, pp. 199-208 Primarily virtual reality display project: Tabletop touch surface, tabletop surface is curved upward (compare e-Parcel publicity photo) so that 3D objects can be projected at height, along with 3D glasses for binocular vision. Examples include 3D digitization of objects and then projected adjacent. Gestures: Bowling with virtual bowling ball projected onto hand. Compare as 3D version of Wellner desk. [BlakeJ11a] Blake, Joshua "Natural User Interfaces in .NET", Manning Publications, 2011: www.manning.com Textbook/tutorial on Microsoft .NET (dotNET) APIs for multi-touch SDK controls, WPF Touch, Microsoft Surface SDK. Comparison of touch APIs in Microsoft WPF, Silverlight, and Windows Phone 7. Natural User Interfaces / NUI defined in terms of interaction style ("innate abilities" of user), GUI/CLI defined in terms of device (e.g. WIMP). ScatterView paradigm. Brief discussion of multi-touch touchscreen technologies: FTIR distinct from Diffused Illumination (Microsoft Surface). Multi-touch Vista open-source to emulate multi-touch input using multiple mice on Tangible User Interface Object TUIO protocol. Rotatable onscreen virtual keyboard. [BlindmannG12a] Blindmann, Gennadi " Multitouch technologies: Overview Acoustic Analog Based Bending Boat BWT Capacitive", Multi Touch Solutions Group, 2012. www.multi-touch-solution.com Overview of multi-touch tablet technologies, primarily on physical construction. Multi-touch capacitive: Surface capacitive, projected capacitive, In-cell capacitive. Multi-touch resistive: Analog resistive, in-cell digital resistive. Multi-touch optical: optical imaging/infrared, rear diffused illumination, infrared grid/optomatrix/digital waveguide touch (DWT) infrared optical waveguide, frustrated total internal reflection (FTIR)/ diffused surface illumination (DSI), Kinect, in-cell optical. Multi-touch wave/acoustic: surface acoustic wave (SAW), bending wave technology (BWT) Multi-touch force-based sensing [Bluecava11a] Bluecava.com "How it works: the unique fingerprint of an electronic device / U.S. Patent 5,490,216 and what it means for you", www.uniloc.com, fetched January 2011 Copy-protection / anti-piracy patent 5,490,216 on generating license code from information of system on which software will execute: litigation involving Microsoft. Applications include acting as "super-cookie" for targeting advertising to particular individuals. File contains additional materials. See also Uniloc file. [BogueA11a] Bogue, Adam "Filtering input stream in a multi-touch system", United States Patent Application 2011/0175827, July 21, 2011 On a multi-touch tablet, only respond to the first touch, ignore additional touches. Or assign additional input streams to the input stream of the first touch. [BoscagliaNS11a] Boscaglia, Nuno Simao; Gaudio, Lucas Avancik; and Ribeiro, Moises Renato Nunes "A Low Cost Prototype for an Optical and Haptic Pen", Proc. Biosignals and Biorobotics Conf., Brazil, January 2011, pp. 1-4 Pen stylus with photodiode and tip-force (pressure) sensors on front, solenoid for haptic feedback on top. Optical photodiode can sense light intensity from LCD or CRT, looks like position sensing was up to the tablet digitizer (e.g. resistive film). [BrakensiekJ13a] Brakensiek, Jörg and Bose, Raja "Method and apparatus for precluding operations associated with accidental touch inputs", United States Published Patent Application 2013/0050133 A1, February 28, 2013 Palm rejection -- or better "light touch" rejection -- on capacitive touchscreen by filtering out light touches (light force) [BrownRG13a] Brown, Robert G. "Introductory Physics I: Elementary Mechanics", Copyright Robert G. Brown, 2011. Lulu Press www.lulu.com First year college physics reference [BrownSR12a] Brown, Sean Rigby; Long, James G.; Patel, Jithesh Kumar; Walmsley, John Geoffery; Egan, James Authur; and Acs, Teresa "Dimensionally Stable White Board", United States Published Patent Application 2012/0178074A1, July 12, 2012



Teamboard: whiteboard digitizer using pressure/force sensitive resistive layer between two conductive layers, advantage is greater physical and coordinate stability. Single-touch, not multi-touch, can do multitouch by dividing into independent segments (?). [BrownSR12b] Brown, Sean Rigby; Long, James G.; Patel, Jithesh Kumar; Walmsley, John Geoffery; Egan, James Authur; and Acs, Teresa "White Board Operable by Variable Pressure Inputs", United States Published Patent Application 2012/0176328 A1, July 12, 2012 Operating a pressure (force) sensitive whiteboard based on gestures, degree of pressure, and multi-touch inputs, also coupled with a display. Appears to be about the user interface. [BruderG13b] Bruder, Gerd; Steinicke, Frank; and Stuerzlinger, Wolfgang "Touching the Void Revisited: Analyses of Touch Behavior On and Above Tabletop Surfaces", Proc. InterAct '13, 2013 pp. 278-296 Combining touchscreen (multi-touch) with stereoscopic 3D display. Physical touch surface does not coincided with projected 3D object. If used 3D detection (proximity height, or optical detection in space), still difficult to match "touch" location to projected 3D object location. Stereoscopic multi-touch tabletop. [BuxtonB11a] Buxton, W. "Resource Page on Early HCI Research of the Lincoln Lab TX-2 Group", http://www.billbuxton.com/Lincoln.html Mentions sources for videos for Sketchpad, Graphical Programming, Genesys, interviews with early researchers before 1966. [BuxtonB12a] Buxton, Bill "Chapter 2:Input Devices: An Illustrated Tour", www.billbuxton.com, fetched 2012 Draft book chapter by Bill Buxton on input devices. Text entry: keyboards, speech recognition, written input (handwriting recognition), graphical/virtual keyboard, sign language gestures, auto-completion computer algorithms. "Each is likely best for something and worst for something else." Cites to Micropad 1979. Mouse: additional thumbwheels, FEELit mouse with haptic push-back feedback. Digitizing tablets (stylus/puck), touch tablets, padmouse multi-touch input 1998, Casio PF-8000 1980, joysticks, armatures (mechanical digitizer), flying mice / bats. Gloves, gyroscopic. Eye-tracking. [BuxtonWA11a] Buxton, Bill "Some Milestones in Computer Input Devices: An Informal Timeline", billbuxton.com: fetched 2011 Lightgun 1950 from SAGE; Lightpen 1959; Glove based keyboard 1963 (DataGlove?); Graphics Tablet as Rand Tablet (Not Stylator?); 3-D three dimensional digitizer 1966; Videoplace dwell-time pointing 1985. [BuxtonWA11b] Buxton, Bill "Buxton Collection: Handheld devices", research.microsoft.com/enus/um/people/bibuxton/buxtoncollection: fetched 2011 Collection of handheld interactive computer devices, many with touchpad/touchscreen input: Apple Newton, AST GridPad, Casio PF-8000, Casio CB-100, General Magic Data Rover 840, IBM Simon, Microsoft UMPC Tablet PC, Microwriter (Datapad?), Palm Pilot, Xerox Parc Parctab, Symbol CS200 Personal Scanner, Psion PDA, etc. [BuxtonWA12a] Buxton, William A. "Technical Perspective: Innovative Interaction -- From Concept to the Wild", CACM, Sep 2012, Vol 55 No 9, p. 90 Commentary on Palm Pilot touchscreen PDA, and unistroke/single-stroke shorthand gestures Notae Tironianae by Marcu Tullius, Casio PF8000 Data Bank 1984 (see file) [CarterT13a] Carter, Tom; Seah, Sue Ann; Long, Benjamin; Drinkwater, Bruce; and Subramanian, Sriram "UltraHaptics: Multi-Point Mid-Air Haptic Feedback for Touch Surfaces", (to appear) Proc. ACM UIST 2013, October 2013, St. Andrews, UK Multi-point haptic feedback / sense of touching over multi-touch tablet using focused ultrasonic sound waves. One application is touch feedback in proximity for eyes-free use of multitouch touchscreen: acoustic radiation force, acoustically transparent display with small holes that pass sound transparently at 20KHz frequencies and higher. Electronic file has additional materials. [ChangSC11a] Chang, Shih Chang "Integrated Touch Screen", United States Patent 7,995,041, August 9, 2011 Touchscreen with capacitive/electrostatic sensor elements integrated between LCD/LED display pixels. (incell) [ChenKY13a] Chen, KeYu; Cohn, Gabe; Gupta, Sidhant; and Patel, Shwetak N. "uTouch: Sensing Touch Gestures on Unmodified LCDs", Proc. CHI '13, April 27-May 2, 2013 Capacitive non-touch touchscreen (uTouch) detects hover, grasp, touch and other gestures by detecting electromagnetic interference (pick up on a room power line) emitted via coupling to user's body. Used machine-learning trainable classifier for recognition. Cites to Abileah, Rekimoto. [ChinaPatent13a] (Chinese names not transliterated) "Graphene capacitive touch screen", China Patent CN203250299U, October 23, 2013 Manufacturing process for multi-touch two-layer patterned graphene touchscreen (instead of ITO). Laser etching to avoid mechanical damage to graphene or to substrate/base glass. [CholewinEA13a] Cholewin, Erik A.; Gorsica, John J.; Lombardi, Michael J.; Prochnow, Natalie, J.; and Snyder, Ian M. "Method for implementing zoom functionality on a portable device with opposing touch sensitive surfaces", United States Patent 8,462,126, June 11, 2013



Touchpad is on back of clamshell handlheld computer, so that display is not obscured. Cites to LucidTouch. Can also have touchscreen on front display for two-handed input. [CieslaCM12a] Ciesla, Cragi Michael and Yairi, Micah B. "User Interface System", United States Patent 8,179,377, May 15, 2012 Tactile display raises bumps in a flexible touch surface by pumping fluid via a fluid network into small reservoirs or cavities under the flexible sheet. May be combined with a touch sensor of any sort, surface bumps define regions and user may press or touch proximately. May provide haptic feedback, also. "Statically haptic"? [CieslaCM12b] Ciesla, Cragi Michael and Yairi, Micah B. "User Interface System", United States Patent 8,154,527, april 10, 2012 Tactile display raises bumps in a flexible touch surface by pumping fluid via a fluid network into small reservoirs or cavities under the flexible sheet. May be combined with a touch sensor of any sort, surface bumps define regions and user may press or touch proximately. May provide haptic feedback, also. [CieslaCM12c] Ciesla, Cragi Michael and Yairi, Micah B. "User Interface System", United States Patent 8,199,124, June 12, 2012 Tactile display raises bumps in a flexible touch surface by pumping fluid via a fluid network into small reservoirs or cavities under the flexible sheet. May be combined with a touch sensor of any sort, surface bumps define regions and user may press or touch proximately. May provide haptic feedback, also. [CliftonTC11a] Clifton, Trevor C.; Taylor, David; Dlishko, Vadim; Woolley, Richard D.; and Carter, Dale J. "Reenable delay of a touchpad or touch screen to prevent erroneous input when typing", United States Patent Application 2011/0316774 A1, December 29, 2011 Filter out unintended input on touchpad or tablet by requiring that stylus (or finger) move a minimum distance and minimum time, in the firmware of the touchpad, instead of in a custom driver (??). Also disable touchpad input when user is typing. Distance may be arc distances / total path length, distance from start, etc. [CodeProject11a] Mukit, Ataul "A very basic alpha blend algorithm", www.codeproject.com, 19 April 2011 Manual implementation of alpha blending / alpha compositing of bitmaps. Useful for seeing how easy it would be to code. [ColgateJE13a] Colgate, James Edward and Peshkin, Michael "Haptic device with controlled traction forces", United States Patent 8,525,778, September 3, 2013 Haptic feedback: create a shear (sidewise) force on a finger by combining oscillation to reduce friction (see ultrasonic feedback), with synchronized lateral (sideways) oscillation of the touchscreen surface. [CopelandL11a] Copeland, Lee "Technically Speaking: Works as Designed", Better Software, May/June 2011, p. 13 (www.stickyminds.com) Editorial on the fallacy of meeting specifications: working to specification does not guarantee reliability for successful function. Example is Japanese atomic reactors in a tsunami. [CorpusE13a] Corpus.e AG "Corpus.E Backgrounder: Body Scanning Technologies", Corpus.e, Stuttgart, Germany, www.corpus-e.com, 2013 (in German) 3D body-scanning using single camera at known position, multiple views of subject. Application is custom fitting of ski boots: shows user with colored socks on special platform with rotating camera and reference pattern. See also structure-through-motion, Chen 2010, Josten 2003. [CortopassiM13a] Cortopassi, Michael and Endejan, Edward "Method and apparatus for using pressure information for improved computer controlled handwriting recognition data entry and user authentication", United States Patent 8,488,885, July 16, 2013 Use pressure information from touchscreen as well as X/Y information for gestures. Increased pressure indicates upper-case, or display attribute (bold). Or combined gestures. Pressure profile in signature verification (reference mistakenly says signature recognition). Has a very long list of citations to patent litigation documents. [Craven-BartleT11a] Craven-Bartle, Thomas and Ericson, Petter "Digital pens and a method for digital recording of information", United States Patent Application 2011/0013001 A1, January 20, 2011 Anoto pattern pen/stylus: optical sensors and light source integregated into writing end of inking pen. Linear polarizers to block ambient light. Used with Anoto optical digitizer paper. [Cypress11a] Cypress Semiconductor "Cypress PSoC Creator Capacitive Sensing (CapSense) Software 2.10", Cypress Semiconductor Corporation, 2011 Software driver documentation for ICs for capacitive touchscreens: include definitions and APIs for multiple virtual devices / widgets (sliders, buttons, etc.) in a touchscreen user interface, including finger and noise thresholds, hysteresis, de-bounce (of buttons). SmartSense tuning for background/manufacturing capacitance, disambiguation of water drops and finger touches (based on contact area size). [Cypress12a] Cypress Semiconductor "Cypress SLIM Single-Layer Independent Multitouch", Cypress Semiconductor Corporation, 2012, touch.cypress.com



Single-layer multi-touch capacitive touchscreens. [DahlT11a] Dahl, Tobias; Vagenes, David; Tuttle, Matthew; and Syversurd, Bjorn Cato "Machine interfaces", United States Patent Application 2011/0254762, October 20, 2011 Sonic/acoustic touchscreen digitizer, press on (stretched) pliable surfaces, sonic transducers spaced below determine location of deformation. Determine force/pressure by amount of deformation. Described as modification of 3D touchless sensing, rather than touchless sensing, that the pliable sheet functions as a point of reference to user without restricting to equivalent of touchpad surface. [DahlT11b] Dahl, Tobias; Birkedal, Geir; and Syversurd, Bjorn Cato "Multi-range Object Location Estimation", United States Patent Application 2011/0103448, May 5, 2011 Sonic/acoustic 3D finger digitizer, detects object (fingertip) by correlating expected impulse response on reflected echo [DahlT12a] Dahl, Tobias; and Syversurd, Bjorn Cato "Touchless Iteration", International Patent Application WO 2012/172322 A2, December 20, 2012 Ultrasonic/acoustic 3D finger digitizer, detects object (fingertip) by correlating expected impulse response on reflected echo. Sends no signal if no input would be appropriate (e.g. no application accepting gestures is waiting for input) [DahlT12b] Dahl, Tobias; and Syversurd, Bjorn Cato "Control of Electronic Devices", International Patent Application WO 2012/017241 A1, February 9, 2012 Turn on a cellphone or other device when a finger/hand comes close to a touch/proximity sensor. [DahlT12c] Dahl, Tobias "Control Using Movements", International Patent Application WO 2012/0206399 A1, August 16, 2012 Identify in air gestures by proximity. [DahlT13a] Dahl, Tobias and Bang, Hans Joergen "User Interfaces", International Patent Application WO 2013/0050080 A1, February 28, 2013 send control signals between two devices using ultrasonic/acoustic signals [DangCT11a] Dang, Chi Tai and Andre, Elisabeth "Usage and Recognition of Finger Orientation for Multi-Touch Tabletop Interaction", Proc. INTERACT 2011, 13th Intl. Conf. On Human-Computer Interaction, vol 3 pp. 409-426 Optical multi-touch using infrared camera (below surface?) on tabletop, determine finger orientation and (pointing) direction from outline of images. Finger angle used to determine better accurate pointing location. Grasping gesture, two-hand (not just two-finger zoom/pinch). [DeCaroRE12a] DeCaro, Robert Edward; Bardyn, Jean-Paul; and Nys, Olivier "Use of Resistive Touch Screen as a Proximity Sensor", United States Patent Application 2012/0218210 A, August 30, 2012 Resistive sheet touchscreen, also do proximity detection (without position information) by using same top sheet as large capacitive sensor and measure change in capacitance to ground. [DeNardiA11a] D. Denardi "grafiti: Gesture Recognition mAnangement Framework for Interactive Tabletop Interfaces: C# Code Project", code.goodle.com, 2011 GUI/User Interface for multi-touch, runs on top of Tuio client, pluggable third-party gesture recognition modules [DempseyJW12a] Dempsey, James W. "Cross process accessibility", United States Patent Application 2012/0331411, December 27, 2012 Accessibility techniques for graphical and touchscreen GUI user interfaces. Accessibility clients registered to received UI events, access UI data structures for multiple applications. Compare with JAWS? [DesourdisRI11a] Desourdis, Robert I. Jr; Vest, Kevin F.; O'Brien, Mark; and Mulholland, David J. "Digital Television for Homeland Security: Broadband Datacast for Situational Awareness and Command Coordination", Proc. 2011 IEEE Int'l Conf on Technologies for Homeland Security (HST), 15-17 Nov. 2011, pp. 32-42 Datacasting full-bandwidth to all receivers, unlike IP protocols: mentions IP video, geospatial visualization (maps), data files, text messages, any digital media. [Disheng12a] Disheng "Chinese Handwriting Tablet USB Input XP Vista Windows 7", eBay.com, fetched 2012 Chinese handwriting tablet with speech output software, dirt cheap (less than $15). 23,000 character of both simplified and traditional Chinese, Hong Kong, and English. Mandarin speech synthesis. Windows XP/Vista/Windows 7. [Disney13a] Disney Researchi/Thomas Anderson "Disney develops algorithm for rendering 3D tactile features on touch surfaces", Press article: www.tgdaily.com, October 8, 2013 Touchscreen tactile feedback by varying the amount of friction on a finger by electrostatic modulation, gives impression of 3D profile of surface. [Displax13a] Displax, Portugal "Multitouch technology: Displax - Interactive Systems - Projected Capacitive multitouch foil - through-glass multitouch", Displax.com, fetched 2013 multi-touch capacitive sensor fabricated from thin foil, can be applied to "any surface". Says projected capacitance touchscreen. Mounted on rear of glass, so says impervious to environment (because it is behind the glass?), less ("no") visual parallax. Refers to palm/arm rejection by recognize and reject large



area of contact, skin multitouch to sense levels of pressure (contact area of finger) electronic file contains additional materials. [DoeppnerTW11a] Doeppner, Thomas W. "Operating Sytems in Depth", John Wiley and Sons, 2011 Upper-level course book on operating systems design, based on Unix (Windows NT was too much code). History notes: MacIntosh OS no VM, multitasking until new version in 2000. Architectural transsition on Windows 95. Multiple definitions of "kernel". [DomingosP12a] Domingos, Pedro "A Few Useful Things To Know about Machine Learning", CACM, October 2012, Vol 55 No 1, pp. 78-87 Machine learning (e.g. neural nets, etc.): article on the "black art" of adaptive classifiers/recognition known in successful projects, but not in the literature. Compare with Tappert articles on handwriting recognition State of the Art? Keep training and test data separate to avoid contamination of generalization by overfitting. Importance of very large training sets (dumb algorithm with much training wins over fancy algorithm with less training). Classifiers learn correlation, not causation. [DunkeldB13a] Dunkeld, Bryan and Lambert, Edward "Method of providing a digital asset for distribution", United States Patent 8,583,556, November 12, 2013 DRM usage rights management. Digital asset has serial number / ID, multiple copies on distribution servers, track the various instantiations/copies by embedding unique user ID and updating data base: documents/assets encrypted per user [EiblM11a] Eibl, Maximilian and Ritter, Marc "Übermedien Übermorgen: Workshopband Mensch und Computer 2011", Workshop processing der Tagung Mensch und Computer 2011, Technische Universität Chemnitz, Germany, August 2011 (in German) (in German and English) Workshop proceedings and papers for tracks Entertainment Interfaces and Mensch und Computer (Human and Computer) includes Photosurface: Multitouch light-table for professional Photographers, Combining Active and Passive Haptic Feedback for Rich Stylus Interactions, Design decisions for tangible objects, Collaborative work in interactive displays, Skintimacy: electronicized skin as music input device, Psychoacoustic audiovision, Haptic Drum Toolkit [ElectronicDesign13a] DeCanne, Bart "Trends in Capacitive Touch Panels", Electronic Design Europe, February 7, 2013: available at electronicdesign.com Trend in ITO capacitive touchscreens to be integrated with LCD display, rather than separate layer. Shows a diamond pattern of conductors in two layer grid/matrix (Compare with Rympalski?). Mentions Cypress SLIM (single-layer independent multitouch) construction without bridges (required finer grid/matrix) [EleftheriouK13a] Eleftheriou, Kostas "Data input system and method for a touch sensor input", United States Patent Application 2013/0021248, January 24, 2013 touchscreen keyboard: find closest dictionary match to word by pattern of logarithmic polar coordinates. For small touchscreen virtual keyboards. [EleftheriouK13b] Eleftheriou, Kostas "User Interface for Text INput", United States Patent Application 2013/0212515, August 15, 2013 touchscreen virtual keyboard: dictionary match to typed words, optical/camera input as touchscreen [EliasJG11a] Elias, John Greer; Westerman, Wayne Carl; and Orr, James Edmund IV "Method of increasing the spatial resolution of touch sensitive devices", United States Patent 7,932,897, April 26, 2011 Circular/radial touchpad sensor. reports radial position. Visually similar to wheel control on small iPod music players. Electrodes are wired in two sets of groups to minimize electronics. Very long list of citations. [EllipticLabs11a] Elliptic Labs "Elliptic Labs: Touchless Acoustic Gesture Demo video", EllipticLabs.com, fetched from youtube.com Ultrasound position sensor with software for recognizing in-air gestures. Ultrasound touchless gesture technology. Demonstration includes hand gestures wave up, down, left, right (compare with flick gestures for scrolling and page turning in PenPoint); variable for farther/faster scrolling; in/out/play pause: hold hand stationary as if "stop", move pointing finger towards then away from display. Appears to have two transducers above display on top corners. [EllipticLabs13a] Elliptic Labs "Elliptic Labs: Windows 8 Gesture Suite", EllipticLabs.com, fetched 2013 Ultrasound position sensor with software for recognizing in-air gestures: gestures correspond approximate to Windows 8 touch gestures: square/rectangle to select Windows 8 tile; wave on right for "charms menu"; wave on left to cycle running applications; wave above for application menu; wave left or right for scroll; downward wave for close. Generals 40Khz signal using transducer, detect with 96KHz internal microphone. [EloTouchsystems12a] Elo Touch Solutions/ Elographics "Elo Touch Solutions: How AccuTouch Technology Works", EloTouch.com.com, fetched 2012 Collection of materials on Elo Touch Solutions accutouch resistive film touchscreens with spacer dots; acoustic pulse recognition touchscreens (for large displays), CarrolTouch infrared touchscreens, surface wave acoustic SAW. Talsk about IntelliTouch Plus SAW multi-touch technology, but is really only dual



touch. Projected capacitive multitouch- really dual touch. [ExpertGlossary12a] ExpertGlossary.com "Input Focus", ExpertGlossary.com, fetched 2012 Definition: Input Focus [EyeSight13a] EyeSight Technology "Eyesight: A Touch-free User Interface -- About the Technology", eyesighttech.com, 2013 Recognition of hand and finger gestures using finger tip tracking with off-the-shelf laptop camera [FindlaterL12a] Findlater, Leah; Lee, Ben Q.; and Wobbrock, Jacob O. "Beyond QWERTY: Augmenting TouchScreen Keyboards with Multi-Touch Gestures for Non-Alphanumeric Input", Proc. CHI 2012, May 5-10, 2012, Austin Texas, pp. 2679-2682 Study of combining touchscreen on-screen keyboard with (mostly) unistroke/single-touch gesture/handwriting gestures.. Punctuation particularly difficult on touchscreen keyboards -- includes @sign, URLs, passwords with punctuation characters. [FindlaterL12b] Findlater, Leah and Wobbrock, Jacob O. "Personalized Input: Improving Ten-Finger Touchscreen Typing through Automatic Adaptation", Proc. CHI 2012, May 5-10, 2012, Austin Texas, pp. 815-824 On ten-finger multitouch touchscreen virtual keyboard, adjust virtual position of keys for sensing to the locations the user actual hits when trying to hit the keys [FoleyMJ11a] Foley, Mary Jo "Microsoft makes OneNote free on the iPhone", zdnet.com, January 18, 2011 Microsoft OneNote note-taking application available on iPhone (iPad?) from Microsoft. Allows collaborative sharing of notes on Windows PC, Window Hpone 7, and iPhones. [FooTW13a] Foo, Teck Wee "Obtaining Keywords for Searching", United States Patent Application 2013/0124551, May 16, 2013 Indexing of videos: recognize an object/image/person in the video, attach a keyword to that point in the video, video can be searched by that keyword. Example: recognize actors in a film index by their name (as the keyword) for all scenes each particular actor is in, when actor appears, can search Wikipedia for information about that actor. Cites to this bibliography (rwservices) for Tappert "State of the Art in Handwriting Recognition". [ForstallS11a] Forstall, Scott; Christie, Greg; Lemay, Stephen O.; Ording, Bas; and Van Os, Marcel "Deletion Gestures on a Portable Multifunction Device", United States Patent 7,934,156, April 26, 2011 Select items with a touchscreen gesture (tap, swipe, etc.), then delete with a gesture. Cites to Bricklin. Compare with PenPoint, Pen Windows? [FuccellaV13a] Fuccella, Vittorio; Isokoski, Poika; and Martin, Benoit "Gestures and Widgets: Performance in Text Editing on Multi-Touch Capable Mobile Devices", Proc. CHI 2013, Paris, pp. 2785-2794 Gestures for editing text work somewhat better than icon/widget based commands for editing text, on multi-touch smartphone. Citations to other gesture-based editing, including gestures on soft keyboard buttons (gestureButtons). compare with caret, insert, up down, cut, copy, paste gestures of PenPoint (not cited). None of studied gestures are actually multi-touch, in contract to title. Few details on recognizer used. [FujiokaA13a] Fujioka, Akizumi; Gotoh, Toshimitsu; Kubota, Akinori; and Uehata, Masaki "Display Device having Optical Sensors", United States Patent 8,416,227, April 9, 2013 LCD touchscreen using infra-red optical sensors for each LCD pixel and visible plus infra-red backlight. Thus no intermediate ITO layer etc. on LCD panel. Shows finger touch images for multi-touch detection. Note: Sharp "In-Pixel"/"In-Cell" sales literature says may be used to scan fingerprints, or documents laid on touch panel, since it is optical. [GargentaM11a] Gargenta, Marko "Learning Android", O'Reilly Publishers, 2011 Development tutorial for Android operating system: tap, long-press (press-and-hold). [GeneralDigital13a] General Digital Optical Bonding Laboratories "Optical Bonding of Flat Panel Displays", www.gdoptilabs.com, fetched 2013 Product and technical information on optical bonding products: liquid adhesive with index of refraction to match top and bottom transparent material. Touchscreens integrated with display. See also optical bonding on Wang "Guide". [GibbsWW13a] Gibbs, W. Wayt "How to Brew Your Own Conductive Ink", IEEE Spectrum, Dec. 25, 2013 Home formulation of conductive ink for drawing functional circuit wiring: uses gallium and iridium. Based on articles by Zheng Yi of Beijing Key Lab of Cryogenics and group at Georgia Tech [GiraudF12a] Giraud, Frederic; Amberg, Michel; Lemaire-Semail, Betty; and Casiez, Gery "Design of a transparent tactile stimulator", Proc. IEEE Haptics Symposium 2012, March 2012, Vancouver, pp. 485-489 Haptic feedback by friction on touchscreen. Touch sensing using force sensors supporting the four corners of the display. Haptic feedback by varying perceived friction by vibrating whole display. [Github12a] Github.com open-source repository "GestureDetector.java", https://github.com/libgdx/libgdx/blob/master/gdx/src/com/badlogic/gdx/input/GestureDetector.java, version of 2012



class detector source code (open-source) for tap, double-tap, drag, presshold, and other simple gestures. Compare with Android, PenPoint? [GoertzM11a] Goertz, Magnus George "Touch Screen for Mobile Telephone", United States Patent 7,880,732, Feb 1, 2011 (filed 2002) Optical touch-screen digitizer, using LED emitters on edge with reflective mirror to direct across/above surface of hand-held display. See also Neonode file. [GoertzM11b] Goertz, Magnus; Eriksson, Thomas; Shain, Joseph; Jannson, anders; Kvist, Nikls; Pettersson, Robert; Sparf, Lars; and Karlsson, John "Pressure-sensitive Touch Screen", United States Patent Application 2011/0181552, July 28, 2011 Optical touch-screen with pressure (force) sensing, using LED emitters. Light emitters and receivers are behind transparent touchscreen digitizer, which is in a housing that permits screen to be depressed. [GoertzM11c] Goertz, Magnus; Eriksson, Thomas; Shain, Joseph; Jansson, Anders; Kvist, Niklas; Pettersson, Robert; Sparf, Lars; and Karlsson, John "Method of determining a touch location on a touch screen", United States Patent Application 2011/0169780, July 14, 2011 Optical touchscreen using infra-red LED diodes and light detectors. Using lenses or mirrors, light from LED shines in a (2-D) cone to two or more detector, each detector detects light from two or more LEDs. The two LEDs or detectors may be side to side, or one above the other (or may effectively be so using a lowangled mirror) to detect at multiple heights above touchscreen. [GoertzM13a] Goertz, Magnus George "Light-based Touch Screen using Elliptical and Parabolic Reflectors", United States Patent 8,587,562, November 19, 2013 Optical touch-screen digitizer, using LED emitters on edge with reflective mirror to direct across/above surface of hand-held display. See also Neonode file. Multitouch - on two diagonal points (ambiguous), suggests assigning same meaning to both points, or disambiguate by amount of ambient light blocked by feature being nearer to sensor. Does not address more than two multitouch point. [Google13a] Google, Inc. "Nexus 7 Guidebook for Android mobile technology platform 4.2.2", Google, Inc. Copyright 2012, 2013, Edition 1.2.2.12 Touchscreen tablet device from Google. Describes TalkBack accessibility screen reader, ExploreByTouch to navigate touchscreen via spoken feedback, magnification gestures for low-vision users. [GrahamRoweD11a] Graham-Rowe, Duncan "Give Your Dashboard the Finger: an in-car interface helps drivers keep their hands on the wheel", Technology Review, February 2, 2011: also available at www.technologyreview.com Press article one multi-modal user interfaces including finger and hand gestures using proximity-sensing touch digitizer and visual at Deutsches Forschungszentrum für Künstliche Intelligenz, Christian Mueller. Dubbed Geremin, reference to Theremin proximity sensor. Gestures are left, right, circle clockwise, triangles in air. Additional materials in file: "DFKI entwickelt Gestensteuerung für das Auto". Compare with BreedDS06a. See also "Geremin: 2D Microgestures for Drivers Based on Electric Field Sensing" [GrayRO12a] Gray, R. O'Neal "Apparatus and method for interactive display with tactile feedback", United States Patent 8,115,745, February 14, 2012 Display a keypad image for user input when in an interactive mode (software expects input), and not display keypad image otherwise, combined with a physical keypad over the display that gives tactile feedback (vibration?) for the locations of the keys being displayed. [GrayT13a] Gray, Tony "Touch panel design considerations and potential trade-offs", Electronic Products News, April 16, 2013 Commercial review (Ocular, ouclarlcd.com) concerning physical and construction trade-off in LCD touchscreens (capacitive). [GrivnaEL11a] Grivna, Edward L. "Multi-axial touch-sensor device with multi-touch resolution", United States Patent 8,072,429, December 6, 2011 Three-axis capacitive touchpad/touchscreen sensor instead of two-axis matrix. Claims that full NxN matrix requires NxN traces, one to each pad, three-axis (120 degrees) has no ghost or ambiguous points if number of multi-touches reported is < 4. To construct in two layers, requires that two axes be connected by jumpers or feed-throughs. Assigned to Cypress Semiconductor. [GrosC12a] Gros, Claudius "Pushing the Complexity Barrier: Diminishing Returns in the Sciences", Complex Systems Vol 21, 2012. Essay on diminishing returns in scientific analysis: systems being studied are sufficiently complex that they are mathematically chaotis e.g. weather/climate modeling. [GrothA13a] Gruzman, David and Ben-Itzak, Yuval "Interacting with a Three-dimensional Object Dataset", WIPO International Patent Application Publication WO 2013/068860 A1, May 16, 2013 Make an anatomical model (e.g. of a patient's heart), make the surface a touchsurface, map touches on the touchscreen surface to the 3D position on the model. Application is surgeon inputting anatomical information (cardiologist report, etc.) Compare with Polhemus?



[GruzmanD12a] Gruzman, David and Ben-Itzak, Yuval "System and Method for Inspecting Dynamically Generated Executable Code", United States Patent 8,141,154, March 20, 2012 Virus/security scanning of JavaScript code modules from internet: intercept download, copy to a security servier, security server returns status about whether appears safe to execute. Because checking is done on server, checking code need not be made available for installation (McAffee, etc.) which also makes it available for attackers to reverse-engineer. Refers to behavioral (versus signature-based) checking as proactive checking. Part of larger system that does behavioral analysis for detecting malware. [GuimbretiereF12a] Guimbretiere, Francois and Nguyen, Chau "Bimanual Marking Menu for Near Surface Interactions", Proc. CHI 2012, May 5-10, 2012, Austin TX, pp 825-828 Optical camera tablet, multi-touch bi-manual gestures using different pinches of different fingers. Uses Kinect (Microsoft) sensor for gesture hand, stylus/tablet for drawing hand. [HagerH11a] Hager, Harald and Burku, Sascha "A historical overview of tablet computing, GUIs and hypertext", Seminararbeit Software Research Center, Univ. of Salzburg, Austria, 2011, available at http://www.softwareresearch.net/fileadmin/src/docs/teaching/SS11/SaI/seminararbeit_burku_hager.pdf Yet another history of pen / tablet computing, going back to NLS, Sage and Sketchpad. Has high-quality color photographs of Rand Tablet, Palm Pilot, Memex, SAGE light gun, Sutherlund Sketchpad, Dynabook, GridPad. PenPoint absent. [HakenAudio13a] Eagan, Edmund and Haken, Lippold "Continuum Fingerboard User's Guide, firmware version 5.50", Haken Audio, March 4, 2013: HakenAudio.com Multi-touch fingerboard/keyboard, up to 16 fingers. Touch areas arranged like white and black keys of a piano keyboard, player/user can interact with tones and notes using position on key and pressure (force) of keyboard: default mapping is Y position to brightness, pressure Z to Midi Expression parameter. Separate PC/Mac software with graphical user interface for sound designer and mixer. [HallidayD11a] Halliday, David; Resnick, Robert; and Walker, Jearl "Fundamentals of Physics, 9th Edition", John Wiley and Sons, Inc., 2011 First year college physics reference [HamblinMA11a] Hamblin, Mark Arthur and Hotelling, Steve Porter "Double-sided touch sensitive panel and flex circuit bonding", United States Patent 8,026,903, September 27, 2011 Matrix grid touchscreen with rows on one side, columns on other side of substrate, all connections can thus be brought out to one side of substrate. [HanJY12a] Han, Jefferson Y. "Multi-touch sensing through frustrated total internal reflection", United States Patent 8,259,240, September 4, 2012 Optical multi-touch touchscreen surface using internal reflection. See also Han 2005 demo, 8,144,271. [HarrisonC11a] Harrison, Chris; Benko, Hrvoje; Wilson, Andrew D. "OmniTouch: Wearable Multitouch Interaction Everywhere", Proc. UIST '11, October 16-19, 2011 Multi-touch sensor on any surface: Project image on surface and recognize touches via video. Prototype projects image on hand, wall, or other surface from wearable computer. User can drag projected menu or UI to desired location, rather than automatic location. [HarrisonC11b] Harrison, Chris; Schwarz, Julia; and Hudson, Scott E. "TapSense: Enhancing Finger Interaction on Touch Surfaces", Proc. UIST '11, October 16-19, 2011, pp 627-634 Tap Sense: recognize type of tap (finger, knuckle, knock, pen, etc.) acoustically, used to give multiple gestures for tap input. Most reliable is discrimination between finger input and stylus. Has anatomical figure of human finger. Tap-and-hold gesture: press-and-hold Electronic file has additional material [HarrisonC11c] Harrison, Chris; Schwarz, Julia; Hudson, Scott E.; and Kafazis, Antonios "TapSense: Enhancing Finger Interaction on Touch Surfaces", Proc. UIST '11, October 16-19, 2011, pp 627-634 Demonstration of paper same name. Cites to Skinput. [HarrisonC11d] Harrison, Chris "TapSense Lets You "Right-Click" On Touchscreens", www.fastcodesign.com, 2011 Video demonstration of TapSense [HarrisonC12a] Harrison, Chris; Sato, Munehiko; and Poupyrev, Ivan "Capacitive Fingerprinting: Exploring User Differentiation by Sensing Electrical Properties of the Human Body", Proc. UIST '12, October 7-10, 2012, Cambridge, Mass. pp. 537-543 On multi-touch capacitive touchscreen, identify/differentiate simultaneous users (at least for same fingers) using swept frequency capacitive sensing (SWCS) profile. See also Touche/Disney Research, same year. [HarrisonC12b] Harrison, Chris and Hudson, Scott E. "Using shear as a supplemental two-dimensional input channel for rich touchscreen interaction", Proc. CHI 2012, May 50-10, 2012, Austin TX, pp. 3149-3152 Review of using shear sensing (sideways force) on touchscreens: extended flick gestures, single-touch shape gestures by shear only, pie/gesture menus. Can touch to select, then perform operation by shear gestures, rather than separate pointing and gesture interactions. Cites to Minsky 1984. [HarrisonC13a] Harrison, Chris "ChrisHarrison.com: Research", chrisharrison.com, 2013 Web postings by Chris harrison, mostly on touchscreen/touchsurface gesture techniques and sensors.



[HarrisonP11a] Harrison, Phil "Hand-Held Device with Touchscreen and Digital Tactile Pixels", United States Patent 8,004,501, August 23, 2011 Touchscreen with actuator-dot tactile pixels on frame around the display. Compare with Optacon, Braille display. [HayesJJ13a] Hayes, John Joseph Jr. and Thornson, Curtis Waynes "Radiotelephones having contact-sensitive user interfaces and methods operating same", United States Patent RE43931E, January 15, 2013 Re-issue of 6,278,888 (see file) [HenriksenB12a] Henriksen, Benjamin; Christensen, Jesper Munch; and Stumpe, Jonas "The Evolution of CERN's Capacitive Touchscreen: From control room to mainstream", IT-University of Copenhagen, summer 2012 Historical paper with interviews on invention of capacitive touchscreen at CERN, 1974, as case-study in (lack of) technology transfer. Touchscreens not commercialized: because of lack of ownership by major companies. [HeoS11a] Heo, Seongkook and KLee, Geehyuk "Force Gestures: Augmenting Touch Screen Gestures with Normal and Tangential Forces", Proc. UIST '11, October 16-19, 2011, pp. 621-626 Touchscreen sense normal and tangential force, used for force gestures. Prototype encases iPod Touch touchscreen in sensing frame. Cites to Minsky. [HeubelRW12a] Heubel, Robert W. and Grant, Danny A. "Multi-touch Device Having Dynamic Haptic Effects", United States Patent 8,098,235, January 17, 2012 Sense more than one touch and generate a haptic effect, including generating the effect only at the points of touch. Compare with optacon devices for accessibility? [HewlettPackard11a] Worthen, Ben "H-P Joins Tablet Scramble", Wall Street Journal, February 10, 2001, page B5 Hewlett Packard announces tablets/smartphones using Palm OS (acquired) in response to iPad and Google Android. App store availability noted as critical business aspect. [HigginsSE11a] Higgins, Steve E.; Mercier, Emma; Burd, Liz; and Hatch, Andrew "Multi-touch tables and the relationship with collaborative classroom pedagogies: A synthetic review", Int'l. Soc. of the Learning Sciences, Computer-Supporte Collaborative Learning, pre-print (accepted 2011) multi-touch tabletop devices in classroom for education: benefits often not equal to cost, not always clear of benefits over regular hands-on with materials. [HinckleyK11a] Hinckley, Ken and Song, Hyunyoung "Sensor Synaesthesia: Touch in Motion, and Motion in Touch", Proc. CHI 2011, May 7..12, 2011, Vancouver, Canada, pp. 801-810 Touch+motion gestures (touch-enhanced motion, motion-enhanced touch) using multi-touch touchscreen and accelerometer of smartpone: e.g. press + tilt is zoom, soft vs. hard touches, gentle swipe vs. drag with hard onset, press on object and shake to undo, touch with two fingers and tilt to selected, etc. File contains video demonstration. [HinckleyK12a] Hinckley, Kennneth P. "Method and apparatus using multiple sensors in a device with a display", United States Patent 8,120,625, February 21, 2012 Proximity/tilt sensor on smartphone/PDA to determine how user is holding device, tilt sensor to rotate display automatically, tilt scrolling. Cites to Rekimoto tilt operations on PDA. [HollemansGG13a] Hollemans, Gerrit G.; Kleinhout, Hulb V.; Hoonhout, Heneriette C.M.; van de Wijdeven, Sander B.B.; and Bull, Vincentius P. "Use of a Two Finger Input on Touch Screen", United States Patent 8,466,893, June 18, 2013 Electrostatic tablet, two-finger zoom gesture using proximity: works with crosspoint ambiguity. Compare with Microsof hover gesture? [HolzC11a] Holz, Christian and Baudisch, Patrick "Understanding Touch", Proc. CHI 2011, May 7..12 2011, Vancouver BC, Canada Study of intrinsic finger touch accuracy, inherent offsts in general perceived general input model (using fingerprint sensing for greater touch accuracy). When a user touches to point to something (target on a tablet or touchscreen), visual parallax, finger posture (angle, may be rolled/rotated slightly), user targets "bright spot" on fingernail which is offset from center of touch area or pressure center of touch contact. Mentions capacitive and FTIR multi-touch. Cites to Wellner and Fingerworks, not to Rekimoto. [HolzC13a] Holz, Christian and Baudisch, Patrick "Fiberio: A Touchscreen that Senses Fingerprints", Proc. UIST '13, October 2013, St. Andrews, UK Optical touchscreen with fiber-optic plate (see Wang project) to that rear-projected images are brought to the top surface, so no parallax, no positional error. Can also detect hovering objects (reflected light), outlines and shapes of objects placed on surface (fiducial markers). Fingerprint verification as biometric authentication. [HolzD13a] Holz, David "Motion capture using cross-sections of an object", United States Patent Application 2013/0182079 A1, July 18, 2013 "Non-contact" touchscreen takes multiple sequential images of user's hands (single camera), construct 3D model of shape and of position of hand/fingertips. Essentially a motion capture system using structure-



through-motion. Compare with ChenJ10a. [HolzD13b] Holz, David "Systems and methods for capturing motion in three-dimensional space", United States Patent Application 2013/0182897 A1, July 18, 2013 "Non-contact" touchscreen takes multiple sequential images of user's hands (single camera), construct 3D model of shape and of position of hand/fingertips. Essentially a motion capture system using structurethrough-motion. Refers to use of shadows and reflections in the image. See also file on "Leap Motion". Compare with ChenJ10a. [HolzD13c] Holz, David "Enhanced contrast for object detection and characterization by optical imaging", United States Patent Application 2013/0182007 A1, July 18, 2013 "Non-contact" touchscreen takes multiple sequential images of user's hands (single camera), construct 3D model of shape and of position of hand/fingertips. Essentially a motion capture system using structurethrough-motion. Infrared light near camera, use fall-off/decrease in illumination with distance as a cue for distance. See also file on "Leap Motion". Compare with ChenJ10a. [HornBL12a] Horn, Bruce L. "Computer System for Automatic Organization, Indexing and Viewing of Information from Multiple Sources", United States Patent 8,280,932, October 2, 2012 Displaying lists or tables of objects (e.g. .JPGs, .MPG3, etc.) automatically sorted by different attributes, and sub-trees (e.g. folders) can be expanded or contracted in the list. Refers to sticky paths for references to objects. [HoskingRH13a] Hosking, Roger H. "Software Defined Radio Handbook, 10th ed.", Pentek, Inc. 2013 General introduction to software defined radio SDR: Nyquist theorem relates to signal bandwidth, not signal frequency: undersampling with band-pass filter for receiving signals using high frequency, frequency reversal, sampling frequency decimation (1 of N samples) [HotellingS11a] Hotelling, Steve Porter and Land, Brian Richards "Double-sided Touch-sensitive Panel with Shield and Drive Combined Layer", United States Patent 7,920,129, April 5, 2011 Electrostatic (capacitive) multi-touch touchpanel digitizer: X and Y traces on opposite sides of substrate, sensing at locations where they cross. Fingerworks? Cites to Rubine '91 thesis for multitouch. [HotellingS13a] Hotelling, Steve; Strickon, Joshua A.; Huppi, Brian Q.; Chaudhri, Imran; Christie, Greg; Ording, Bas; Kerr, Duncan Robert; and Ive, Jonathan P. "Gestures for touch sensitive input devices", United States Patent 8,479,122, July 2, 2013 Multipoint/ multi-touch gestures in user interface, combined with targeting to GUI objects to determine which is targetted. Compare with ink targetting in Slate, Pencept, elsewhere. Extremely long citation list. [HotellingS13b] Hotelling, Steven Porter; Yousepfor, Marduke; chang, Shih change; and Zhong, John Z. "Integrated Touch Screen", United States Patent 8,363,027, January 29, 2013 In-cell touchscreen and display: capacitive elements used both as electrodes/capacitors for LCD, and as capacitive touch sensors. Cites to Westerman thesis for capacitive/electrostatic multi-touch tablet. [HuaL11a] Hua, Liu and XiaoPing, Jiang "Capacitance Sensing Matrix for Keyboard Architecture", United States Patent 8,059,015, November 15, 2011 Electrostatic/capacitive touch sensing / conductive objects: interpolate/extrapolate positions of keyboard keys. Compare with Rekimoto? [HuangYP12a] Huang, Yi-Pai "Auto-stereoscopic 3D Display and Its Future Developments", SID Webinar, May 11, 2012 Seminar on autostereoscopic displays, emphasizing visual convergence (two eyes) and accommodation (single eye focusing). Stereoscopic is with glasses (shutter, polarization, anaglyph -- dichroic red/blue, HMD), auto-stereoscopie without glasses (holography, volumetric, multi-plane, multi-plexed 2D (spatial/time/tracking), electrically-variable-refractive-index lenses Final section on 3D displays an interactivity with touchscreen/3D-gesture input. Thin-Sight touchscreen, shadow-based tracking (compare: Wellner), computer vision (Touch-Light). [HuangYP12b] Huang, Yi-Pai "Advance Display Optics Lab, Lecture 6: Other Display Applications", Tutorial/seminar: 2012 Display Optics conference Tutorial on electronic paper, ferroelectric, Cholesteric, Nematic LCD display; ferrochromic, electrochromic films; electrode-wetting and other displays, E-Ink and other flexible displays. Microsoft Sphere and second-light prototypes. Additional section on interactive touchscreen and other technologies: Microsoft IR retro-reflective emitter/sensor, shadow-based tracking, touchlight binocular 3D sensor; rear-camera behind diffuser, touch sensed by image being sharp rather than blurred by diffuser. Special section on Samsung Sensible Backlight (see 2010 paper). [HuangYP13a] Huang, Yi-Pai; Wang, Guo-Zhen; Tian-Sheuan, Chang; and Chen, Tsu-Han "Three dimensional virtual touch display system for multi-user applications", To appear in IEEE Journal of Display Technology, 2013 Non-contact 3D "touchscreen" using multiple infrared lightpens and optical sensors integrated in-cell into LCD display. Uses dispersion size and intensity of light on LCD to determine Z. Cites to Brown 2010 paper. [HuotS11a] Huot, Stephane; Chapuis, Olivier; and Dragicevic, Pierre "TorusDesktop: Pointing via the Backdoor is



Sometimes Shorter", Proc. CHI 2011, May 7-12, 2001, Vancouver, Canada, pp 829-838 Mouse desktop navigation using desktop as torus / cursor wrapping, exit on left enters on right, etc. Faster navigation when moving to target across the screen. Works better with dead zone / buffer at edges, for ease of pointing at edges. [HurterC12a] Hurter, Christophe; Lesbordes, Remi; Letondal, Catherine; Vinot, Jean-Luc; and Conversy, Stephane "Strip'TIC: Exploring Augmented Paper Strips For Air Traffic Controllers", Proc. AVI '12, May 21-25, 2012, Capri Island, Italy printed pattern (Anoto) added to paper strips used in air traffic control stations: notes that purelyelectronic systems are too rigid, ATC controllers at times to break procedure to hand off traffic to another controller - by handing them the slip of paper used to record an aircraft. Compare with articles on affordances of physical paper. [IFixit.com12a] www.ifixit.com "iPhone 5 Teardown", www.ifixit.com, fetched september 2012 Disassembly of iPhone 5, does not show separate touchscreen [IFixit.com12b] www.ifixit.com "Nexus 7 Teardown", http://www.ifixit.com/Teardown/Nexus+7+Teardown/9623/1 Teardown/disassembly of Google Nexus 7 touchscreen tablet/e-book reader: states more repairable than iPhone, because held together with screws. Touchscreen integral with display, not separate. [IGesture12a] iGesture.org "iGesture: A General Gesture Recognition Framework", www.igesture,org, fetched Sep. 2012 Trainable gesture-recognition software in Java. Uses Rubine gesture algorithm (which see). Open-source project from 2007 [IMG12a] IMG Presents / IMG Real World Press Innovation Management Group Inc. "OnScreen: for Windows: OnScreen Keyboards with Assistive Technology: Version 1.79: Users Guide", Immersion Corporation, 2011 Electronic file includes additional information. On-screen adaptive keyboard for touchscreens and accessibility for Microsoft Windows. Original version 1993. Includes heads-up display (semi-transparent keyboard image), auto word completion, additional features. [IRIS12a] I.R.I.S. "IRISNotes Express 2 Quick Start Guide / IRISNotes for Mac 1.0 Quick User Guide / IRISNotes Executive 1.0 Quick Start Guide", I.R.I.S., 2012 Sonic (?) digitizer stylus, records data in stylus (compare Anoto), transfer via USB to PC for handwriting recognition application as JPG or electronic ink (no annotation). Easy Note Taker / MyScript Notes Lite (separate apps). Stylus also use in mouse mode (but absolute positioning?), tap-and-hold / press-and-hold for right mouse button. Direct input into Micorosft Office (scanned/OCR input only via clipboard buffer?). [IlmonenT11a] Ilmonen, Tommi "Interactive Display Device with Infrared Capture Unit", United States Patent Application 2011/0227876 A1, September 22, 2011 Optical in-cell multi-touch touchscreen (MultiTaction). Infra-red camera behind projection screen, infrared illumination in screen, camera detects reflected infrared light from objects on or near the translucent / semi-transparent screen. Infrared-absorbing reflection inhibitor (e.g. sooty surface) behind camera to absorb stray light and reflections. [IlmonenT12a] Ilmonen, Tommi "Interactive Display Device", United States Patent Application 2011/0227876 A1, September 22, 2011 Optical in-cell multi-touch touchscreen (MultiTaction). Display also illuminates with infrared light, and captures the reflected infrared. Diffusion layer in display placed on rear (rather than front), so that is spreads out reflected light on the back side to reduce ambient internal reflections that would mess up the imaging. In infrared images (from rear), contact areas show up as shadows. Infrared is alternated at different illumination, variance in shadows to distinguish proximity and touching input. [IlmonenT12b] Ilmonen, Tommi "Automatic orientation of items on a touch screen display utilizing hand direction", United States Patent Application 2012/0060127 A1, March 8 2012 Multi-touch tabletop surface, automatically orient graphical objects (text documents, etc.) to the right position for the user based on detecting the position and orientation of the hands (from the 5 fingertips). [IlmonenT13a] Ilmonen, Tommi "Correction of Touch Screen Camera Geometry", International PCT Patent Application WO 2013/011188A1, January 24, 2012 Multi-touch rear-projection touchscreen using infrared reflection. User touches targets on projected image, system calculates correction tables for input coordinates -- handles net distortion of projected image and sensed positions. Compare with 1980's tablet corrections. [Immersion11a] Immersion Corporation "Adding haptic effects to your Android application", Immersion Corporation, 2011 Tutorial on MOTIV SDK Development Platform from Immersion.com for programming haptic effects on Android UI. Haptic effects include textures, explosions, engines, weapons, alerts, bounces, impacts on different materials (wood rubber meal). SDK supports multiple Android devices with different haptic hardware. UHL Universal Haptic Layer. [Immersion12a] Immersion Corporation "Immersion Case Study The Haptic Workstation", Immersion Corporation,



2012

FasTrak virtual reality simulator: seat, with head motion/position sensor (Polhemus 3D electromagnetic digitizer), eyeglass displays, mechanical force-feedback gloves for haptic touch feedback. [InformationWeek11a] Information Week (Gina Smith) "Slideshow: Verizon iPhone 4 Teardown", www.informationweek.com, Feb. 7 2011 Disassembly of iPhone 4, Verizon CDMA version (versus AT&T GSM): digitizer/touchscreen overlay. [InformationWeek12a] Information Week "Apple iPhone 5 Teardown: Visual Tour", www.informationweek.com, October, 2012 Disassembly of iPhone 5, does not show separate digitizer/touchscreen overlay. [InputDynamics13a] InputDynamics "Input Dynamics: The Big Fat Soft-key -- Can we make Smartphones smarter?", www.inputdynamics.com, 2013 Touchscreen-type input for handsets, smartphones, small devices using acoustic detection with the microphone already present in the device. [IsarA12a] Isar, Ali; Kim, Seung-Chan; Poupyrev, Ivan; and Stec, Jan "Surround Haptics: Tactile Feedback for Immersive Gaming Experiences", Proc. CHI 2012, May 5-10, 2012, Austin Texas, pp. 1087-1090 Haptic feedback / haptic display using vibratory transducers in coarse grid, by interpolation gives impression of high-resolution touch input. Vibrators mounted in back and seat of a chair (whole body), where tactile resolution of human skin is low. [IsarA12b] Isar, Ali; Bau, Olivier, Kim, Seung-Chan; Poupyrev, Ivan "Tactile Feedback on Flat Surfaces for the Visually Impaired", Proc. CHI 2012, May 5-10, 2012, Austin Texas, pp. 1571-1576 Haptic feedback / haptic display for the blind for images from a camera: blind user feels image with fingers, which have higher tactile resolution. [JajszczykA12a] Jajszczyk, Andrzej (ed.) "A Guide to the Wireless Engineering Body of Knowledge, 2nd Edition", IEEE Press, 2012 IEEE reference/tutorial on wireless communications e.g. standards, cellphone GSM, Wifi, etc. Tutorial section on TCP/IP and VoIP/SIP in packet switched networks, antennas, facilities, radio theory including polariztion, antenna theory, atmospheric propagation effects. [JanakM11a] Janak, Miroslav; Barna, Jozef; Novak-Marcincin, Jozef "Scanning of Spacious Objects with use of Microscribe System", Manufacturing Engineering, Vol 3, pp 48-50, 2011 pantographic/mechanical 3-D digitizer used to scan (contact scanning) larger objects: printers, small busts, with capturing physical texture of surfaces [JersaAT13a] Jersa, Andrea Theresa; Jersa, Michale Joseph III; Bhardwaj, Avinash Girish "Write Wound Vibrating Device", United States Patent 8,519,834, August 27, 2013 Wrist-mounted (like wristwatch) touchscreen display/dialer coupled to smartphone, with haptic (vibrating) buzzer to signal user. Compare with Opticon, accessibility-technology units in Carroll catalog? [JoguetP11a] Joguet, Pascal and Largillier, Guillaume "Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen", United States Patent 8,049,730, November 1, 2011 Includes description of JazzMutant/Stantum multi-touch touchscreen, overcomes masking/shadowing effect of simple switch matrix designs by using active matrix switching array (similar to active matrix LCD circuitry) to measure each coordinate in grid of perpendicular wires separately. Also refers to measuring resistance drop in ITO along long paths and short paths for conductive traces. Filters isolated touch pixels as noise, computes barycenter (center of gravity) of blobs from fingertip touching. See also 2007 SID article by Largillier [JotaR13a] Jota, Ricardo; Ng, Albert; Dietz, Paul, and Wigdor, Daniel "How Fast is Fast Enough? A Study of the Effects of Latency in Direct-Touch Pointing Tasks", Proc. CHI 2013, Session: Touch Interaction, Paris, April 27..May 2, 2013 pp. 2291-2300 Study of latency on users of multi-touch/single-touch touchscreens. Suggests latency needs to be < 1msec to be imperceptible to user. Latency of 10 msec affects user performance in direct manipulation operations (e.g. in drag operations). Experimental system uses optical tracking on tabletop. Developers fall back to tap operations as a result. [KaneSK11a] Kane, Shaun Kk; Wobbrock, Jacob O.; and Ladner, Richard E. "Usable Gestures for Blind People Understanding Preference and Performance", Proc. CHI 2011, May 2..12, 2011, Vancouver BC Canada, 10 pp. Gestures by visually impaired (completely blind) users are less accurate than for sighted users: examples include triangle, square, circle. Line segments tend to be less straight / more wavy. Strong blind user preference for multitouch gestures, rather than drawing, or preference for gestures based on screen corners and edges (see Windows 8 gestures?) Compares iPad VoiceOver and Google Eyes-Free Shell for Android, which however have no common gestures (lingua franca). Citations to other gesture user interfaces for the blind: NavTouch, Slide Rule, Talking Fingertip Technique, EarPod, No-Look Notes, etc. [KawaharaY13a] Kawahara, Yoshihiro; Hodges, Steve; Cook, Benjamin S.; Zhang, Cheng; and Abowd, greGory D. "Instant Inkjet Circuits: Lab-based Inkjet Printing to Support Rapid Prototyping of UbiComp Devices", Proc.



UbiComp '13, Conference on Ubiquitous Computing, Zurich, September 8..12, 2013, pp. 363-372 3D printing to fabricate passive RFID tag circuits, touch-sensitive devices: plush toys, touchscreens, etc. Conductive double-sided tape. Conductive silver ink. [KayAC13a] Kay, Alan Curtis "An Interview with Computing Pioneer Alan Kay: interviewed by David Greelish", Time Magazine, April 2, 2013 Interview with Alan Kay, discussion of unfulfilled educational opportunities of Dynabook. Repeats illustrations from 1972 Dynabook: stylus shown in one sketch, but no description of touchscreen or stylus beyond criticism of iPad UI. States that GRAIL project (see) had "very good character recognitzer", but keyboard still required. Says handwriting (cursive) recognition bad idea to hand-printing. Newton politics "very complicated" [KeefeD12a] Keefe, Daniel; Krueger, Antonio; Steinicke, Frank; and de la Riviere, Jean-Baptiste "Touching the 3rd Dimension", Dagstuhl Seminar 12151, Dagstuhl Reports Vol 2 No 4, April 2012 Seminar collection of presentations on 3D touch input and 3D visualization, including multi-touch touchscreen gestures for 3D input, stereoscopie viewing, 3D sketching (medical/surgery), stereoscopic 3D cursors. [KhalighHH13a] Khaligh, Hadi Hosseinzadeh and Goldthorpe, Irene A. "Failure of silver nanowire transparent electrodes under current flow", Nanoscale Research Letters, 2013, Vol 8 No 235, available at www.nanoscalereslett.com/content/8/1/235 silver nanowire transparent electrodes (replacement for ITO) not stable due to heat if carring much current (e.g. solar cells): recommend cupronickel, since thicker nanowires makes less transparent [KidaY12a] Kida, Yasuo; Kocienda, Ken; and Furches, Elizabeth Caroline "Language Input Interface on a Device", United States Patent 8,289,283, October 16, 2012 Virtual keyboard for Ideographic/Asian and other languages: enter text phonetically, displays possible characters from a drop-down list: extensive references, but no references before 1990. [KienzleW13a] Kienzle, Wolf and Hinckley, Ken "Writing Handwritten Messages on a Small Touchscreen", Proc. Mobile HCI 2013 - Touch and Text Input - August 29, 2013, Munich, pp. 178-182 combine handwriting (hand-printing) recognition and sketch/drawing input in one user interface: draw/write mode slected. User can point to segmentation errors, system re-does with second-best segmentation (?). [KimD12a] Kim, David; Hilliges, Otmar; Izadi, Shahram; Butler, Alex; Chen, Jiawen; Oikonomidis, Iason; and Olivier, Patrick "Digits:Freehand 3D Interactions Anywhere Using a Wrist-Worn Gloveless Sensor", Proc. UIST '12, October 7..10, Cambridge MA; pp. 167..176 Wearable 3D pointing and hand-gesture system, similar to data gloves: angled video sensors on wrist image palm and fingers, determine shapes and positions of fingers. Accelerometer for approximate wrist/arm movement: no absolute determination of position. [KimS12a] Kim, Sunjun; Yu, Jihyun; and Lee, Geehyuk "Interaction Techniques for Unreachable Objects on the Touchscreen", Proc. OZCHI '12, November 2-30, 2012, Melbourne, Australia, pp. 295-298 User study for off-screen gestures: edit triggering (touch edged and move in) better than large-touch (similar to press-and-hold) [KimSC13a] Kim, Seung-Chan; Israr, Ali; and Poupyrev, Ivan "Tactile Rendering of 3D Features on Touch Surfaces", Proc. UIST '13, October 8..11, 2013, St. Andrews, UK Haptic/tactile display on flat touchscreen. Changes friction on fingertip moving on touchscreen through electrovibration: alternating electrostatic force attracts/releases finger, feels like lateral friction. User perceives friction as bulges of 3D three dimensional objects. Single point, no multitouch. Integrated with electrostatic/capacitive touchscreen. [KindsmuellerMC11a] Kocienda, Kenneth; and Williamson, Richard "Photosurface: Ein Multitouch- Leuchttisch f\ür Berufsfotografen", Workshop-Proceedings der Tagung Mensch und Computer 2011, T.U. Chemnitz, Germany Multitouch table display surface, used for editing/tagging/viewing/sorting photographs. Stated as tool for professional photographers, but display quality is not yet high enough. [KirkDS12a] Kirk, David S.; Izadi, Shahram; Hilliges, Otmar; Taylor, Stuart; Sellen, Abigail; Banks, Richard "At Home with Surface Computing", Proc. CHI 2012, Touch in Context, May 5-10, 2012, Austin TX, pp. 159-168 multi-touch tabletop with overhead camera, study of use by subjects at home for sorting photos, etc. Simulated physics of virtual objects useful for referential transparency. Collaboration *not* good with tabletop, contrary to many references. Natural interact for kids was mouse and double-click, not physical touching. [KociendaK12a] Kocienda, Kenneth; and Williamson, Richard "Method, System, and Graphical User Interface for selecting a Soft Keyboard", United States Patent 8,179,371, May 15, 2012 Display small icons for multiple virtual keyboard, user selects the one they want: alphabetic, numeric, punctuation. Selected keyboard becomes active. [KoehlyR11a] Koehly, Rodophe "Fabrication of Sustainable Resistive-Based Paper Touch Sensors: Application to



Music Technology", Ph.D. Thesis, McGill University, Montreal QC Canada, August 2001 Fabrication of force sensitive resistors (FSRs) in any shape or size using conductive paper. Papers included OTS industrial papers designed originally for specific colors. Conductive paper and inks more ecological (greener) alternative to conducting polymers for touch sensors, cheaper than industrial sensors. [KrahenbuhlJ11a] Krahenbuhl, John; Athale, Anant; Shenman, Arnold; Sklovsky, Vladimir; and van Bosch, James A. "Method and Apparatus for an Adaptive Touch Screen Display", WIPO International Patent Application Publication WO 2011/025619 A1, 3 March, 2011 In proximity sensing on tablet, show pop-up menu (circular, rectangular, dialog, etc.) as stylus/finger passes over an active element/icon. [KramerRK11a] Kramer, Rebecca K.; Majidi, Carmel; and Wood, Robert J. "Wearable tactile keypad with stretchable artificial skin", ICRA 2011: IEEE Intl. Conf. on Robotics and Automation, May 2011, pp. 1103-1107 elastomeric keypad (pressure-sensitive resistive material and matrix of conductors) that can be attached to the skin, usint conductive liquid-filled microchannels that are compressed. Hyperelastic pressure sensors, stretchable/flexible and liquid materials cited back to 1995. Compare with force-sensing rubber materials with embedded particles, other robotic skin sensors. [KranerRH11a] Kraner, Ramon Hofer "Tracking Technologies for Interactive Tabletop Surfaces", ScD thesis, ETH Zürich, Summer 2011 Review and engineering analysis of prototypes for tracking/touchscreen digitizer technologies for interactive tabletops (multi-touch, multi-user). Anoto pattern (optical), FTIR internal reflection, Infrared optical touchscreen. Rear and front projection displays. [KurokawaY12a] Kurokawa, Yoshiyuki and Ideka, Takayuki "Solid-state image sensing device and semiconductor display device", United States Patent Application 2012/0032193, February 9, 2012 Optical touchscreen sensor, uses optical sensors and mask with optical holes to detect angles of incident light from approaching finger tip (proximity sensing). Discussion of chemistry of sensors. [Kyocera11a] Kyocera "Kyocera Echo product information", www.echobykyocera.com, fetched Feb 2011 Android smartphone with dual touch screens instead of keypad and touchscreen, screens can be folded flat to create effective larger touchscreen ("Tablet mode"). Announced with Sprint/Nextel. Additional materials in file. Compare with Kyocera Refalo [LGElectronics11a] LG Inc. "LMS-100 Mouse/Scanner product information", www.lg.com, 2011 Optical mouse with hand-held scanner in the base, includes software for stitching together scans to cover entire sheet of paper. Scan resolution specified as up to about 300 dpi. [LapstunP11a] Lapstun, Paul and Silverbrook, Kia "Coding Pattern with Replicated Local Tag Data in Each Tag", United States Patent 8,011,595, September 6, 2011 optical digitizer pattern on paper using "tags" comprised of cells. Cites to Anoto patent. "Netpage" links from paper to URLs. [LeapMotion13a] Leap Motion, Inc. "Leap Motion Motion Controllers", www.leapmotion.com, 2013 Optical motion sensor using structure-with-motion (see P. Chan) and single infrared camera to detect hand and figure gestures in the air. Sold as retail product. See also D. Holz. [LeeB12a] Lee, Bhoram; Lee, Hyunjeong; Limi, Soochul; Lee, Hyungkew; Hand, Seungiu; and Park, Joonah "Evaluation of Human Tangential Force Input Performance", Proc. CHI 2012, May 5-10, 2012, Austin, TX, pp. 3121-3129 Measure tangential/shear force on PDA/Smartphone touchscreen using force sensors on back and on side edges of touch surface. (Ipad in an enclosure, sensors on enclosure where iPad rests.) Users more likely to use tangential force for zooming rather than positioning. [LeeJU13a] Lee, Jong-Uk; Lim, Jeong-Mook; Shin, Heesook; and Kyung, Ki-Uk "A Haptic Touchscreen Interface for Mobile Devices", Proc. ICMI '13, December 9-13, 2013, Sydney Australia, pp. 311-312 Tactile (vibration) haptic feedback on touchscreen using electrostatic force (2400 voltes??) between two transparent plates with ITO. Because of system latency (Android has delays in responding to user input), actuated by separate controller. [LewisJE11a] Lewis, James E. and Hieb, Jeffrey Lloyd "AC 2011-940 Promoting Faculty Adoption of Tablet PCs in Upper Level Engineering Courses", American Society for Engineering Education, 2011 Study of using Tablet PCs in engineering/college education: mentions success of multiple-display of lecture screen, instructor's screen, and (selected) student screen. Cites to DyKnow and Classroom Presenter whiteboard systems. Compare with GroupWare and Whiteboard systems, especially multi-user/multistation. Extended desktop (off-display places to keep things) very useful, also use of tablet PC permits instructor to face class while writing on whiteboard. [LiaoC12a] Liao, Chunyuan and Guimbretiere, Francois "Evaluating and Understanding the Usability of a Penbased Command System for Interactive Paper (PapierCraft)", ACM Trans. on Computer-Human Interaction, Vol 190 No 1, Article 3, March 2012 Paper-based interfaces became popular (again) with Anoto 2002, vs. Wellner. Anoto gestures (PapierCraft)



for PADD augmented paper, gestures to select, add links. Vibration /haptic feedback added to pen/stylus. Cites to PenTop. [LietzkeMP11a] Lietzke, Matthew P.; Dudkowski, Eric Edward; Humphrey, Timothy; Locker, Howard; Skinner, Jeffrey E.; Smith, Adam Miles; Stewart, Aaron Michael; and VanBlon, Russell Speight "Emulating Fundamental Forces of Physics on a Virtual Touchable Object", United States Patent Application 2011/0029927 A1, February 3, 2011 Tiled user interface for pen computer (or touchscreen): simulate physics on tiles when user moves, flicks, or drags them: effects include larger objects moving more slowly (inertia), gravity/attraction between objects (snap to join, unjoin. Initial trajectory taken from user gesture. Properties may be different for different objects (parameterized). Compare with Artificial Reality project? [Liine12a] Liine "Lemur User Guide 3.1.0 for iOS", Liine, April 2012 IDE/GUI design package for Music/Midi controllers for iOS/iPhone/iPad devices on touchscreen. Relationship to JazzMutant Lemur? File contains additional materials from 2010 [LimSE12a] Lim, Seung E. "Control of electronic games via finger angle using a high dimensional touchpad (HDTP) touch user interface", United States Patent 8,169,414, May 1, 2012 Use high-dimensional touchpad (see Rekimoto), detect yaw, pitch, roll, rotate angles of fingertip by shape of blob image, use angle etc. as input in a games. Cites to Ludwig for underlying touchscreen sensor. [LinJC12a] Lin, Jao-Ching; Chu, Lin-Abel; Liu, Yen-Fu; and Shen, Chung-Yi "Method for identifying a single tap double taps and a drag and a controller for a touch surface employing the method", United States Patent 8,164,575, April 24, 2012 tap, double-tap, drag gestures recognized/detected using two time thresholds and a movement threshold [LiquidPubs12a] Liquid Pubs "Apple: Their Tablet Computer History", LiquidPubs.com, fetched 2012, dated Nov 2008 History of Apple tablet/touchscreen devices from 1979 through 2008: Apple Newton MessagePad 2100 (1998); Apple Graphics Tablet (Summagraphics BitPad one?) (1979); (1983) Bashful / Snow White concept design tablet/stylus computer with separate computer; (1894) 24HourMac pen-sensitive LCD computer (small display); (1985)BookMac; (1984) TelephoneMac with stylus. Steve Jobs' tablet/pen computer proposals to Apple/Scully rejected in 1985. 1985 flat-panel version of Apple IIc. Dynabook-style P2 Portable Computer Concept (1989), Knowledge Navigator / Figaro tablet computer; PenMac (1992); VideoPad (1993: MacExpo 1995). video with VideoPad. [LiuS12a] Liu, Shenwei and Guimbretiere, Francois "FlexAura: A Flexible Near-Surface Range Sensor", Proc. UIST '12, October 2012, Cambridge, MA. pp. 327-330 Optical multitouch sensor with proximity sensing: small LEDs and phototransistor sensors on a flexible substrate, can be bent/formed around a cylindrical status/pen or other objects. Used to sense gestures of the inside position of the hand when gripping the stylus (e.g. handwriting grip vs. drawing grip), as well as environmental positions (stylus in "ink cup" or lying down on a surface). [LjLies11a] LjLies "Sharp Zaurus PDA Video 1/2", YouTube.com, posted 2011 Demonstration of Sharp Zaurus PDA circa 1994, Product released only in Japan: handwriting recognition, virtual on-screen keyboard. [LjLies11b] LjLies "Sharp Zaurus PDA Video 2/2", YouTube.com, posted 2011 Demonstration of Sharp Zaurus PDA circa 1994, Product released only in Japan: handwriting recognition, virtual on-screen keyboard. [LjLies11c] LjLies "MVK 0989 Newton Messagepad 2000 PDA Video", YouTube.com, posted 2011 Demonstration of Newton Messagepad 2000 Circa 1995 (Copyright date in code): handwriting recognition, virtual on-screen keyboard. [LockerH11a] Locker, Howard; Dudkowski, Eric Edward; Lietzke, Matthew P.; Skinner, Jeffrey E.; Smith, Adam Miles; Stewart, Aaron Michael; and VanBlon, Russel Speight "Finger Touch Gesture for Joining and Unjoining Discrete Touch Objects", United States Patent Application 2011/0029934 A1, February 3, 2011 Touchscreen graphical user interface: two-finger tap to associate two objects, single-finger tap to select the objects. For use with application tiles. Also flick gesture, different objects have simulated mass, inertia, gravity, friction, velocity, and haptic force feedback. Two finger gesture to join objects, one finger on each object. Press-and-hold to do un-join. Long press-and-hold to remove a tile/object from a joined group. [Logitech11a] Logitech "Logitech Tablet Mouse for Android 3.1", Logitech.com, 2011 Android 3.1 support primary-button-only input for bluetooth mouse on tablets (such as Motorola). Appears to part of support for Bluetooth HDI keyboard similar to bluetooth keyboard for Apple iPad: plugging in Bluetooth keyboard on Android causes "mouse cursor" to appear on screen (according to support blog note). [LueH11a] Lue, Hao and Li, Yang "Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures", Proc. CHI 2011, May 7-10 Vancouver, CA, pp. 207-216 Overlay a window on existing touchscreen application: for doing fine control on touchscreen user



interfaces, reverse-engineering of actual application user interface. User sketches shape of (small) UI object on screen, Avatar finds object with closest shape match (also smallest city-block/manhatten distance), user can then move drawn shape as proxy/avatar for actual. Also write (handwriting recognition) single letter, finds first/next element with that letter. Compare with PenPoint/Palm file dialogs (first letter), Pencept recognition macros, smart targeting in PenPoint and Buxton virtual keyboards. [LueH12a] Lue, Hao and Li, Yang "Gesture coder: A tool for programming multi-touch gestures by demonstration", Proc. CHI 2012, May 5-10, 2012, Austin TX, pp. 2875-2884 Attempt for trainable multi-touch gesture recognizer: multi-touch gestures tend towards direct manipulation also, not gesture recognition by shape for a discrete action. Separates finger configuration (number of finger touches) from movement of fingers. [MackeyB12a] Mackey, Bob "Synaptics Innovations in Touch Sensing", Slide presentation, SID 2012, Boston MA Technical presentation by principal scientist of Synaptics on touchscreen/touchpad sensing technologies: Resistive: 4-wire, 5-wire Matrix. Capacitive: Surface, Projected (Trans-Capacitance (Mutual), AbsoluteCapacitance (Self)) Acoustic: SAW, Bending wave, Optical: Frustrated Internal Reflection, Above-surface (IR). Placement of capacitive sensor at different optical layers. Synaptics primarily capacitive: Suggests thin wire grid or other structured transparent conductor (e.g. carbon nanotubes) better electrically than ITO, can be made effectively invisible. Transparent conductor films include BlueNano, Cabrios silver fibers, Carestream. [MakinenV11a] Makinen, Ville; Suvanto, Petro; and Linjama, Jukka "Method and apparatus for sensory stimulation", United States Patent 7,982,588, July 19, 2011 Tactile/haptic electrosensory (not electrovibratory) display. AC-modulated signal coupled capacitively to fingertips at discreet pads, sensation is direct Coulomb force capacitive to Pacinian mechanoreceptor nerve cells in fingertip, not physical vibration or electrostatic friction. Grid arrangement of pads for multi-touch. Assigned to SenSeg.com [MatiasRJ12a] Matias, Ricardo J.S.; Cunha, Manuel Bernardo; Mota, Alexandre Manuel; and Martins, Rui Manuel "Modeling Capacitive Coupling Systems for Body Coupled Communications", Proc. of BODYNETS 2012, September 24-26, Oslo, Norway. pp. 113..119 Estimating signal (and data rate) transmitting data between two users via capacitive sensing/coupling through their body. Mentions body area networks (BAN), near field communications (NFC) and body couple communications (BCC). Cites to Zimmerman 1995 [McAvinneyP12a] McAvinney, Paul "Paul McAvinney, biography", obtained 2012, original date probably earlier Autobiographical notes from Paul McAvinney: states that Sensor Frame, Video Harp in 1980's was first optical multi-touch input device, computer mouse invented by Doug Engelbart at SRI prior to his going to Xerox PARC, thus Xerox corporate and other references are in error. [McCoyC13a] McCoy, Charles; Wong, Ling Jun; and Xiong, True "Method and apparatus for identifying available IPTV devices on a network", United States Patent 8,621,507, December 31, 2013 Loading code (Javascript, Java, or other browser-loadable code) to view a multimedia file/stream. Determining reachability of devices by sending a message to them, receiving a response. [McDermidWJ13a] McDermid, William J. "Methods and Apparatus for Determining Coordinates of Simultaneous Touches on a Touch Sensor Pad", United States Patent 8,355,009, January 15, 2013 resistive-grid multi-touch touchpad/tablet. Uses contact between conductors to identify conductors X/Y for first location, then measured resistance/voltage to determine area of contact. Says change in resistance from a second touch may be used to correct location. Not much information on electronics. Compare to Kaplow? [McPhersonA11a] McPherson, Andrew and Kim, Youngmoo "Design and applications of a multi-touch musical keyboard", Proc. Sound and Music Computing, Padova, Italy, 2011 Musical keyboard (piano/midi) with X/Y capacitor touchpads on keys to sense position and pressure/force/area of contact for musical expression. Tracks touches by comparing for minimal motion. Note shaping (pitch, timber, volume, voice selection) using single and multi-finger gestures. Cites to Moog 1990. [McPhersonA12a] McPherson, Andrew "TouchKeys: Capacitive Multi-Touch Sensing on a Physical Keyboard", Proc. New Interfaces for Musical Expression, Ann Arbor, Mich. 2012 Musical/piano keyboard with capacitive multi-touch touchpads on keys, construction uses CapSense ICs from Cypress Semiconductor. Pinch and slide gestures with multiple fingers to control pitch, timbre, vibrato. [McPhersonAP13a] McPherson, Andrew P.; Gierakowski, Adrian; and Stark, Adam M. "The Space Between the Notes: Adding Expressive Pitch Control to the Piano Keyboard", Proc. CHI 2013, Paris France, April 27-May 2, 2013, pp. 2195-2204 Musical/piano keyboard with capacitive multi-touch touchpads on keys. [Mechaless13] Mechaless Systems GmbH "Helios 3d HMI touchless gesture system: product information", Mechaless Systems GmbH, Bruchsal, Germany, 2013



in-air touchless gesture recognition using infrared light: includes Windows 8 touch language gestures: swipe, flick, in-air flick, etc. [MenschComputer11a] Eible, Maximilian and Ritter, Marc, Eds. "Workshopband Mensch und Computer 2011", Conference Proceedings, Technische Universität Chemnitz, 2011: www.uebermedien.org (mostly in German) Conference workshop on human/computer interaction systems. Includes tabletop systems (Tabletop), Photosurface: multi-touch light table (tabletop) for professional photographers, personal territories in tabletop applications. [MicroTouch12a] 3M MicroTouch "3M MicroTouch Capacitive TouchSense System - The Tactile Feedback Touch System", 3M MicroTouch, 2012 MCT MicroTouch capacitive touchscreen with two or more screen actuators for simulating feel of depressing mechanical buttons: ClearTek II touch sensor, EX II tactile feedback controller. [Microsoft11a] Microsoft "SHRecognizeGesture", msdn.microsoft.com: fetched Feb 2011, original date probably earlier press-and-hold / tap-and-hold gestures in Windows CE 5.0, which bring up context menu in Windows OS Shell. Documents software confusion between recognition/tablet events (recognition of gesture) and the mouse events (WM_CONTEXTMENU) that are emulated: compare with mouse versus tablet events in Windows for Pen Computing. [Microsoft11b] Microsoft "Events and Delegates", msdn.microsoft.com: fetched April 2011, original date probably earlier Definition of event as a message sent by an object. [Microsoft11c] Microsoft "MSDN: Windows Touch Gestures Overview (Windows)", msdn.microsoft.com: fetched april 2011, original date probably earlier Touch gestures: press and tap is press on target and tap using second finger (right-click), scrolling, mouse drag/ selection, pinch gestures for zoom, two-finger rotate, two fingers tap (using midpoint), press-andhold, flicks. Not really multi-touch? press-and-tap could be done on "averaging" resistive tablet. [Microsoft11d] Microsoft "MSDN Timeline of Mouse Messages and System Events TabletPC", msdn.microsoft.com: april 2011, original date probably earlier (see Microsoft03) Sequence of mouse messages and touch event messages, showing that mouse messages and touch events are not synonymous, sequence may differ. [Microsoft11e] Microsoft "Microsoft Touch MSDN API Documentation", msdn.microsoft.com: april 2011, original date probably earlier (see Microsoft03) Extract of MSDN documentation on Microsoft touch gestures, touch events, touch programming. WM_SCROLL not identical to flick scrolling behavior, Inertia, Expansion and Rotation. Real-time stylus electronic ink has low data rate from tablet? [Microsoft11f] Microsoft "Microsoft Tablet PC Knowledge Base", msdn.microsoft.com: 2011, collection of Tablet PC knowledge base articles. Microsoft Tablet PC / Windows Vista Knowledge Base articles. Stylus/pen with barrel/pen button, eraser button (e.g. Wang Freestyle). Flick and other gestures, gesture hot spot. Eight directional flick gestures in Windows Vista. Recommends zooming UI for text input (larger input) using StylusInput API. Discussions of ink smoothing in Windows electronic ink, preserving cusps from Bezier rounding. [Microsoft11g] Microsoft "Selecting and Editing Ink: Barrel Button on Tablet PC", msdn.microsoft.com: 2011, collection of Tablet PC knowledge base articles. Selection by lasso gesture, toolbar button (modal), or barrel button on stylus. Notes that behavior of barrel button (and other behaviors) is a recommended behavior, not standardized by operating system. [MilekicS12a] Milekic, Slavko "Using Eye- and Gaze-tracking to Interact with a Visual Display", Presentation, Electronic Visualisation and the Arts (EVA 2012), London, UK, 10 - 12 July 2012 Review of technologies and applications for gaze-tracking/eye-tracking. Eye gestures (flick left/right/up/down) as alternative to dwell gesture, switching between gaze/observation and input/control modes. Eye tracking to direct spotlight automatically to where someone is looking in museum display. [MoellerJ12a] Moeller, Jon and Kerne, Andruid "ZeroTouch: An optical multi-touch and free-air interaction architecture", Proc. CHI 2012, May 5-10, 2012, Austin TX, pp. 2165..2174 Includes review of optical light-sensing multi-touch tablet technologies: Thinsight, Echtler, Pixelsense, FTIR, FLATIR, Reverse FTIR, Light Plane, Diffuse Illumniation (shadows), LED/photosensor array (occlusion), Corner-Camera, interference. Occuded-light design with beams at multiple angles *mostly* reduce shadowing problem, at least when only input is finger tips, and not long objects (pen lying sideways). [MontserratTJK13a] Montserrat, Toni-Jan Keith; Panday, Anshul Vikram; Zhao, Shengdong; and McGee, Kevin "NoteVideo: Facilitating Navigation of Blackboard-style Lecture Videos", Proc. CHI 2013, Video Showcase, Paris, 2013, p. 2897 (extended version of paper also in files) NoteVideo and NoteVideo+ browsing/indexing of video lectures, focusing on whiteboard, system



recognizes, acts of drawing, erasing/scrolling, and explaining. NoteVideo+ includes transcript text of spoken lecture with snapshot images. [MoonK13a] Moon, Kyu; Kim, Jae Il; Lee, Hanleem; Hur, Kongheon; Chim, Woon Chun; and Lee, Hyoyoung "2D Graphene Oxide Nanosheets as an Adhesive Over-Coating Layer for Flexible Transparent Conductive Electrodes", Scientific Reports, Vol 3 No 1112, 23 January 2013 Graphene overcoating on silver nanowires in conductive/resistive transparent films: substitute for ITO on touchscreens [MoskowitzSA11a] Moskowitz, Scott A. and Berry, Mike W. "Method and Device for Monitoring and Analyzing Signals", United States Patent 7,949,494 May 24, 2011 Cites to 7,346,472. Digital watermarking of audio/visual signals/steganography. Compare abstract/analysis of a given signal with stored abstracts of reference signals in a database to find a match. [MoskowitzSA12a] Moskowitz, Scott A. and Berry, Mike W. "Method and Device for Monitoring and Analyzing Signals", United States Patent 8,214,175, July 3, 2012 Cites to 7,346,472. Digital watermarking of audio/visual signals/steganography. Database of abstracts of reference signals: compare abstract/analysis of a given signal with stored abstracts of reference signals to find a match. [Motorola11a] Motorola "Motorola Admiral User Guide (Smartphone)", Motorola Corp., 2011 User guide for Admiral smartphone running Android operating system. Multiple home screens, haptic feedback (vibrate), multitouch touchscreen. [Motorola11b] Motorola "Motorola User Guide Droid2 Global (Smartphone)", Motorola Corp., 2011 User guide for Droid2 smartphone running Android operating system. Multiple home screens, haptic feedback (vibrate), multitouch touchscreen. Swype touchscreen keypad: draw single connected stroke through letters on keyboard to form a word. Multi-touch keyboard for shift key and regular key. [Motorola11c] Motorola "Motorola User's Guide Droid Razr (Smartphone)", Motorola Corp., 2011 User guide for Droid Razr smartphone running Android operating system. Touch tips (Gestures): Touch, Touch and Hold, Drag, Flick, Zoom (two finger pinch/unpinch). Multi-touch keyboard (touchscreen) vs. Swype keyboard. [Motorola11d] Motorola "Motorola User Guide Motorola XPRT(Smartphone)", Motorola Corp., 2011 User guide for XPRT smartphone running Android operating system. Touchscreen and multi-touch keypad (physical keypad). [Motorola11e] Motorola "Mastering Your Device: Motorola Droid Razr", Motorola Corp., 2011 User guide for Droid Razr smartphone, include Webtop "PC" when using external keyboard and display. [Motorola11f] Motorola "Motorola Droid X2 User Guide / Manual", Motorola Corp., 2011 User guide for Droid X2 smartphone. Touch & hold (press/hold) gesture. Automatic screen rotation/orientation landscape/portrait in "many" apps, pinch/zoom, flick, drag gestures. Double-tap to zoom/unzoom in Google maps. Haptic feedback (vibration) on press/hold time threshold. Swype text entry select characters from virtual keyboard at inflection points. multi-touch used for shift keys on virtual/software keyboard. [Motorola11g] Motorola "Motorola Droid2 Smartphone", Motorola Corp., 2011 (physical device) Motorola Droid2 smartphone with touchscreen, slide-out physical keyboard. Two units [Motorola12a] Motorola "User Guide: Droid XYBoard 10.1 by Motorola (Verizon model)", Motorola Mobility, 2012 User guide for small-profile DROID Android 4.1 ("Jelly Bean") tablet. Verizon version with pre-installed apps. [Motorola12b] Motorola "User Guide: Droid XYBoard 8.2 by Motorola", Motorola Mobility, 2012 User guide for small-profile DROID Android 4.1 ("Jelly Bean") tablet, generic version (not Verizon) [Motorola13a] Motorola "Motorola Force-Sensing Touch Technology", Motorola Solutions USA, 2013 Transparent touchscreen using top PET Polyethylene Terephthalate layer and lower piezo-resistive layer to sense actual force of touch. States that can be incorporated with both resistive and electrostatic/capacitive multi-touch touchscreens/digitizer (Why would separate touchscreen sensor be required?) [Motorola13b] Motorola "Motorola Moto X User Guide", Motorola USA, 2013 Motorola Moto X Android smartphone with touchscreen. Touchless control == speech recognition. Gestures include flick (from edges) Touch & Hold (press-and-hold/dwell), double-tap. Multiple virtual keyboards including Swype gesture keyboard, trace finger through letters. Explore by touch and TalkBack: compare with VoiceOver accessibility. Two finger zoom/pinch gesture. [Motorola13c] Motorola "Motorola Moto X Droid Smartphone", Motorola USA, 2013 (physical device) Motorola Moto X Android smartphone with touchscreen. Operational unit. Touchless control == speech recognition. Gestures include flick (from edges) Touch & Hold (press-and-hold/dwell), double-tap. Multiple virtual keyboards including Swype gesture keyboard, trace finger through letters. Explore by touch and TalkBack: compare with VoiceOver accessibility. Two finger zoom/pinch gesture. [MuellerJS11a] Mueller, Jennifer S.; Melwani, Shimul; and Goncalo, Jack A. "The bias against creativity: Why



people desire but reject creative ideas", Cornell University ILR school digital commons: digitalcommons.ilr.cornell.edu/articles/450, 2011/ Business article on why creative individuals are undesired in organizations: they break the routine. [MuellerS11a] Mueller, Sebastian "SmellTagger: Augmenting Design and Code Reviews with Multi-Touch Technology", Master thesis, University of Zurich, April 19, 2011 Multi-touch / multi-user interface using Microsoft Surface for collaborative review of "code smells": source code not matching automatic metrics of code structure (anti-patterns; shotgun surgery from insufficient abstraction between classes, etc.) Shows rear-projection / rear-camera Microsoft surface hardware. [Mueller-TomfeldeC11a] Mueller-Tomfelde, Christian and Fjeld, Morten "A Short History of Tabletop Research Technologies and Products", Presentation at CSIRO.au, July 2011 History of tabletop/virtual-desk technology, cites to Wellner, DigitalDesk, ActiveDesk, metaDESK, some whiteboard systems. Notes transition from single-touch touchscreen surfaces to multi-touch about 2001. [Mueller-TomfeldeC12a] Mueller-Tomfelde, Christian and Fjeld, Morten "Tabletops: Interactive Horizontal Displays for Ubiquitous Computing", IEEE Computer, February 2012, pp. 78-81 History of tabletop/virtual-desk touchscreen technology, cites to SmartBoard / ClearBoard / ActiveDesk approximately 1991/1992. States peak of "Hype cycle" on overly high expectations about 2002. [MujibiyaA13a] Mujibiya, Adiyan and Rekimoto, Jun "Mirage: Exploring Interaction Modalities Using Off-Body Static Electric Field Sensing", Proc. UIST '13, October 2013, St. Andrews, UK, pp. 211-220 electrostatic/capacitive sensing of body, hand, gesture position and motion using passive fields, no active signal like touchscreen. Include direct manipulation of graspable object, hand and body gestures, multipleuser / multi-touch differentiation. Cites to Rekimoto, DiamondTouch, other active touchscreen sensors. [MullenbachJ13a] Mullenbach, Joe; Shultz, Craig; Piper, Anne Marie; Peshkin, Michael; and Colgate, J. Edward "Surface Haptic Interactions with a TPad Tablet", Proc. UIST '13, October 2013, St. Andrews, UK, pp. 7-8 Demonstration paper for variable friction touchscreen/tablet computer for haptic feedback. Modified Kind tablet. Variable friction through ultrasonic vibration: vibration perceived as less friction, microscopically "bouncing" finger from surface. Demonstration of remote touch feedback. No multi-touch haptic. [MultiTaction13a] Cornerstone.com / Multitaction.com "MultiTaction Large Size Multitouch LCD displays", multitaction.com, 2013 Electronic file from multitaction web site. Large LCD display panels with white light LEDs for standard LCD backlighting, and infrared LEDs for shining IR light through panel, detectors to detect infrared light reflected back for touch/proximity sensing. Or can detect shadows in ambient light. Combination (alternating scans) for "multitaction" hybrid tracking. Panels can be combined to get effective touch wall of any size. Compare images with Rekimoto 2002. Multitouch Cornerstone Engine in multitouch cell displays. Shadow detection used to detect when a user is standing in front of display. Electronic file includes Cornerstone, multi-touch multitaction support site: SDK API for multitouch touchscreen input and applications. [MyAppleSpace11a] MyAppleSpace.com "Apple Graphics Tablet 1979 Summagraphics Magnetostrictive", MyAppleSpace.com, fetched 2012 Summagraphics magnetostrictive tablet for Apple computers, 1979 [NPR11a] National Public Radio "Timeline: A History of Touch-Screen Technology / The Touchy-Feely Future of Technology -- NPR", www.pnr.org, December 26, 2011 Timeline of touchscreen technology, taken from (and with links to) Bill Buxton web site. IBM Simon, but no Newton or PenPoint. Cites Apple ADB desktop bus as multi-touch, because two pointing devices could be connected. [Neonode11a] Neonode "Neonode zForce touchscreen technology for", Neonode.com, 2011. Paper file contains additional material. Optical touch-screen technology for cell phones, dating to approximately 2002. Infra-red LEDs? See also Goertz US 7880732, filed 2002. [Neonode12a] Neonode "Neonode zForce multisensing touch technology", Neonode.com, 2012 Optical touch-screen technology with multi-touch (and proximity sensing?) [NewScientist12a] Marks. Paul "Touchpad steering wheel keeps eyes on the road", NewScientist, Issue 2895, 14 December 2012 Touchpad put on spoke of steering wheel in car, driver uses simple swipe gestures (Yes/No/etc.) to answer voice annunciator ("Turn on headlights?") rather than physical buttons [NgA12a] Ng, Albert; Lepinski, Julian; Wigdor Daniel; Sanders, Steven; and Dietz, Paul "Designing for low-latency direct-touch input", Proc. UIST '12, October 7-12, 2012, Cambridge USA pp. 453-464 Latency (typically 50-100ms) in touchscreen GUIs distracting to user. Users perceive improvement down to 1msec latency. Suggested archicture: embed some of immediate visual response directly into low-level controls, leave full visual response to traditional MVC model-view-controller application. Cites VideoHarp, other early multi-touch, but not to Buxton 1985 or Apple. Compare with papers on deferred recognition,



raw ink shown for immediate feedback with recognized result to follow? [NguyenM11a] Nguyen, Mai; Wang, Xin; Chen, Eddie J.; and Tadayon, Bijan "System and Method for Rights Offering and Granting Using Shared State Variables", United States Patent 8,001,053, August 16, 2011 DRM usage rights management, rights transfered from a rights supplier to a rights consumer, with some rights derived from a meta-right / meta-right is the right to receive certain rights [NoguchiA13a] Noguchi, Anna; Kurosawa, Toshifumi; Suzuki, Ayaka; Sakamoto, Yuichiro; Oe, Tatsuhito; Yoshikawa, Takuto; Shizuki, Bunjtarou; and Tanaka, Jiro "Evaluation of a Soft-Surfaced Multi-touch Interface", Proc. HCII 2013 Human-Computer Interaction, pp. 469-478, 2013. WrinkleSurface force-sensing touchscreen. Modification of FTIR multi-touch table with smooth gel-underplastic layer on top. Sideways/tangential and twisting forces cause characterizable wrinkles in gel, detected by FTIR camera below transparent surface. Twist gestures, clock-wise CW and counter-clock-wise. [Nokia11a] Nokia "Nokia N900 SmartPhone User Guide", available at nokia.com, June 2011 Touch-screen smartphone with physical keyboard, no on-screen keyboard [Nokia11b] Lublin, Joann S. "Nokia, Microsoft talk cellphones", Wall Street Journal, February 10, 2011, page B1 Nokia discussing abandoning Symbian and Meego smartphone operating system for either Windows/Microsoft or Google/Android. [NowackaD13a] Nowacka, Diana; Ladha, Karim; Hammerla, Nils Y.; Jackson, Daniel; Ladha, Cassim; Rukzio, Enrico; and Olivier, Patrick "Touchbugs: Actuated Tangibles on Multi-Touch Tables", Proc. CHI 2013, pp 759..762 Tangible objects camera-based multi-touch surface. Objects can move themselves (haptic feedback also?), transmit data via infra-red to camera, sense orientataion (rotation): mentions shaking and tapping gestures. [Nuigroup11a] NUI Group Authors "Getting Started with MultiTouch", available at nuigroup.com, Feb 2011 Open-source hardware project for multi-touch hardware, mostly optical. rear diffusion / diffused illumination. FTIR Frustrated Total Internal Reflection) hardware. Laser Light Plane LLP. Diffused Surface Illumination DSI. Also Touchlib open-source library for multi-touch interaction surfaces, tracking blobs of light. [OTMTechnologies13a] OTM Technologies "Optical Translation Measurement VPen Device", OTMTechnologies.com, fetched 2013 (copyright dates to 2009) VPen electronic digitizer pen about the size of an Anoto pen, no special patterned paper. Handwriting (printing?) recognition built in, tilt sensing can be used as joystick. Laser and optical sensor in tip to measure "absolute" (?) position on surface? Mentions barcode scanning and OCR text scanning, 3D sensing. Also motion sensors (TMD Remote MOtion Detection) using projected laser pattern and optical sensors. Product names NaviLite and Light Speed Innovation. See also Kinrot, Lederer. [OdomG13a] Odom, Gary "User-Selectable Signatures", United States Patent 8,429,415, April 23, 2013 computer login using user's choice of multiple input devices: mouse, keyboard etc. and multiple types of input: e.g. actual password, keystroke timing, etc. [OlberdingS13a] Olberding, Simon; Gong, Nan-Wei; Tiab, John; Paradiso, Joseph A.; and Steimle, Juergen "A Cuttable Multi-touch Sensor", Proc. UIST '13, October 2013, St. Andrews, UK Flexible multitouch touchscreen/surface that can be cut with scissors to multiple shapes, intended for adding multitouch sensing to physical objects. Rather than grid, star of conductors to sensing points/pad from a central point in the original sheet: can be cut to any shape as long as the center is not cut and every pad has a line-of-sight to the central point. Also wiring up pads in pairs / small groups so that if one is disconnected (conductor trace cut), position can be measured using the other pads; or "spare" extra conductor traces running to pads. [OnoM13a] Ono, Makoto; Shizuki, Buntarou, and Tanaka, Jiro "Touch & Activate: Adding Interactivity to Existing Objects using Active Acoustic Sensing", Proc. UIST 13, October, 2013, St. Andrews UK Add touch sensing to objects (hand-held: toys, PDAs, dolls, cups) usint vibration speaker/activator and piezo-electric microphone. Interactive touches/ grasping gestures recognized by FFT resonance response to swept signal: touching/hold the object changes the resonance. Some ability to recognize pressure or firmness of grasp. [OpenStax12a] OpenStax College, Rice Univesity: Senior Contributing Authors Paul Peter Urone and Roger Hinrichs "College Physics", OpenStax College, June 21, 2012: available at cnx.org/content/col11406/latest/ First year college physics reference [OrdingB11a] Ording, Bas; Forstall, Scott; Christie, Greg; Lemay, Stephen O.; Chaudhri, Imran; Williamson, Richard; Blumenberg, Chris, and Van Os, Marcel "Portable Electronic Device, Method, and Graphical User Interface for Displaying Structured Electronic Documents", United States Patent 7,864,163, January 4, 2011 Show a document in structured form, a gesture (touchscreen) on an element selects and it is zoomed. Multi-touch de-pinch (zoom) gesture. [OrdonezCD12a] Ordonez, Cynthia Duque "Netbooks y tabletas", Posted to de.scribd.com, January 6, 2012 (In Spanish / en espan~ol) Overview history of tablet and notebook computers (pen computers). Cites to



Dynabook, Apple 1983 Frog Design tablet computer, Cambridge Z88, Linus Wirte-Top, GRiDPad 1910, Fujitsu PoqetPad, GRiD 2260, Apple Newton, EO PenPoint, etc. No mention of Pencept etc. (because not portable?) but no mention of Pen Windows either. Mentions Apple Axiotron Modbook ASUS R2H 2007 Mac tablet. [OuyangTY12a] Ouyang, Tom Y. and Li, Yang "Bootstrapping Personal Gesture Shortcuts with the Wisdom of the Crowd and Handwriting Recognition", Proc. CHI 2012, May 5-10, 2012, Austin TX, pp. 2895-2904 Gesture Marks: crowd-source what handwriting gestures (usually short cursive letter combinations, e.g "Ph" for phone) are used by most users for bring up apps or functions, versus user making up own gestures. Start with crowd-sourced gestures, gradually learn individual user's gestures on touchscreen/PDA. [OuyangTY12b] Ouyang, Tom Yu "Understanding Freehand Diagrams: Combining Appearance and Context for Multi-Domain Sketch Recognition", Ph.D. Thesis, M.I.T., February 2012 Domain-specific sketch recognition using also syntactic/structural rules for specific domain applied to context of nearby input: analog electronics symbols in circuit diagrams, chemical structure sketches. Speculates on combining pointing gestures and speech input commands ("Delete that"), multi-touch gestures, etc. [PacchierottiC12a] Pacchierotti, Claudio; Chinello, Francesco; Malvezzi, Monica; Meli, Leonardo; and Prattichizzo, Domenico "Two finger grasping simulation with cutaneous and kinesthetic force feedback", Proc. Eurohaptics '12, 2012 Int'l. Conf. on Haptics: perception, devices, mobility, and communication, pp. 373-382 Servo-actuated mechanical display to give haptic feel of a surface. Only works for one finger. [PanC13a] Pan, Caofeng; Dong, Lin; Zhu, Guang; Niu, Simiao; Yu, Ruomeng; Yang, Qing; Liu, Ying; and Wang, Zhong Lin "High-resolution electroluminescent imaging of pressure distribution using a piezoelectric nanowire LED array", Nature Photonics, 2013, published online August 11, 2013 (partial copy) Topical touchpad sensor using piezo-phototronic optical-emitting pressure sensors, forms full pressure map image of surface, light intensity changes with pressure. 6000 DPI resolution, 80 msecs. Electronic file also contains additional materials from Zhong Lin Wang: "Device for Capturing Signatures and Fingerprints Uses Tiny LEDs", Georgia Tech Research News. Gallium-nitride film, ITO film [PaperPrototyping12a] Snyder, Carolyn "Paper Prototyping: The fast and-easy way to design and refine user interfaces", PaperPrototyping.com, 2012 Companion website to "Paper Prototyping" by Carolyn Snyder for GUI design: references back to 1993. [ParkSM11a] Park, Su-Min; Lee, Kwangyoung; and Kyung, Ki-Uk "A New Stylus for Touchscreen Devices", Proc. 2011 IEEE Intl. Conf. on Consumer Electronics (ICCE), pp. 491-492 Touchscreen stylus for haptic feedback: electromagnetic coil in stylus tip, magnetic (ferrous?) sheet behind LCD display, magnetic force unter program control. Used to present sense of dragging when dragging objects, clicking, boundary lines to move over, textures (vibration). [PedersenEW11a] Pedersen, Esben Warming and Hornbaeck, Kasper "Tangible bots: interaction with active tangibles in tabletop interfaces", Proc. CHI 2011, Vancouver, Canada, May 2011, pp. 2975..2984 Tangible objects that respond to touch (touch tabletop) input and other gestures: on 2D table surface (camera touchscreen), haptic feedback by snapping to certain angles (on wheels), attraction or repulsion to movement by user. Command inputs by hand gestures. One example: simulate rotatable control knobs with resistance. [PeitzA11a] Peitz, Alexander "Konzeption einer Mehrbenutzerumgebung für eine multitouch-basierte PrototypingAnwendung", Diploma Thesis, Technische Universität Dresden, Germany, May 2011 ScrMT (Scribble MultiTouch) sketching web page layout with multitouch user interface. Prototype uses diffuse illumination (internal reflection) multitouch tablet surface. [Peratech12a] Peratech.com "QTC Quantum Tunnelling Composite Materials", Peratech.com, fetched 2012 composite elastomeric material for resistive touchscreens, resistance changes when under compressive force / pressure. Example applications are touchscreen whiteboards, touch-sensitive keyboards. [PetzoldC12a] Petzold, Charles "Programming Windows 6th Edition: Writing Windows 8 Apps with C# and XAML", Microsoft Press, 2012 Newer version of Petzold programming series for Windows: explains use of Tap gesture to describe click (mouse-button) events, Holding event (gesture) can simulate a right tap (WM_RBUTTON) but system generates both events. Mentions pinch gesture (pinch event). [PhilippH11a] Philipp, Harald and Brunet, Samuel "Touch Sensitive Screen", United States Patent 7,932,989, April 26, 2011 Two-layer capacitive grid touchpads for button and appliance controls [PhilippH11b] Philipp, Harald "Touchscreen Electrode Arrangement", United States Patent Application 2011/095997 A1, April 28, 2011 Two-layer capacitive grid touchscreen, electrodes are spaced with increasing density across the touchscreen, position determined by how many electrodes are affected by finger.



[PiezoSystems11a] Piezo Systems, Inc. "Introduction to Piezo Transducers", Piezo Systems Inc, Woburn MA Description of piezoelectric mechanical actuators, transducers, and sensors. Laminated construction increases mechanical movement. [PiezoSystems11b] Piezo Systems, Inc. "Piezo Systems Component Catalog", Piezo Systems Inc, Woburn MA Piezo systems transducers / actuators: parallel transverse compression torsion / mechanical flapper; bending sensors, resonant devices. Same element can be both sensor or generator. [PiezoSystems12a] Piezo Systems, Inc. "Introduction to Piezo Transducers", Piezo Systems Inc, Woburn MA Description of piezoelectric mechanical actuators, transducers, and sensors. Laminated construction increases mechanical movement. [PortraitDisplays12a] Portrait Displays, Inc. "Pivot Product Overview Flexible Display Orientation", Pivot.com, 2012 Rotatable display software driver for Windows (2000, XP, Vista), multiple simultaneous displays. Compare with Radius/Rotational Technologies. Compare with built-in feature of Windows for display orientation? [PoupyrevI12a] Poupyrev, Ivan; Harrison, Chris; and Sato, Munehiko "Touch�: Touch and Gesture Sensing for the Real World", Proc. UbiComp '12, ACM Conf. on Ubiquitous Computing, September 5-8 2012, Pittsburgh USA, pp. 536-492 Using a single capacitive touch sensor (not a touchscreen) and a swept frequency, get different profiles of touch for different grasps, contacts, body parts, due to different capacitive construction of parts of human body. Thus can instrument for gestures on any object (object used as sensor electrode) or conductive fluids (e.g. water). One example is grasping gestures on door knob as a form of biometric key. Disadvantage is profiles must be trained per-user, as users' bodies vary. May be combined with touchscreen for additional input degree. Can recognize hand gestures on liquids, hand gestures on user's own body (touching hands together at fingertips). Disney Research. Electronic file contains additional material from www.disneyresearch.com [PryorTR11a] Pryor, Timothy R. "Method for providing human input to a computer", United States Patent 8,072,440, December 6, 2011 Touchscreen (with rear projection), determine position optically by detecting Schlieren changes in polarization or other optical effects of force applied to (semi-)transparent screen. Multi-touch. Force/pressure by degree of distortion. Tactile feedback -- actual touching of surface -- or vibration, inflatable bladders, air blasts, etc. Separate section uses optical targets on user's body: glove, hat, etc. See also 7,973,773, 5,982,352. [Pyreos12a] Pyreos "Pyreos: User Guide -- Thin Film Pyroelectric IR Gesture Sensor Demonstrator Kit -- For low power, high performance gesture control", pyreos.com, February 2012 "Touchless Gesture control": Passive infra-red optical video sensor for detecting hand and body gestures. 4x4 sensor pixels: able to distinguish broad swipe/flick gestures of hand above surface. [RaabF13a] Raab, Felix; Wolff, Christian; Echtler, Florian "RefactorPad: Editing Source Code on Touchscreens", Proc. EICS '13, June 24-27, 2013, London, pp. 228-228 Adding touch gestures to context-sensitive source code editing in IDE: user study of what gestures user's preferred: predominantly single hand, multi-touch up to two fingers, preferences for stylus (more accurate) than finger, two buttons on stylus used as mouse left/right buttons. Suggests adding pen input to touch interfaces. [RajaMA13a] Raja, M.A.; Arunya, A.; Tadav, Dipti; Kanitha, G.; and Maheshwari, R. "Portable Refreshable EBraille", Int'l Jnl of Engineering Research and Applications IJERA, Wol 3, No 1, Jan/Feb 2013, pp. 1393-1397 Braille book display supporting eBooks in multiple formats. Also display SMSs from smartphone via bluetooth. Small braille cells: two in prototype for small size to match smartphone. Arduino processor, capacitive touch sensor / touchscreen for scrolling gestures. Cites to references for gesture/pen/touchscreen devices for visually impaired/blind. [RakRP12a] Rak, Roman P.; Griffin, Jason T.; and Ladouceur, Norman M. "Ramped-key keyboard for a handheld mobile communication device", United States Patent 8,162,552, April 24, 2012 Blackberry keyboard for two-thumb thumb typing: rounded keys, full alphanumeric layout at bottom of display [RedHat13a] Red Hat "Red Hat Enterprise Linux 6 Storage Administration Guide", RedHat.com, 2013 Administrative manual for Red Hat linux. External file systems can be mounted either with ACL or linux traditional file permissions for access control. On traditional, "effective rights" are OR of user/group/world permissions. [RekimotoJ13a] Rekimoto, Jun "Traxion: A Tactile Interaction Device with Virtual Force Sensation", Proc. UIST '13, October 8..11, St. Andrews, U.K., pp. 427..431 Haptic transducer gives illusion of virtual force, asymmetric vibration: one direction strong enough to be perceived, other direction not. Paired transducers to cancel mechanical vibration. [ReynoldsJK11a] Reynolds, J. Kurth "Methods and Systems for Detecting a Position-based Attribute of an Object using Digital Codes", United States Patent 7,868,874, January 11, 2011



Capacitive digitizer tablet with X/Y matrix grid, CDMA-like / spread-spectrum modulation of the signal to filter out noise and other effects. Compare with syncronous demodulation in Blesser, Rekimoto? [RizzoT12a] Rizzo, Tom "Programming Microsoft's Clouds: Azure and Office 365", Wrox Programmer to Programmer, John Wiley and Sons, Inc., 2012 Additional authors. Microsoft hosted services. Windows Azure cloud VM virtual machine environgment: Office 365 SharePoint document sharing, Exchange email services, Lync integrated messaging services. [RotoView13a] Innoventions, Inc. "RotoView: Tilt and Scroll User Interface / Smartphone Display", Innoventions.com, 2013 tilt-and-scroll gestures of handheld devices, PDAs, smartphones. Using accelerometer/gyroscope sensors in device (may be added), user can tilt device to scroll a small display to show a larger virtual display. Or acceleration of hand movements. Tilt, scroll, throw, glide gestures. Exit navigation mode to fixed mode when device held steady for an inactivity/dwell time. Non-linear dynamic response (NLDR) to tilt. Electronic file contains additional materials. [RoudautA11a] Roudaut, Anne; Pohl, Henning; and Baudisch, Patrick "Touch Input on Curved Surfaces", Proc. CHI 2011, May 7-12, Vancouver Canana, pp. 1011-1020 Study of touch/stylus accuracy on curved/spherical surface: it is worse than on flat touchscreens/tablets, for a variety of reasons. Convex/Concave surface gives less/more contact touch area. Prototype used clear plastic/elastomeric with FTIR camera to detect touches. File includes video presentation. [RoudautA13a] Roudaut, Anne; Rau, Andreas; Sterz, Christoph; Plauth, Max; Lopes, Pedro; and Baudisch, Patrick "Gesture Output: Eyes-Free Output Using a Force Feedback Touch Surface", Proc. CHI 3012, April 27-May 7, 2013, Paris. For eyes-free user interface (e.g. in hand-held or desktop device), display gestures as output by moving the top surface of a the touchscreen/touchpad used for user input. Haptic, but not vibration, instead physically move a top surface a small distance (1cm..4cm) in X/Y/theta rotation using mechanical actuators. Suggests using mnemonic gestures, used same one-stroke gestures for input and output (i.e. graffiti), so that users learn one vocabulary of gestures: helps users learn Graffiti unistroke input. Numerous citations to other work in same area. [RoudautA13b] Roudaut, Anne; Karnik, Abhijit; Loechtefeld, Markus; and Subramanian, Sriram "Morphees: Toward High Shape Resolution in Self-Actuated Flexible Mobile Devices", Proc. CHI 3012, April 27-May 7, 2013, Paris. Haptic output by output device changing shape: flat surface that bends (prototype uses pull wires in back), change in curvature, area, porosity,stretchability, etc. Compare with Gesture Output paper same conference. File includes video presentation. [RubinM12a] Rubin, Moshe; Matitya, Moshe; Melnick, Artem; Touboul, Shlomo; Yermakov, Alexander; and Shaked, Amit "Method and system for adaptive rule-based content scanners", United States Patent 8,225,408, July 17, 2012 lexical-rule based scanning of content to block malware download: compare with Cylant/Hezi firewall rules with pattern matching? [SahooSK12a] Sahoo, Soyuj Kumar; Choubisa, Tarun; and Prasanna, S. R. Mahadeva "Multimodal Biometric Person Authentication: A Review", IETE Technical Review, Vol 29 No 1, January-February 2012, pp. 54-ff (partial copy) Biometrics work better than passwords, may be re-used securely. Gives error rates for several biometric devices/systems. Suggest using multiple methods integrated. [SampK11a] Samp, Krystian "The Design and Evaluation of Graphical Radial Menus", Ph.D. thesis, Digital Enterprise Research Institute, National Univ. of Ireland, Galway, December 2011 Graphical radial menus / marking menus, in terms of readability and minimizing visual search, versus as a guide to making marks of marking menus. Hierarchical/cascading pie menus may overlap (small spacing OPM) or not overlap (larger spacing NPM). Concentric rings of radial menus -- inner and outer menus. Mouse button clicks work better than using dwell time / dwelling (compare to press-and-hold). [SatoM12a] Sato, Munehiko; Poupyrev, Ivan; and Harrison, Chris "Touch�: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects", Proc. CHI '12, May 5-10, 2012, Austin, Texas, USA, pp. 483492 Using a single capacitive touch sensor (not a touchscreen) and a swept frequency, get different profiles of touch for different grasps, contacts, body parts, due to different capacitive construction of parts of human body. Thus can instrument for gestures on any object (object used as sensor electrode) or conductive fluids (e.g. water). One example is grasping gestures on door knob as a form of biometric key. Disadvantage is profiles must be trained per-user, as users' bodies vary. May be combined with touchscreen for additional input degree. Disney Research. [SayyadiB12a] Sayyadi, Babak; Wu, Amandu; Manalo, Gil; Flick, Kevin; Fuelling, Sarah H.; Becker, Michael; Shetty, Santosh; and Lane, David M. "Pointing Device with Customization Options", United State Patent 8,094,124, January 10, 2012



Mouse with optional/replaceable mechanical and firmware features: eg. skates for better gliding/friction on different surfaces, additional buttons/scrollwheels/display elements. Firmware in mouse customizable (e.g. dynamic/accelleration sensitivity for particular users). [ScheurichD13a] Scheurich, Doug and Stuerzlinger, Wolfgang "A One-Handed Multi-Touch Mating Method for 3D Rotations", Proc. CHI 2013 Extended Abstracts, April, 2013 On touchscreen, one-hand gestures for rotating/moving a 3D virtual object to be on another. One finger to drag/move, two fingers, second finger rotates object around implied "Z" axis (no zoom, these are simulated real-world objects). One finger anchor, move left/right below first anchor to rotate around X axis. Hold with tap by second finger to move: object snaps to virtual surface, then user may rotate in plane of the virtual surface. [SchmidtDH12a] Schmidt, Darlene H. ed.; with Ward, Betty Jeanne; Small, Roxanne and Erbaugh, Julie "Building Better Learners: The Snapp Approach", Trafford.com publishers, 2012 Physical-therapy techniques of Ed Snapp for neurological development in younger children. Compare with "patterning". [SegevBarM13a] Segev-Bar, Meital; Landman, Avigail; Nir-Shapira, Maayan; Shuster, Gregory; and Haick, Hossam "Tunable Touch Sensor and Combined Sensing Platform: Toward Nanoparticle-based Electronic Skin", ACS Appl. Material Interface, June 4, 2013, Vol 5 No 12, pp. 5531-5541 Force/pressure sensing touch detector using nanoparticle flexing force sensors. Described as electronic skin technology vs. touchpad surface. Electronic file contains additional materials. [Semtech12a] Semtech Corporation "Semtech Touch Interface Solutions Selection Guide", Semtech Corporation, 2012 Semtech capacitive sensors: SAR Specific Absorption Rate sensors, for lowering RF power when a human body part is near versus some other material. 8-channel touchbutton controller, 12 channel matrix controller for 6x6 button array, Touch sensor with integrated haptic controller (vibrating motor), multitouch resistive (?) touchscreen. Detects human/inanimate touch by profiles for specific absorbtion rate / SAR for differnt materials. Electronic file contains further references on Touch Interface Solutions, Digitally Enhanced Human Sensing, capacitance-to-voltage and capacitance A/D conversion components, permittivity of common substances (glass, acrylic, wood, air, etc.). Sensor electrodes for proximity sensing circle with surrounding ring. [Semtech13a] Semtech Corporation "Semtech SX8674 Haptics Enabled Multitouch 4/5-Wire Resistive Touchscreen Controller with Proximity Sensing Semtech", Semtech Corporation, 2013 Semtech resistive-sheet four-wire touchscreen controller with multi-touch input and proximity sensing (?), haptic output built into controller. Electronic file includes datasheet, information on evaluation kit. [SeoJK12a] Seo, Joon-Kyu; Kim, Jyun-Jin; Lee, Ju-Youn; and Park, Min-Kyu "Apparatus including multiple touch screens and method of changing screens therein", United States Patent Application 2012/0306782, December 6, 2012 Combining multiple (multi-touch) touchscreens together into one continuous touchscreen: fold-out configuration, compare with Reefalo etc. Multi-screen gestures. [SilverbrookK11a] Silverbrook, Kia and Lapstun, Paul "Optical Imaging Device with Motion Sensing Accelerometers", United States Patent 7,908,661 B2, March 15, 2011 Electronic/optical digitizer pen (similar to Anoto?) including accelerometers to detect motion across page without relying on optical input. Mentions Sekendur technology. Examiner cited to this bibliography by Jean Renard Ward on prior art, 1995-2000. Continuation of 2000 patent application. [SimmonsJP11a] Simmons, Joseph P.; Nelson, Lief D.; and Simonsohn, Uri "False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant", Psychological Science, Vol 22 No 11 pp 1359-1366, 2011 Research results (not just psychological studies) biased towards false-positive: flexibility in data collection, determining end of data collection only after initial results seen, analysis determined after-the-fact, choice of what results to publish (positive results more likely to be published). See also Nagy/Candide, Gelman, Ioannids. [SinclairM13a] Sinclair, Mike; Pahud, Michel; and Benko, Hrvoje "TouchMover: Actuated 3D touchscreen with haptic feedback", Proc. ITS '13, October 6-9, 2013, St. Andrews UK pp 287-296 force-feedback haptic feedback for three-dimensional 3D shapes: user pushes on display, display pushes back according to 3D shape of object underneath. Experimental system, simple 3D shapes -- cone, cylinder, convex and concave pyramids, etc. [Sitepoint12a] SitePoint.com ":focus (CSS Selector)", www.sitepoint.com, fetched 2012 Definition Input Focus: attribute that matches any element that has keyboard input focus [SmartTech11a] SMART Technologies "SMART digitizer product information", www.smarttech.com SMART board interactive whiteboard digitizer: SMART Board 885ix interactive whiteboard system, SmartTablet interactive learning center, similar to Microsoft Surface or Mitsubishi interactive table.



Interactive learning centers, interactive pen displays, document cameras (compare with DigitalDesk research unit from approx. 1992?) [SmartTech11b] SMART Technologies "SMART Table interactive learning center", www.smarttech.com multi-touch (multi-user?) tabletop tablet product, rear-projection, marketed for educational applications. Includes whiteboard system, SMART document camera for scanning in printed materials. [SmartTech12a] SMART Technologies "Specifications Smart Board interactive whiteboard system Model X885ix885ix-SMP", www.smarttech.com multi-touch whiteboard product, up to four touches. Says can be multi-user (two users) at up to touch touches each, does not say how/whether users are identified. [SmithAM11a] Smith, Adam Miles; Dudkowski, Eric Edward; Lietzke, Matthew P.; Locker, Howard; Skinner, Jeffrey E.; Stewart, Aaron Michael; and VanBlon, Russell Speight "Behavior and Appearance of Touch-optimized User Interface Elements for Controlling Computer Function", United States Patent Application 2011/0029904 A1, February 3, 2011 Tiled user interface for pen computer (or touchscreen): simulate physics on tiles when user moves, flicks, or drags them: effects include larger objects moving more slowly (inertia), gravity/attraction between objects (snap to join, unjoin. Initial trajectory taken from user gesture. Properties may be different for different objects (parameterized). Compare with Artificial Reality project? [SongH11a] Song, Hyunyoung; Benko, Hrvoje; Guimbretiere, Francois; Izadi, Shahram; Cao, Xiang; and Hinckley, Ken "Grips and Gestures on a Multi-Touch Pen", Proc. CHI 2011, May 7-12, 2001, Vancouver, BC, Canada, pp. 1323-1332 Touch sensors on pen/stylus in tablet digitizer, e.g.: holding stylus in different grips gives ink/paintbrush/highlighter; tapping stylus with index finger after lasso makes it selection vs. ink marking; flick (versus ink) gesture only with flat grip. Cites to other work on pen/stylus rolling, back-of-device touchpad/PDAs. Cites to other work on pen/stylus rolling, back-of-device touchpad/PDAs. Cites to other work on pen/stylus rolling, back-of-device touchpad/PDAs. Cites to other work on pen/stylus rolling, backof-device touchpad/PDAs. [StallingsW12a] Stallings, William "Operating Systems: Internals and Design Principles, 7th Ed.", Prentice Hall, 2012 upper-level textbook on operating systems/OS design and architecture. Covers UNIX SVR4, Windows 7, Mac OS X, Solarit, Linux. Concurrency mechanisms (mutex etc.), security (access control lists), cluster servers, B-trees in file indices. Historical section on Windows available separately (See files). Networking, process management, queuing on-line only. [Stantum12a] Stantum "Stantum: Unlimited Multitouch - Latest News Zforce touch product", stantum.com, 2012 Combined multi-touch finger-touch touchpad and stylus digitizer in one sensor: distinguish by contact size and peak? iVSM interpolated voltage sensing matrix. [StefikMJ13a] Stefik, Mark J. and Pirolli, Peter L. "System and method for rendering digital content in accordance with usage rights information", United States Patent 8,370,956, February 5, 2013 DRM usage rights management, usage rights control what rendering/use is granted to a document or object. Destination system must also be authorized/trusted. [StefikMJ13b] Stefik, Mark J. and Pirolli, Peter L. "System and method for distributing digital content to be rendered in accordance with usage rights information", United States Patent 8,393,007, March 5, 2013 DRM usage rights management, usage rights control what rendering/use is granted to a document or object. Rights sent as separate step. Destination system must also be authorized/trusted. [StephenJ11a] Stephen, Joseph and Damery, Eric "Custom Labeler for Screen Readers", United States Patent 8,028,240, September 27, 2011 JAWS: user can define custom labels for UI elements in non-accessible user interfaces. [StroustrupB13a] Stroustrup, Bjarne "The C++ Programming Language, Fourth Edition", Addison-Wesley, June 2013, ISBN 978-0-321-56384-2 Standard for C++ object-based programming language. Includes Standard Library, Standard Template Library (STL), Abstraction mechanisms [Synaptics11a] Synaptics Incorporated "Synaptics PS/2 TouchPad Interfacing Guide", synaptics.com, 2011 Synaptics touchpad with built-in detection of tap and glide/slide gestures for simulating PS/2 mouse buttons. tap-and-slide (two-part) gesture for "glide" of first mouse button. [Synaptics12a] Synaptics Incorporated "Synaptics: Intelligent Touch for Intelligent Devices: Product Briefs", synaptics.com, 2012 Multitouch transparent capacitive multi-touch touchscreens: ClearPad 200 Product Brief for smartphones, displays up to four inches. ClearPad 2300 Product Brief single-layer sensor-on-lens for smartphones. ClearPad Series 7 7020, 7300, 701 large-format (12 inches) [Synaptics13a] Synaptics Incorporated "Synaptics ClickPad Solutions", synaptics.com, 2013 Multitouch transparent capacitive multi-touch touchscreens: ClickPad 1.5 and 3.0, driver implements some



simple gestures (double-tap, tap-and-slide) ClickSmart button support to eliminate need for physical mouse buttons on a laptop touchpad. Accidental Contract Mitigation ACM for palm-rest problem. [Synaptics13b] Synaptics Incorporated "Synaptics ForcePad", synaptics.com, 2013 (fetched 2015) force/pressure sensings capacitive multi-touch touchscreen. Series 3 appears to use area of contact of finger ("Full Image Sensing / Image Sense Pressure"), Series 4 states 4 pressure (force) sensors. Suggests pressure for mouse-button, or speed of action in gestures, etc. Driver/firmware with gesture recognition / force-enhance gestures built-in: three-finger flick, twist/rotate, edge swipes, two-finger scrollen. Example: one finger click, two finger is drag. Compare with Rekimoto? [Synaptics13c] Synaptics Incorporated "Synaptics Tag Archives: 3D-touch / Hands-On with AirView on The Samsung Galaxy S4", synaptics.com, December 18, 2013 Capacitive multi-touch touchscreen, senses proximity of fingers (finger hover), used in Samsung Galaxy S4 smartphone touchscreen. Also video demonstration file: shows hover for pop-up menus/details, finger over scrollbar in movie to show highlight: similar to mouse-over. [Tactilus13a] Sensor Products "Tactilus Technology: Contact Surface Pressure Mapping", www.sensorprod.com, 2013 piezoresistive force/pressure-sensitive grid touchscreen, very thin. Used for pressure mapping: up to floor mat size, 80x35 inches. Includes software for filtering, rendering visual detail. Can be stretchable using appropriate materials. Compare with other piezeoresistive touchscreen technology. [Tactus12a] Tactus Techology "Taking Touch Screen Interfaces Into a New Dimension", Tactus Technology, Inc. 2012 Touchscreen static haptic: microfluidic bubbles under surface can be raised to give tactile feel buttons on touchscreen, or other surface (car door) [TatmanT11a] Tatman, Thomas "Bassoon Reed with Tubular Undersleeve", United States Published Patent Application 2011/0117044, Jan. 27, 2011 Bassoon reed with cylindrical metal sleeve inside throat where it fits on the bocal. Additional information in file for www.sleevereed.com [Teamboard13a] Teamboard, Inc. "Teamboard product information T4 is Multitouch, etc.", teamboard.com, 2013 Product information on Teamboard multi-touch large format physical whiteboard: may also be used as a front-projection surface. "Multiforce compliant", touch pressure sensing. Refers to 25 ms response time (driver latency included?) [TechSpot12a] Vilches, Jose "Synaptics rethinks laptop touchpad, keyboard with pressure sensing", TechSpot.com, August 20, 2012 Synaptics multi-touch capacitive touchscreen, using pseudo-force sensing for force gestures: mouse click by pressing harder, force scrolling by pressing one finger harder than another, etc. [TheEngineer11a] Harris, Stephone "Acoustic touchscreens could enable cheaper smartphones", The Engineer (UK), 18 April 2011: available at www.theengineer.co.uk TouchDevice, acoustic digitizer determines where user has touched a smartphone screen using existing smartphone microphone. Can be applied to entire device, or to a table or other object. States that mechanical buttons can be replaced by buttons that make a unique clicking sound, to be picked up by microphone. [ThomasD11a] Thomas, Dave "Celebrating 25 Years of Smalltalk: Travels with Smalltalk", www.mojowire.com, fetched 2011, reprinted from "Celebrating 25 Years of Smalltalk" History of Smalltalk from Xerox Parc in 1970's: mentions Momenta using Smalltalk for pen computing. Cites Electronic Boo, ARM-based Smalltalk-80 Pen computer in 1980's. [Thumbscript11a] Thumbscript.com "Thumbscript: Universal Text Input for Small Devices", www.thumbscript.com fetched 2011 Unistroke recognition using 3x3 zone-based input. Can be implemented in a very small touch screen input (size of an iPod Micro), or on a 3x3 sets of buttons from the telephone handset of a mobile call phone (not necessarily a SmartPhone). [TouchInternational12a] Touch International "Touch International Technical Literature: Resistive, Capacitive, and other touchscreen technologies", www.touch-intl.com, fetched 2011 Touch International tutorials and technical descriptions on touchscreen and touchpad technology and fabrication. [TouchInternationalBlog11a] TouchGuy "Ask Touch Guy Blog", www.touch-intl.com/toughblog?p=172, etc. 2011 Blog by Touch International on touchscreen manufacturing and fabrication. [TouchTable11a] TouchTable Inc. "TouchTable TT32 System T8-0078-TT32-Datasheet-v1.0, product information", TouchTable.com, 2011 Product information on multi-touch tabletop/whiteboard touchscreens. General configuration is wall-mount, or angled as a group drafting/collaborating desk. [Touchfire13a] TouchFire.com "Touchfire Screen-Top iPad Keyboard: How It Works", www.touch.com, fetched



2012

Touchscreen physical keyboard, transparent flexible/elastomeric cover for iPad (also iPod/iPhone) with raised dots at positions of the standard OS virtual keyboard keys. Accessibility aid for the blind, especially on the iPhone/iPod. [TransparentConductiveFilms12a] (various) "Transparent conductive films", (various) File of vendor information on transparent conductive film products. [TransparentConductiveInks12a] (various) "Transparent conductive inks", (various) File of vendor information on transparent conductive ink products. [TrewinS13a] Trewin, Shari; Swart, Cal; and Pettick, Donna "Physical Accessibility of Touchscreen Smartphones", Proc. ASSETS '13 15th International ACM SIGACCESS Conf. on Computers and Accessibility, Article No. 19 Evaluation of accessibility for dexterity/physical impaired/handicapped better on tablets than on smartphones as portable computer and information devices: extra-long touches (Android); single-button to display menus instead of multi-touch gestures on touchscreen (Apple iPad), etc. [ULIS13a] Buckley, Paul "New generation of thermal sensor arrays overcome the limitations in single-element sensors", EE-Times, May 15, 2013 Press report on Micro80P infrared/thermal sensor array, part of MIRTIC Micro Retina Thermal Infrared project. 80x80 array, 50Hz frame rate at room temperature. [Uniloc11a] Uniloc.com "Intellectual Property: the Uniloc Patent Platform Story", www.uniloc.com, fetched January 2011 Copy-protection / anti-piracy patent 5,490,216 on generating license code from information of system on which software will execute: litigation involving Microsoft. File contains additional materials. See also Bluecava file. [VanEeAM12a] van Ee, Anne Marijn "Touch-Based Organization of Patent Collections", Master's Thesis, TU Delft, Faculty EEMCS, August 28, 2012 Experimental system for patent document examiners on Microsoft Surface: Documents organized in stacks (compare Freestyle?). Based on earlier TouchPat system. PaperCube displays structural relationship of related patents / patent families. PushLens to move edges of graphs out of the way, alignment guides for positioning. Gestures include specific one/two/three finger gestures (not by shape) for semantic organization of patents. Gestures pinch/zoom. Hexagonal pop-up context menus on one-finger-hold (press-and-hold), double-tap for semantic zoom, flick to scroll through figures, lasso selection. Display of citation links for related documents. LAMP Local Affine Multidimensional Projection to position data items automatic layout. [VermaA11a] Verma, Amit "SBIT National Seminar: Capacitive Touch", Shri Balwant Institute of Technology, India, CSE.06.204, 2011 Overview of capacitive touch sensor/touchscreen technology: transcapacitance (mutual capacitance) and absolute capacitance (self capacitance). States that Apple iPhone/iPad multitouch touchscreen may use either (?), also that it uses a flexible transparent surface with spacer dots (?) (resistive?). Also Microsoft Surface (?). Reference to Sony Ericsson Motion Sensing Earphones as using capacitive touch sensing. [VertanenK13a] Vertanen, Keith; Memmi, Haythem; Kristensson, Per Ola "The feasibility of eyes-free touchscreen keyboard typing", Proc. ASSETS '13, 15th Intl. ACM SIGACCESS Conf. on Computers and Accesibility, Article 69 Touchscreen typing by recognizing approximate pattern of finger touches without exact physical keyboard. Notes that error rate still high. [VetterST12a] Vetter, Sebastian Timm "Theoretische und experimentelle Analysen zum Fitts'schen Gesetz bei der Interaktion mit großflächigen berührungssensitiven Bildschirmen", Ph.D. Thesis, Rheinisch-Westfälischen Technische Hochschule Aachen, 06.11.2012 (in German) Study of Fitts Law on interactions with large touch-sensitive surfaces / touchscreens, e.g. DiamondTouch, Microsoft Surface 2.0.a States that large touch tablet on a table surface currently has more application uses than multi-touch (compare with iPhone/iPod?) [VollmerA11a] Vollmer, Alexander "Schriftliche Ausarbeitung zur GFS mit dem Thema Apple", Nicolaus-KistnerGymnasium, Mossbach Germany, May 5, 2011 (in German) Historical notes on Apple products and business. Apple Newton PDA in 1993, iPhone in 2007, iPad 2010. [VuT12a] Vut, Tam; Baid, Akash; Gao, Simon; Gruteser, Marco; Howard, Richard; Lindqvist, Janne; Spasejevic, Predrag; and Walling, Jeffrey "Distinguishing Users with Capacitive Touch Communication", Proc. MobiCom '12, August 22-26 2012, Istanbul, pp 197..208 Touchscreen authentication of user on capacitive touchscreens: user wears an electronic signet ring generating an idendification/authentification binary signal. Shows sample circuit with body resistance, capacitive coupling to touchscreen electrodes. Ring generates strong signals, touchscreen electronics detect as (artificial) touch events in a known pattern. Low data rate still suitable for distinguishing multi-users on touch surface.



[VukosavljevicM12a] Vukosavljevic, Milan; Dresevic, Bodin; Letic, Nikola; and Predovic, Goran "Editing 2D structures using natural input", United States Patent 8,213,719, July 3, 2012 Mathematical expression editing: render typed mathematical expressions to idealized electronic ink (simulated writing) representation, user can edit while in ink form, convert back to text fonts when done. Refers to tablet PC, touchscreen. Cites to Forcier. Compare with van Raamsdonck? [WachsJP11a] Waches, Juan Pablo; Koelsch, Mathias; Stermn, Helman; and Edan, Yael "Vision-Based HandGesture Applications", CACM, Feb 2011, Vol 54 No2, pp. 60-71 Visual-detected hand-gesture UIs, mentions medical applications since no physical touching to contaminate surgeon's hands. Extensive bibliography to WII controller, Virtual Reality UIs, hand-held interface devices, etc. Electronic files include demonstration video. [Wacom11a] Wacom, Inc. "Wacom Inkling portable digitizer", Wacom.com Acoustic/infrared digitizer (25 Khz to 75 Khz), clips to top of paper with special pen, drawings stored in pen and can be uploaded to computer. Compare with Anoto LiveScribe, which uses patterned paper (and can identify which sheet from the pattern). [Wacom11b] Wacom, Inc. / Softpro "Wacom Intuos 4 Technische Daten - Grafiktablett - optional mit TintenSpezialstift", Softpro.de - The Signature Professionals Technical specifications for Wacom Intuos 4 electromagnetic tablet (in German): 200 pts/second, 2048 pressure levels ("non-interpolated"), no battery in stylus. Intended for signature capture (A6 size) [WagnerJ12a] Wagner, Julie; Huot, Stephane; and Mackay, Wendy E. "BiTouch and BiPad: Designing Bimanual Interaction for Hand-held Tablets", Proc. CHI 2012, Session Triple T: Touch, Tables, Tablets, Austin Texas, May 5-10, 2012 pp 2317-2326 Experimental system for using both hands (thumb and fingers of supporting hand holding the device) for slate/tablet computers, versus bimanual systems where tablet/tabletop is supported mechanically. Which corner being held can determine landscape/portrait, tap (press-release) gesture of supporting hand thumb, press-hold for on-screen shift key, direction of thumb swipe. [WalkerG11a] Walker, Geoff "Short Course S4: Fundamentals of Touch Technology and Applications", Tutorial, IMS Research, May 15, 2011 Tutorial on touchscreen technologies: includes list of 15 performance tests -- compare with Ward article from 1987/1985, Wang/Freestyle 1990? Cites to Apple patent for multi-touch -- does not mention Fingerworks. [WalkerG11b] Walker, Geoff "SID Display Week 2011 Review: Touch Technology", SID Information display, Vol 27 No 08, July/August 2011 Report on SID display week: first time projected capacitive multi-touch touchscreen technology common. Technology exhibited also analog resistive, multi-touch resistive, SAW, Infrared, electromagnetic digitizer, in-cell LCD touch (optical>?), DST dispersive acoustic, APR acoustic pulse, camera-based optical, forcesensing. "Intentional overscan margin" by Perspective Pixel: compare with Wang Guide? Wacom large-size multi-touch capacitive, N-Trig dual-mode touch/stylus. Hanvon dual-mode (pen/finger) with EMR digitizer and force sensing touch. F-Origin force sensors at corners of display. [WalkerG12a] Walker, Geoff "SID Display Week 2012 Review: Touch Technology", SID Information display, Vol 28 No 09, September 2012 Report on SID display week: in-cell touchscreen integration "becomes real": Synaptics, Innolux. [WalkerG13a] Walker, Geoff "Fundamentals of Touch Technologies", Course notes, Short course, SID Display Week 2013 Geoff Walker's ongoing seminar on touchscreen sensing technologies. Cites to Buxton, but no mention of Anoto, or 1990's SID publications. [WalkerG13b] Walker, Geoff "Status and Future of Touch Technologies", FDP China 2013, Shanghai, March 21, 2013 Report on touch technologies, focus on manufacturing and materials. ITO replacement materials mostly about using existing manufacturing processes (from LCD production), not materials: Metal mesh, silver (AG) nanowires, carbon nanotubes -- less so conductive polymers, graphene [WalkerG13c] Walker, Geoff "Display Week 2013 Review: Touch Technology", Information Display, May 2013 pp. 8-12 Review of touchscreen industry 2013: dominant tablet technology is electrostatic proximity: p-cap. Review Pixel Systems, Cima silver mesh as replacement for ITO electrodes. Report on touch technologies, focus on manufacturing and materials. ITO replacement materials mostly about using existing manufacturing processes (from LCD production), not materials: Metal mesh, silver (AG) nanowires, carbon nanotubes -less so conductive polymers, graphene [Wanderbook11a] Wanderbook "EO Personal Communicator 2: Apple tech songster extraordinare", www.wanderbook.org/blog, fetched 7/6/2011 Historical notes comparing Penpoint/EO with Apple iPad, includes PC Week (?) press clipping (1993?) on



tablet-sized Newton, 3lb, 8.5x11 inch, codenamed The Bic. [WangGZ13a] Wang, Guo-Zhen; Huang, Yi-Pai; and Chen, Tsu-Han "Bare Finger 3D Air-Touch System Using an Embedded Optical Sensor Array for Mobile Displays", To appear in IEEE/OSA Jol. of Display Tech, July 2013 Optical in-air touch/finger sensing (3D touchscreen) using infra-red illumination from sides of surface, determine Z value in part from amount of light reflected. Optical sensors integrated into pixels of display. Cites to Brown 2010 SID paper. [WangS13a] Wang, Shunhua; Zhang, Xu; and Zhao, Weiwei "Flexible, Transparent, and Conductive Film Based on Random Networks of Ag Nanowires", Journal of Nanomaterials, Vol 2013, Article 456098, 22 March 2013 Silver nanowire conductive/resistive film as alternative to ITO: ITO brittle, cracks. Cites to conducting/conductive polymers, graphene films. Production methods include solution precipitation, spin coating, drop casting, rod coating, air spray. Mosly concerned with use as transparent heating electrode element on glass. Compare with Cambrios silver-nanowire touchscreens. [WeibelN11a] Weibel, Nadir; Signer, Beat; Morrie, Moira C.; Hofstetter, Hermann; Jetter, Hans-Christian; and Reiterer, Harald "PaperSketch: A Paper-Digital Collaborative Remote Sketching Tool", Proc. IUI '11, February 13..16, 2011, Palo Alto CA, pp. 155-164 Whiteboard (multi-user) application using Anoto touchpad/pens, integrated with Skype. Most useful features were thos possible with paper sketching: simple drawing with colors, erase, attach/save image to email, etc. [WestermanW11a] Westerman, Wayne Carl and Strockon, Joshua "Bottom-up watershed dataflow method and region-specific segmentation based on historic data to identify patches on a touch sensor panel", United States Patent 7,916,126, March 29, 2011 Watershed algorithm for determining connected regions (touch regions) on a multi-touch digitizer. Small connected regions probably two separate fingertips, large connected regions may be thumb or palm touch. Compare to palm rejection, Rekimoto. Cites to Westerman 1999 Ph.D. Thesis, Rubine Gestures 1991. [Wikipedia11a] wikipedia.org "Event (synchronization primitive)", wikipedia.org Definition Event: wait/set/clear [Wikipedia11b] wikipedia.org "Event (computing)", wikipedia.org Definition Event: event-based programming, in particular interactive event-driven user interfaces such as Windows, with queueing [Wikipedia11c] wikipedia.org "Emulator (computing)", wikipedia.org Definition Emulator: one system programmed to duplicate behavior of (and appear to be) another [Wikipedia11d] wikipedia.org "Event (disambiguation page)", wikipedia.org Definition Event: software message, in particular interactive event-driven user interfaces such as Windows, with queueing [Wikipedia11e] wikipedia.org "Pinhole glasses", wikipedia.org Pinhole glasses/ stenopeic glasses: opaque glasses with multiple pinholes instead of lenses, tend to give sharper vision due to narrow field of view, similar to pinhole camera [Wikipedia11f] wikipedia.org "Dithering", wikipedia.org Definition Dithering: pattern of red and blue at fine scale appears violet/purple [Wikipedia11g] wikipedia.org "Ordered Dithering", wikipedia.org Definition Ordered Dithering: crosshatch patterns [Wikipedia11h] wikipedia.org "Halftone", wikipedia.org Definition Halftone: halftone patterns (with examples) [Wikipedia11i] wikipedia.org "Layout manager", wikipedia.org Definition Layout manager [Wikipedia11j] wikipedia.org "Standard Widget Toolkit", wikipedia.org Definition Standard Widget Toolkit in Java, distinct from Swing widgets [Wikipedia12a] wikipedia.org "Zooming User Interface", wikipedia.org Definition: Zooming User interface [Wikipedia12b] wikipedia.org "Focus / Input Focus", wikipedia.org Definition: Focus / Input Focus [Wikipedia12c] wikipedia.org "Scrolling", wikipedia.org Definition: Scrolling [Wikipedia12d] wikipedia.org "Touchscreen", wikipedia.org Definition Touchscreen: includes use of stylus [Wikipedia12e] wikipedia.org "Indium Tin Oxide (ITO)", wikipedia.org Definition indium tin oxide / ITO: best recommended material for transparent conductive/resistive, used in touchscreens, but getting expensive [Wikipedia12f] wikipedia.org "Transparent conducting film", wikipedia.org Transparent conducting films include ITO, other transparent conductive oxides such as fluorine doped tin



oxide (FTO), carbon nanotubes, transparent conducting polymers. [Wikipedia12g] wikipedia.org "Active Window", wikipedia.org Definition of Active Window with input focus. [Wikipedia13a] wikipedia.org "Capacitance", wikipedia.org Definition of Capacitance [Wikipedia13b] wikipedia.org "Capacitive Sensing", wikipedia.org Definition of capacitive sensing [Wikipedia13c] wikipedia.org "Mutual Capacitance", wikipedia.org Definition of Mutual capacitance versus self capacitance [Wikipedia13d] wikipedia.org "Virtual Retinal Display", wikipedia.org Direct projection display onto retina of eye, bypassing lens and cornea (and thus bypassing nearsightedness) [Wikipedia13e] wikipedia.org "Palm (PDA)", wikipedia.org History of Palm PDAs, including network wireless web browsing [Wikipedia13f] wikipedia.org "Palm VII", wikipedia.org Palm VII with Location Based Service and Palm.net wireless network services [Wikipedia13g] de.wikipedia. "Chipkarte", de.wikipedia.org Chipkarte / Chip card: invention in 1975, with PIN to control access to data. [Wikipedia13h] en.wikipedia. "PIN Personal Identification Number", en.wikipedia.org PIN / Personal Identification Number -- history, and algorithms [Wikipedia13i] en.wikipedia. "Crossbar switch", en.wikipedia.org Crossbar switch / non-blocking crossbar switch: see Kaplow patent on touchscreen with multi-touch. [Wikipedia13j] en.wikipedia. "Superimposition", en.wikipedia.org Superimposition: conceal [Wikipedia13k] en.wikipedia. "LCD projector", en.wikipedia.org LCD projector introduced in 1987 [Wikipedia13l] en.wikipedia. "Specific absorption rate", en.wikipedia.org Specific absorption rate SAR: rate of RF electromagnetic energy absorbed by human body. May be applied to ultrasound or other energy. Standards for safety/health levels for cellphones, MRI, medical instruments. [Wikipedia13m] en.wikipedia. "Alpha compositing / alpha blending", en.wikipedia.org, 2013 Alpha blending for combining images for transparency or semi-transparency: pixel-by-pixel color blending with percentages (alpha factor) for each of the two image pixel. [Wikipedia13n] en.wikipedia. "Bit blit", en.wikipedia.org, 2013 Bit blit / Bitblt bit-level block transfer: source/destination bitmaps, mask bitmap, ROP raster operation. Not equivalent to alpha compositing (alpha blending). Blitter dedicated/separate hardware. Origin Smalltalk-72 in 1975. Masked blit can be implemented with two BitBlt operations and AND and OR raster operation. [Wikipedia13o] en.wikipedia. "Stippling", en.wikipedia.org, 2013 Small discrete dots used to give the appearance of shading, no grayscale [Wiktionary11a] wiktionary.org "Simulate", Wictionary.org Definition Simulate [Wiktionary11b] wiktionary.org "Dither", Wiktionary.org Definition Dither: as a pattern [Wiktionary11c] wiktionary.org "Threshold", Wiktionary.org Definition Threshold: beginning point or level. (also dictionaryreference.com, www.merriam-webster.com, www.learnersdictionary.com) [Wiktionary13a] wiktionary.org "Superimpose/Superimposed", wiktionary.org Definition Superimpose/Superimposed [WintergerstG11a] Wintergerst, Goetz; Jagodzinki, Ron; Held, Matthias; Hemmert, Fabian; Mueller, Alexander; Joost, Gesche "Combining Active and Passive Haptic Feedback for Rich Stylus Interactions", Übermedien Übermorgen: Workshop-Proceedings der Tagung Mensch und Computer 2011, Technische Universität Chemnitz, Germany, pp. 7-11 Haptic stylus for touchscreens, has vibrating motor for active feedback (textures, etc.) and electromagnetic coil and steel ball for braking effects to increase friction where steel ball rolls for passive feedback (edges, textures on surface). [WintergerstG11b] Wintergerst, Goetz; Jagodzinki, Ron; Giles, Peter "Hap.pen: Sketching Haptic Behavior", Übermedien Übermorgen: Workshop-Proceedings der Tagung Mensch und Computer 2011, Technische Universität Chemnitz, Germany, pp. 13-14 Cordless haptic stylus/pen for use on touchscreen digitizers integrated with a display: haptic feedback is determined by color/brightness of what is displayed, photo sensor in tip of stylus. [WobbrockJO11a] Wobbrock, Jacob O. "EdgeWrite Text Entry", depts.Washington.edu/ewrite, June 2011



EdgeWrite unistroke text entry based on moving to edges (or corners) of synaptics touchpad: includes unistroke gestures for European characters (umlauts, accented), PC keyboard cursor keys, word space, trackball buttons, editing. [WoeldeckeB11a] Wöldecke, Björn; Marinos, Dionysios; and Geiger, Christian "Berührungslose Interaktion für Musikdarbietungen im virtuellen Studio", Workshop-Proceedings der Tagung Mensch und Computer 2011, T.U. Chemnitz, Germany, 2011 (in German) Multitouch surface touchscreen for musical input, user sees view of a semitransparent musicalnote wall (special musical keyboard layout) integrated with user's video image. Says works well in spite of limitations of the visual feedback. [WordNetWeb11a] WordNetWeb "Emulate (computing)", wordnetweb.princeton.edu Definition Emulate - imitate function as by modifying hardware or software [Wysips13a] Wysips.com "WYSIPS Technology: Glass Substrate", wysips.com, fetched 2013 Non-transparent touchscreen technology: glass substrate, with other embedded elements: RF antennas, electromagnetic shields. Micro lenses in micrometic patterned thin film materials allow user to see image behind embedded device. [XWinMan11a] chapman, Matt "Window Managers for X: A world of choice for your Linux/UNIX desktop", xwinman.org, fetched 2014 Resource list for X windows managers and desktops: FVWM, FVWM95, TWM/VTWM, MWM, CTWM, OLWM/OLVWM, wm2/wmx, AfterStep, AmiWM, Enlightenment, WindowMaker, SCWM, IceWM, Sawfish, Blackbox, Fluxbox, Metacity, Others... Desktops: GNOME, KDE, CDE, XFce, Others... Some examples back to 1999 and earlier. [XiaoPingJ11a] XiaoPing, Jiang "Two-pin buttons", United States Patent 8,004,497, August 23, 2011 Electrostatic/capacitive touch sensing: interpolate/extrapolate positions of more buttons than there are sensor elements, e.g. with two conductor sensors, sense over each sensor pad or in-between. Compare with Rekimoto, Kaplow? [XiaoPingJ13a] XiaoPing, Jiang "Apparatus and Methods for Detecting a Conductive Object at a Location", United States Patent 8,519,973, August 27, 2013 Electrostatic/capacitive touch sensing: interpolate/extrapolate positions of more buttons than there are sensor elements, e.g. with two conductor sensors, sense over each sensor pad or in-between. Compare with Rekimoto? [XiaoR13a] Xiao, Robert; Harrison, Chris; Willis, Karl D.D.; Poupyrev, Ivan; and Hudson, Scott E. "Lumitrack: Low Cost, High Precision, High Speed Tracking with Projected m-Sequences", Proc. ACM UIST '13, October 2013, St. Andrews, UK Motion tracking / gesture tracking at extremely precise detail: sub-millimeter, 1000 frames/second. Light projectors for bar-code like pattern, optical sensors on target pick up pattern, determine angular position from pattern. Two projectors patterns orthogonal give X/Y position, more projectors up to 3D 6DOF input. Compare m-Sequence to Anoto pattern (cited): all n-bit subsequences are unique. [YamaokaJ13a] Yamoaoka, Junichi and Kakehi, Yasuaki "dePENd: Augmented Sketching System Using Ferromagnetism of a Ballpoint Pen", Proc. UIST '13, October 2013, St. Andrews, UK X/Y positionable magnet under the surface of a tabletop to give haptic feedback to ferrous ballpoint pen. Touchscreen/tablet position sensing using infrared and/or ultrasonic sensors. Gestures for drawing lines (two points), copy-and-draw. Combination can function as telautograph (see), or haptic feedback (writing guides) to user. [YanX12a] Yan, Xiaoqi; Fu, Chi-Wing; and Hanson, Andrew J. "Multitouching the Fourth Dimension", IEEE Computer, Vol 4 No 9, September 2012 pp. 80-88 Study of using multi-touch tablet user interface for manipulating 3D projection of a 4D hypercube. Multitouch wins over mouse+keyboard. [YangXD12a] Yang, Xing-Dong; Grossman, Tovi; Wigdor, Daniel; Fitzmaurice, George "Magic Finger: AlwaysAvailable Input through Finger Instrumentation", Proc. UIST '12, October 7-10, 2012, Cambridge, MA, 10 pages "reversed" finger optical touchpad: small video camera and illumination source on fingertip with thimblelike mounting. Tracks relative motion, contact, and can also recognize textures of surface, so gestures (e.g. tap) can be contextual (e.g. tap on cell phone to silence a call), gestures crossing from one surface to another. May be able to recognize bar codes or identifying marks. Electronic files include video demonstration. Cites to Anoto. [YoonD13a] Yoon, Dongwook; Chen, Nicholas; and Guimbretiere, Francois "TextTearing: Opening White Space for Digital Ink Annotation", Proc. UIST '13, October 2013, St. Andrews UK, pp. 017-112 (Demonstration video not on file.) Pen/Tablet computer: Annotation for electronic ink, user can tear/move columns of text, etc. to create empty whitespace in electronic document to get a place for electric ink annotation. Compare with Knowlton Freestyle electronic ink editor, van Raamsdonk (neither cited). Tearing may be two-finger multitouch or bimanual: compare with transparentt tools by Bier (not cited).



[ZDNet11a] ZDnet.com "Up close with Sony's S1/S2 tablets (photos)", ZDNet.com, 4/26/2011 Sony Android tablet product, front-facing camera. S2 unfolds to have two screens that can be used as one, or display virtual keyboard on one (compare to Refalo) [ZhaiS12a] Zhai, Shumin and Kristensson, Per Ola "The Word-Gesture Keyboard: Reimagining Keyboard Interaction", CACM Vol 55 No 9, September 2012, pp. 91-101 Keyboard single-stroke gestures: draw multi-segment stroke with pauses or inflections/turns on the characters of the word. "word-shorthand gesture keyboard". Cites to ShapeWriter, Swype, SlideIT, T9 Trace, FlexT9, TouchPal, and Shark Text (2004). [ZhaiY13a] Zhai, Yan; Zhao, Guoying; Alatalo, Toni "Gesture Interaction for Wall-Sized Touchscreen Display", Proc. UbiComp Conf. on Ubiquitous Computing, Zürich, September 8-12, 2013, pp 175-178 Near-mode (touch) and far-mode (WII/Kinect 3D sensor) gestures in front of active wall. Use dwell (compare press-and-hold) for "pick" and "drop" gestures. [ZhengY13a] Zheng, Yi; Zhang, Qin; and Liu, Jing "Pervasive liquid metal direct writing electronics with roller-ball pen", Cornell University Archive, September 4, 2013 Conductive ink usable in roller-ball (regular) ink pen. See also IEEE article on similar topic. Gallium / Indium GaIn droplets. Citations back to 2000. [ZielkeTL13a] Zielke, Terry Lynn and Hanson, Derek W. "Scrolling Inertia", United States Patent 8,587,521, November 19, 2013 Mouse driver computes scrolling inertia (momentum/speed) based on mouse events, scrolling coasts after end of user input, with automatic decrease in inertia (decelleration). Motion stops when user moves scroll wheel again. [ZimmerM12a] Zimmer, Mark; Stahl, Geoff; Hayward, David; and Doepke, Frank "Three Dimensional User Interface Effects on a Display by Using Properties of Motion", United States Patent Application 20120036433, February 9, 2012 Determine user's location of eyes/gaze (direction of gaze) using front-facing camera on smartphone, or extrapolation from accelerometor, compass, gyrometer, GPS, etc. to project more realistic 3D threedimensional image on display. Touches on touchscreen may then be matched to virtual 3D objects long line-of-sight. [ZizziS13a] Zizzi, Stephen "User authentication system and method for encryption and decryption", United States Patent 8,359,476, January 22, 2013 Authenticate user using biometric device containing cryptographic keys, API access to device once authenticated: fingerprint scanner? biblio10 (Prev page)

Main Page







Main Page

(Next page) biblioxx

[AMS14a] AMS Ag "Product information, AMS Touchless Gesture Sensor", ams.com, fetched 2015 Reflective infra-red IR photodiode illumination, optical sensors recognizing hand gestures in front of display screen. Up/down/left/right north/south/east/west gestures, activated by proximity detection. Also detect proximity of smartphone to user's ear. Includes broadcast of 1D barcodes using LED (compare with Morse code). [AbleData14a] AbleData "Voice Output Program", abledata.com, fetched 2014 U.S. Dept. Edu. historical list of screen readers and voice output systems for visually impaired: 26 pages. Historial list, many products discontinued before 2002: AppleWorks Companion, DragonDictate, Vocal-Eyes, etc. [Aito14a] Aito Touch "Aito: Touch. Click. Feel. Product information on touch controllers", AitoTouch.com, 2014 Touchscreen/touchsurface controllers for touch panel overlays/underlays for "any material" (wood, glass, metal, fabric). Piezo single-layer disk (stacked?), for both sensing and haptic feedback actuator, including audio. No interpolation for position(?) [AleksandarP14a] Aleksandar, Pance "Transparent Electronic Device", United States Patent 8,890,771, November 18, 2014 Double/multiple transparent display screens, one is touchscreen: basically a transparent tablet computer. Show a transparent window in the screen, overlay graphics on real-world image for augmented reality applications. Video overlay possible. [AmentoB14a] Amento, Brian; Li, Kevin Ansia; Purdy, Kermit Hal; and Stead, Larry "Devices and methods for transferring data through a human body", United States Patent 8,908,894, December 9, 2014 Biometric authentication of users: send known acoustic signal through body, subtract original signal from received signal to get modification, identify user by unique modification/alternation to signal. e.g. at a doorknob to grant entry. Signal can also pass through two users (holding hands) to transfer data. (Acoustic, not capacitive coupling like at M.I.T., Rekimoto, etc.) [AnisimovAS14a] Anisimov, Anton S.; Brown, David P.; Miklada, Bjoern F.; O Suilleabhain, Liam, Parikh, Kunjal; Soininen, Erkki; Sonninen, Martti; Tian, Dewei; Varjos, Ilkka; and Vuohelainen, Risto "Printed Touch Sensors



Using Carbon NanoBud Material", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 200..203, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Lower electrical resistance than ITO, more transparent (0.6%); less reflected light than silver nanowires. [Anoto15a] livepdf.net (Anoto) "Live PDF - paper document sharing made simple", livepdf.net, fetched 2015 Anoto pattern smart-pen Live Pen application: Live PDF / Live Paper app prints PDF forms with anoto pattern, smart pen sensor can then capture "wet signatures" and handwriting. Works with @LiveAddress storage of document definitions. [AryalM14a] Aryal, Mukti; Geddes, Joseph; Seitz, Oliver; Wassei, Jonathan; McMackin, Ian; and Kobrin, Boris "Sub-Micron Transparent Metal Mesh Conductor for Touch Screen Displays", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 194..196, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 NanoWeb commercial mesh, 1 micron filaments of silver / aluminum, alternative to ITO for touchscreens. Low electrical resistance. [Atmel15a] Atmel Corporation "Atmel maXTouch U Series capacitive touchscreen controllers", Atmel Corporation, 2015 capactivie touchscreen, both self-capacitance and mutual capactitance modes, adapt for water droplets on surface, gloves, proximity sensing. Compare with Apple/Fingertouch? [BerglindR14a] Berglind, Richard and Rosengren, Erik "Optical Multi-Touch on a Circular Devices", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 619..622, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Circular optical multi-touch touchscreen: optical sensors around entire circumference, smaller number of (infra-red) light sources around, checks for occlusion of light to sensors. No reference to shadowing problem. [BilbreyBC14a] Bilbrey, Brett C.; King, Nocholas V.; and Aleksandar, Pance "Synchronized interactive augmented reality displays for multifunction devices", United States Patent 8,625,018, January 7, 2014 Live video (from on-board camera of actual environment) on a tablet computer with touchscreen input: annotations / electronic ink information layer overlaid on live video as augmented reality. [BlindmannG15a] Blidmann, Gennadi "Knowledge base / Multitouch technologies", www.multitouchsolution.com/en/knowledgebaseen, fetched 2015 Overview of touchscreen/tablet technologies, comparison charts. Includes in-cell resistive multitouch tablets, reverse-diffused optical, FTIR, etc. No mention of electromagnetic, acoustic/sonic, only tablet technologies author said would work with an (ungloved) finger touch. Compare with Geoff Walker? [BornscheinJ14a] Bornschein, Jens "BrailleIO - a Tactile Display Abstraction Framework", Proc TacTT '14, November 16, 2014, Dresden Germany Software framework package for tactile output desplays (for the blind), so that custom software does not have to be written and developed for every devices. Internally implements multiple screens (windows?), user can switch applications quickly. Keyboard and arrow keys, or touch tablet surface (cites to others). Basic gesture recognizer for pointing, swipes, pinch, circles, multi-finger/multi-touch drag. Display is a viewport onto a portion of the nominal visual display. [BornscheinJ14b] Bornschein, Jens and Prescher, Denise "Collaborative Tactile Graphic Workstation for TouchSensitive Pin-Matrix Devices", Proc TacTT '14, Workshop on Tactile/Haptic User Interfaces for Tabletops and Tablets, ACM ITS 2014, November 16, 2014, Dresden Germany Tactile output desplays (for the blind) and for the sighted for collaborative work: extension of tabletop/tablet to visually impaired. Pin-matrix display (BrailleDis7200), touch input. Button and arrow-key rocker user interface for blind (so that touching the display to see the display does not cause user input?). [BricklinD14a] Bricklin, Dan "AlphaRef Reader: Tablet-first design of an app for reading reference material", www.bricklin.com, fetched 2015 Essay on design process for a tablet-based reader for reference material: not page-oriented (as PDF), remembers recently accessed information. Two-dimensional scroll: vertical for coarse/regular positioning, then horizontal motion for fine possitioning for long texts. Suggest design layout and features first on (larger) tablet, later user interface adapt for smaller smartphone/screens: once you know what it important. [BrownE14a] Brown, Ed "Three Trends Shaping the Human-Machine Interface", Automation and Control, Oct. 7 2014 Essay piece: Three trends in machine control interfaces: multi-touch touchscreen sensors, communication with mobile devices (or more correctly, from mobile devices e.g. smartphones and tablets to industrial equipment in a factory), and better human factors in the software. [Cambrios14a] Cambrios Technologies Corporation "ClearOhm High Performance Transparent Conductive Material", www.cambrios.com, 2014 Silver nanowire coating material, alternative to ITO indium tin oxide. File contains additional materials: Technology Review articles, etc.



[ChenXA14a] Chen, Xiang 'Anthony'; Schwarz, Julia; Harrison, Chris; Mankoff, Jennifer; and Hudson, Scott E. "Air+Touch: Interweaving Touch and In-Air Gestures", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 519..525 Experimental system for combining touch/multi-touch gestures with in-air proximity gestures, e.g. highjump gesture of two taps with finger going high between to select; in-air pigtail hand/finger gesture to copy to clipboard. Prototype uses depth camera over smartphone touchscreen: capacitive/electrostatic touchscreen sensing to proximity from 1.5 to 4.0 cm. [ChenZ14a] Chen, Zheng; Sun, Jian-Tao; and Huang, Xuedong "Web Information at Your Fingertips: Paper as an Interaction Metaphor", IEEE Computer, March 2014, pp. 62-66 Touch-to-search gesture: bi-manual / two-hand gestures used for search user interface: user holds touchscreen with touch on one side, other finger can highlight/select text for search on web. Example: eaiser to find other web pages about a product listed on Amazon.com. (Microsoft Surface) [ChoiYM14a] Choi, Young-Man; Kim, Kwang-Young; Lee, Eonseok; and Lee, Taik-Min "Reverse-Offset Printed Single-layer Metal-Mesh Touch Screen Panel", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 197..199, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Reverse-offset printed mesh/grid of silver/Ag nanoparticle conductive ink trace wires for touchscreen touch panel: single layer printing on flexible plastic substrate. At 10-micrometer width, invisible (transparent) to human eye. Capacitive/electrostatic sensing, does not give details of single-layer sensing -- cites to Ho 2009? Illustrations show grid with 10-micrometer breaks in traces, no sensing pad matrix array? [ChristianssonT14a] Christiansson, Tomas and Wassvik, Ola "Touch-sensitive system and method for controlling the operation thereof", United States Patent 8,686,974, April 1, 2014 Optical multitouch touch tablet, thin profile. Light emiters inject light from points around the sides, detected by sensor on other sides. Touch causes weaking of light along path due to FTIR total internal reflections. Not clear how it deals with shadowing problem for multiple touches: multiple sensing paths in different directions. Compare with Rapt Touch? [ConiP14a] Coni, Philippe; Berthon, Julie; Perbet, Jean-Noel; Sontag, Yves; Abadie, Jean Christophe; Dubau, Christophe; and Balihaut, Cedric "Touchscreen System Architecture for Safety Critical Applications", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 1600..1603, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 High-reliability measures for capactitive grid (multi-touch) touchscreens for avions, other safety-critical application. Basically detect a broken trace or line in the grid, and use the average/combined sensing of the two ajacent conductors. [CuiJ14a] Cui, Jie; Ye, Zhi-Cheng; Sun, Nan-Ling; Zhend, Jun; and Shieh, Han-Ping D. "Optical Touch Screen based on Grating and Waveguide", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 200-203, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Transparent touchscreen using grid of polymer waveguide/lightguides figers with monochromatic light: finger touch affects FTIR total internal reflection, light along that fiber is "shadowed". [DeberJ15a] Jota, Ricardo; Ng, Albert; Dietz, Paul, and Wigdor, Daniel "How Much Faster is Fast Enough? User Perception of Latency and Latency Improvements in Direct and Indirect Touch", Proc. CHI 2015, Crossings, Seoul, Korea, April 18-23, 2015, pp. 1827..1836 Continuation of Jota 2013 paper. Effects of latency on users of multi-touch/single-touch touchscreens. Cites to lag/latency in VR virtual reality displays. Tapping latency less noticable than dragging/directmanipulation on touchscreens. [FlatFrog14a] FlatFrog Laboratories AB "FlatFrog Touch Technology - optical in-glass", www.flatfrog.com, June 18, 2014 multi-touch optical touchscreen, infrared light beams to multiple receivers at edges of glass, touch affects internal reflection within glass. Mentions palm rejection, no details. See Christiansson patent. [GeislerA14a] Geisler, Alexander; Valkov, Dimitar; and Hinrichs, Klaus "Void Shadows: Multi-Touch Interaction with Stereoscopic Objects on the Tabletop", Proc. SUI'14, October 4..5, 2014, Honolulu, pp 104..112 3D three dimensional manipulation using a flat 2D multi-touch surface: 3D images are shown below surface, with a simulated vertical "shadow" (cone shaped projection, narrowing as comes up) from virtual object up to touch surface. User can manipulate object by multi-touch on shadow at tabletop. Eliminates visual convergence problems, since fingers and shadow "surface" are on same plane. [GelmanA14a] Gelman, Andrew and Loken, Eric "The Statistical Crisis in Science", American Scientist, AAAS, Vol 102, Nov-Dec 2014, pp. 460-465 Essay on "cooking" of experimental statistics (examples mostly in the social sciences) by phrasing questions after the collection of data: questions can be chosen/re-phrased for any particular data collection in such a way to appear to support almost any hypothesis. See also Ioannidis 2005. [GoertzM14a] Goertz, Magnus "User Interface", United States Patent 8,650,510, February 11, 2014 Touchscreen user interface: touch on icon, slide off, perform function based on slide. Compare with Buxton keyboard?



[GogueyA14a] Goguey, Alix; Casiez, Gery; Vogel, Daniel; Chevalier, Fanny; Pietrzak, Thomas; and Roussel, Nicolas "A three-step interaction pattern for improving discoverability in finger identification techniques", Proc. UIST'14, October 5-8, 2014, Honolulu, Hawaii, pp. 33-34 (demonstraction) Prototype two-hand/multi-touch user interface: static fingers of non-dominant hand on touchscreen change "mode" of gestures in dominant hand, similar to shift/function keys with mouse. Prototype uses colored band (tagged ring) on fingernails and overhead optical camera to identify fingers. Compare with Buxton on two-handed user interaction? [HagaH14a] Haga, Hiroshi; Yoshinaga, Kazuhide; Yanase, Jiro; Sugimoto, Daisuke; Takatori, Kenichi; and Asada, Hideki "Electrostatic Tactile Display Using Beat Phenomenon of Voltage Waveforms", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 623-626, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Tactile/haptic feedback on electrostatic multi-touch screen by driving X and Y lines for a particular point for feedback at frequencies above human detection (1000 Hz), beat frequency is perceptible (240Hz). Not highly localized, perhaps usable for keyboard images. Appears to be able to do only one location -- shadow effect? [HawkinsRW14a] Hawkins, Robert W. "Input cueing emersion system and method", United States Patent 8,760,391, June 24, 2014 Video image of hands combined semi-transparentlly with computer image, so that user perceives hands as touching directly on image, without hands obscuring image. Any touchpad/tablet technology: also proximity sensing? Compare to Wellner, systems with rear-surface touching, 3D manipulation / virtual reality systems. [HinckleyK14a] Hinckley, Ken; Pahud, Michel; Benko, Hrvoje; Irani, Pourang; Guimbretiere, Francois; Gavriliu, Marcel; Chen, Xiang Anthony; Matulic, Fabrice; Buxton, Bill; and Wilson, Andy "Sensing techniques for tablet+stylus interaction", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 605..614 Combine multi-touch hand and stylus (e.g. able to sense orientation, type of grip) for gestural context based on behavioral context. Ignore unintentional touches (palm rejection) by context. Two-handed multimodel gestures: e.g. pinch gesture with while tucking pen in hand gives magnifying glass for drawing; touch/point with left hand while angling stylus above surface gives airbrush tool. Stylus grips: writing, tuck (stylus held like chopstick while writing hand is gesturing), palm (stylus held in fist), extension (point with a finger while tick or palm grip). Review of user-interface work on grip sensing, proximity, 3D position or orientation of stylus, hand gestures. Handing tablet to other user brings up per-user acetate drawing layer. [HughesJF14a] Hughes, John F.; van Dam, Andries; McGuire, Morgan; Sklar, David F.; Foley, James D.; Feiner, Steven K. and Akeley, Kurt "Computer Graphics: Principles and Practice, 3rd edition", Addison-Wesley, 2014 Reference text on computer graphics: similar to Foley (also contributor). Described alpha blending as "coverage" (opacity/translucency/transparency). No index listing for BitBlt(!). [JapanDisplay14a] Japan Display Inc. "JDI Reveals Embedded Touch Technology", Japan Display Inc., 2014 Marketing and product information on touchscreen display with capacitive multi-touch integrated: appears to be sheet of ITO within glass on front of display elements/LCD, matrix of scan electrodes within glass on rear. [KangasPD14a] Kanga, P. Daniel and Ranck, Daniel M. "Touchscreen gestures for virtual bookmarking of pages", United States Patent 8,650,509, February 11, 2014 Bookmark pages temporarily in electronic book by touching them with multiple fingers, like gesture of holding finger between pages. Multi-touch touchscreen. [KentJ14a] Kent, Joel "Touch-Technology Diversity in Commercial Applications", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 615..618, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Commercial (Elo Touch Solutions) appeal that capacitive (projected) touchscreen technology will dominate, but not appropriate for all solutions. P-cap difficulty with any stylus (vs. resistive, optical), contaminant (e.g. water) rejection, high transparency (vs. SAW and optical), extremely large surfaces (resistive, SAW). [KhosravyM14a] Khosravy, Moe; Novik, Lev; Woodcock, Katrika Marie "System and Method for Converting Gestures into Digital Graffiti", United States Patent 8,872,767, October 28, 2014 In-air gestures using both position and orientation of a hand-held device (WII?), for drawings/markings (graffiti) on image. [KimHJ14a] Kim, Hyeon-June; Jo, Dong-Shin; Yang, Jun-Hyeok; and Ryu, Seung-Tak "A Low-Power Fast Readout Circuit using a Dual-Mode Sensing Algorithm for Medium-Size Capacitive Touch Screen Panels", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 540..543, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Electrostatic grid touchscreen, reduce power consumption by scanning fast/coarse until a touch is detected, then scan at full resolution. Compare with Pencept/Numonics course/fine scanning on electromagnetic tablets. [KimS14a] Kim, Steve; Cha, Sung-Hyuk; Monaco, John V.; and Tappert, Charles C. "A Correlation Method for Handling Infrequent Data in Keystroke Biometric Systems", 2014 Int'l Workshop on Biometrics and Forensics (IWBF), 27..28 March 2014, Valletta, Malta, pp. 1..6



Biometric authentication of user by keyboard/keystroke dynamics. Notes that infrequent typing characters are hard to get statistics on (need large samples), but also have lower correlation on inter-keystroke timing. File includes presentation slides. See also Saltzer 1975 (not cited). [KoundinyaP14a] Koundinya, Pranav; Zhao, Xi; Feng, Tao; Theril, Sandhya; and Shi, Weidong "Touch-Fingerprint Display for Supporting Identity Sensing", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 1610..1613, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Proposal for combined fingerprint/multi-touch touchpad device: high resolution matrix, scanned at low resolution (every "N" trace lines) for touch input: for fingerprint input, when finger is found, re-scan just that area at high resolution (don't skip any traces). Can be implemented in-cell (with display) by sizing electrodes/sensor points as sub-pixel spacing. [KyoungK14a] Kyoung, Kyuwon and Hattori, Reiji "A Low-Side Self-Capacitive Sensing Method for Large Area Touch Screen", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 1604..1607, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Electrostatic sheet touchscreen, single-touch: User finger is active probe, signal input by wrist-strap: ratiometric measurement of signal picked up by resistive sheet. Appears very similar to Scriptel. [LandryG14a] Landry, Greg; Kolokowsky, Steve; and Wright, David G. "Single Layer Multi-touch Capacitive Sensor", United States Patent 8,692,799, Aril 8, 2014 Interdigitated single-layer capacitive tablet: separate triangular traces. Position determined by which pair of traces (from opposite sides), and charge ratio (see). Variations include small teeth and indentations on sides of long triangles. Corrections for edge effects on bottom and top rows. [LeeSG15a] Lee, Sang-Gyu; Lee, Suk-Joo; Park, Bong-Ki; and Moon, A-Ram "Film with Codes, Reader Used for Reading the Film and Electronic Device with a Display Device for the Film", United States Patent Application 2015/0014553 A1, January 15, 2015 Stipled pattern on transparent film, optical video/camera in stylus to determine position based on pattern. Flourescent dots for pattern, exemplary pattern is dots offset from grid. Assigned to NeoLab (see file). Compare with IBM display, Anoto? [LeighD15a] Leigh, Darren "Fast Multi-Touch Post Processing", United States Patent Application 2015/0015497 A1, January 15, 2015 Low-latency method for interpolating local maximum of touch points on multi-touch touchscreen ("heat map"), tracking touch points between frames. Shows diamond pattern of capactive grid. Compare with Westerman, Rekimoto, Rympalski 1987? [LeighD15b] Leigh, Darren; Forlines, Clifton; Wigdor, Daniel; and Sanders, Steven Leonard "Latency Measuring and Testing Method", United States Patent Application 2015/0097803 A1, April 9, 2015 Mechanical system for determining latency in user interface (touchscreen) by measuring time between input event and system response. Includes dragging latency. [LinPY14a] Lin, Po-Yang; Huang, Chen-Shuo; Chen, Ji-Feng; Tu, Chia-Hsun; Hwu, Keh-Long; Cahuang, ChingSang; and Lin, Yu-Hisn "A Curvature Sensing Circuit for Flexible Display", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 126-129, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Flexible touchscreen display sensor, matrix of small printed strain gauges. Compare with flexible/deformable input devices. [LindlbauerD14a] Lindlbauer, David; Aoki, Toru; Walter, Robert; UEMA, Yuji; Höchtl, Anita; Haller, Michael; Inami, Masahiko; and Müller, Jörg "Tracs: Transparency Control for See-through Displays", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 657..661 Transparent touchscreen with display, two users collaborating on opposite side, can switch touchscreen among transparent (direct communication), translucent/semi-transparent (collaboration and video), and opaque (displays for both users only). Not sure it comments on mirror-image/reversal between users. Compare with Whiteboard/collaboration systems. [LionPrecision14a] Lion Precision "Noncontact Capacitive and Eddy Current Displacement and Position Sensors", lionprecision.com, fetched 2014 Capactivie linear displacement and capacitive position sensors, primarily for industrial use: mostly linear sensors, not X/Y. Also eddy-current/inductive linear sensors. Includes TechNote L03-0020, Capacitive Sensor Operation and Optimization tutorial. [LiuSY14a] Liu, Shi-Yu; Wang, Yi-Jun; Lu, Jian-Gang; and Shieh, Han-Ping D. "One Glass Solution with a Single Layer of Sensors for Projected-Capacitive Touch Panels", SID Symposium Digest of Technical Papers, Vol 45 No 1, pp. 548-550, Proc. of SID Symposium, San Diego, CA, June 1-6, 2014 Single-layer capacitive touchscreen (not multi-touch) with one layer of electrode strips: X coordinate by which trace finger is near, Y coordinate by modeling resistance along trace as two resistors divided by a capacitor to ground where finger is along the trace. [LouchJO14a] Louch, John O. "Variable Device Graphical User Interface", United States Patent 8,631,358, January 14, 2014



Shake or tilt a smartphone (shake/tilt gesture), as an indication to detect user mistakes on user interface, and adjust the user interface (move/shift/re-size icons). [LyonsM14a] Lyons, Micahel and Fels, Sidney "How to design and build musical interfaces", Workshop presentation, ACM SIGGRAPH Asia 2014, Shenzhen, China, Dec. 3, 2014 Slides of continuing workshop series NIME ("New Interfaces for Musical Expression", since 2001 CHI) on musical instrument interfaces. Historial discussion of theremin with capacitive proximity sensing. Haptic, touchscreen, force-feedback, force (pressure) sensing, torque/rotation/translation forces (side-ways force), proximity, Polhemus and Ascension flock-of-birds 3D sensors. TouchGlove and TouchStrip, TapTile FSR (force-sensitive resistor). Visual-input piano (optical/video sensor), ReacTable movable puck/object sensors. Kinect/ Hugh Le Caine keyboard touch sensor 1940s. [McCannyB15a] McCanny, Benjamin; Forlines, Clifton; and Wigdor, Daniel "Reducing Control Response Latency with Defined Cross-Control Behavior", United States Patent Application 2015/0077355 A1, March 19, 2015 Reduce user interface latency on touchscreen input by having second (faster/low-latency) processing of user input to give immediate feedback based on static definitions of UI elements. Mentions low-latency system reducing events sent to high-latency application. Compare with PenWindows/PenPoint/Freestyle? [Metaio14a] Kurz, Daniel "Thermal Touch: A New Augmented Reality Interface for Wearables", metaio.com, 2014 Two Video presentations: one for conference, other as Webinar. "Turn any surface into touchpad": using Google glass or other optical head-mounted sensor, detect residual heat thermal signature on surface briefly after touched by human finger. Requires that optical camera respond to / distinguish infrared? Prototype / proof of concept only, company (German) does Google glass / virtual reality applications. File contains additional materials. [Metec14a] Metic Ingenieur AG / HyperBraille "HyperBraille: Das grafikfaehige Display fuer Blinde", www.hyperbraille.de, metec-ag.de, 2014 (in German) HyperBraille/BrailleDis 7200/9000 product information and technical sheet. 60x120 pin-matrix graphical display for the blind, includes touchscreen input with 1440 touch sensors. Also Modul D2 data sheet: 2x10 single-character Braille output, building block for BrailleDis with 2 touch sensors each (capacitive?). [Microsoft14a] Microsoft "Surface Pro 3 User Guide With Windows 8.1 Pro", Microsoft Corp., 2014 Windows 8.1 Surface touchscreen: OneNote electronic ink note-taking. Gestures: Tap, Tap-and-hold (press-and-hold), tap-slide (tap-tap slide / tap and a half), pinch/stretch, rotate, slide/drag, swipe to select (quick small flick), swipe from edge, two-finger-tap (like 2nd mouse button), two-finger slide to scroll. Surface Pen (stylus) has rear button (compare Freestyle), right-click button (compare Freestyle): stylus proximity turns off touch (finger). [Microsoft15a] Microsoft "A history of Windows", windows.microsoft,com, fetched 2015 Corporate history page on Microsoft Windows. Interface Manager code name, announced as operating system 1983, released as Windows 1.0 1985. Windows 1.0 in 256Kb Ram. Windows 2.0/2.11 1987 Control Panel. Windows 3.0 operating system. Windows NT new operating system 1993. 32-bit. Mentions nightly build/stress tests in development. Very brief mention of Tablet PC, none of Pen Windows: Windows Touch on Windows 7. [MilekicS15a] Milekic, Slavko "Slavko Milekic, Profile", University of the Arts, Pennsylvania: fetched from www.uarts.edu Fine Arts faculty profile for Slavko Milekic. Artistic presentations use gaze-tracking, touchscreen input. Mentions KiddyFace for art exploration with children. [MobilePoint15a] MobilePoint Corporation. "MobilePoint: Simplicity of Pen and Paper / Power of a Computer", MobilePoint.com, fetched 2015 MobilePoint. forms-based pen computing applications for field sales. SalesTrak Mobile for iPad. Information stored locally (does not require live internet connection). Medical sales features. [MonnaiY14a] Monnai, Yasuaki; Hasegawa, Keisuke; Fujiwara, masahiro; Yoshino, Kazuma; Inous, Seki; Shinoda, Hiroyuki "HaptoMime: Mid-Air Haptic Interaction with a Floating Virtual Screen", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 663..667 Single point (one fingertip) haptic/tactile in-air feedback using focused ultrasound, combined with halfmirror 3D three dimensional display (Aerial Imaging Plate API ASUKANET commercial unit). 40 kHz signal has focal spot about size of a fingertip (one wavelength), modulated at 100 Hz to produce sensation. Compare with UltraHapics project at U. Bristol. [MorishitaY14a] Morishita, Yusuke "Portable Terminal Having Double-Sided Touch Screens, and Control Method and Storage Medium Therefor", United States Patent Application 2014/0292697, October 2, 2014 Tablet/hand-held computer (smartphone) with second touch screen on the back/rear side, with rotation/orientation sensor. If held vertically, only bottom half of rear touchscreen is active. [MyScript15a] MyScript (formerly Vision Object "MyScript product information: e-Pen Reinvented: Smart, Sleek, and Enterprise Ready (MyScript)", myscript.com, fetched 2015



Handwriting and shape recogniton with applications components for SmartPHones: MyScript formerly VisionObjects (see file). Developer documentation: single-character, line, multi-line input widgets. Includes announcement of integration with NeoLAB smart pen, similar to Anoto? [NakaiY14a] Nakai, Yuriko; Kudo, Shinya; Okazaki, Ryuta; and Kajimoto, Hiroyuki "Tangential Force Input for Touch Panels Using Bezel-Aligned Elastic Pillars and a Transparent Sheet", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 111..112 Sides of touchscreen mounted on small pillars, can detect horizontal (and vertical) shear forces. User interface to deform virtual objects. Compare to Margaret Minsky? [NationalInstruments14a] National Instruments "Introduction to LabView Graphical Programming", www.ni.com, fetched 2014 Tuturial/Overview on LabView system and graphical programming language, 2014. Focuses on programing of virtual lab instruments in LabView VirtualBench. Describes VirtualBench APIs: the graphical programming interfaces between LabView virtual instrument (VI) objects. [NeoLab15a] NeoLAB Convergence "NeoLAB: product information -- Neo Smartpen N2 nNote for N2 Your personal notebook", neosmartpen.com, fetched 2015 Product information on Neo SmartPen: patterned paper in special notebooks, optical camera in pen to recognize position (and which paper form) from "Ncode" pattern. Include NeoNote app software for IOS and Android smartphone to enter handwriting notes (to EverNote?). press-and-hold popup graphical menus to select ink color, width. Integration with voice notes (compare with Wang Freestyle?). CogniVue optical camera hardware. See allo MyScript (formerly VisionObjects) handwriting and shape recognition. Compare with Anoto, IBM pattern? [Ocular14a] Ocular LDC "Ocular Projected Capacitive Touch Panels - EDGE", ocularldc.com, fetched 2014 Ocular: Projected capacitive / electrostatic touch panel tablets: Crystal Touch / Edge. Thin glass. Opticallybonded glass cover optional intead of air gap between layers. Shows flush and bezel mounting. Includes multi-touch demonstration software with finger tracking. Recommends touch areas (accuracy?) of 10 mm plus spacing. Mentions two-finger pinch/zoom without multitouch. [OksE14a] Oks, Eduard and Rosen, AlexLDC "Gesture Control System", United States Patent 8,890,803, November 18, 2014 Optical/camera detection of hand gestures in a smartphone, for controlling a projector built into the phone (or other device). Waving hand left/right to scroll. Basically only detecting gross motion of any kind, not discrete gestures. [PattersonI14a] Patterson, Isaac "Trustworthy system development through high-level synthesis", Thesis project, Naval Postgraduate School Monterey, CA, September 2014 Design project for secure CPU processor: programs in specific "trusted" area of memory protected from modification by software in other areas. Compare with DEP data execution protection, memory mapping, Yee protected processor? [Penclic14a] Frank Füller "Bluetooth-Maus in Stiftform soll Mausarm verhindern", Neuerdings.com, September 1, 2014 Penclic Mouse B3: Mechanical roller mouse in form of pen with roller in tip, compare with PenMouse from 1980's (in collection). Wireless bluetooth interface. www.penclic.se product literature in file includes cable versions. Multiple barrel buttons for left/right mouse button, center roller, page up/down. Illustrations seem to imply detection of rotation of stylus/pen. [Penclic14b] Penclic.se "Penclic NiceTouch product information", Penclic.se, September 2014 Penclic external touchpad: with four buttons for left/right click, forward/back keys (for browser), additional physical spin wheel /scroll wheel for scrolling. [PfeufferK14a] Pfeuffer, Ken; Alexander, Jason; Chong, Ming Ki; and Gellersen, Hans "Gaze-touch: Combining Gaze with Multi-touch for Interaction on the Same Surface", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 509..515 User-interface combining eye-tracking (gaze) with multi-touch gestures on touchscreen: Gaze/eye-tracking selects, touch manipulates. Gestures can be made at any location on objects selected by gaze "dwell". compare with Wellner etc. large wall displays. Problems noted include multiple-select (essentially lose multi-touch for select), "muscle memory" gestures (activating PIN keypad) etc. where user is not gazing at the object to be manipulated. [PiotJ14a] Piot, Julien; Kolundzija, Mihailo; Korchagin, Danil; Dokmanic, Ivan; Vetterli, Marting; and Drumm, Owen "Optical Touch Tomography", United States Patent Application 2014/0152624 A1, June 5, 2014 2014 Optical sensing touchscreen: optical emitters at sides direct beams withing glass to multiple optical sensors spaced on opposite side, touch on surface of glass disturbs (reduces) frustrates total internal reflection (FTID). Comparision of disruption of multiple beams with pre-analyized disruptions by known reference touchs, determine multi-touch inputs. See Rapt Touch file (assignee). Compare with FlatFrog? [PolyIC14a] Poly IC "PolyTC resistive conductive transparent films", polyic.com, 2014



Resistive/consductive films, 200-micrometer-sized metal mesh in polyester/PET film substrate, alternative to ITO indium tin oxide, moldable to curved surfaces. "Touchscreens mit transparenten, flexiblen und leitungsfäigen Folien" [PrescherD14a] Prescher, Denise "Redesigning Input Controls of a Touch-Sensitive Pin-Matrix Device", Proc. TacTT, November 16, 2014, Dresden User study of BrailleDis 7200 tactile pin-matrix display for the blind. User interface has multiple controls: on-screen sliders and buttons, function keys, cursor keys, separate touchpad. Function button to switch to gesture input (vs.s. "view" mode where user scanns display). Separate touchpad not as good as touchscreen input. [Pyreos14a] Pyreos "Pyreos Gesture Sensor FAQs: Pioneering Low Power Gesture Sensor Enabling Touchless Control for Mobile Devices", pyreos.com, 2014 "Touchless Gesture control": Passive infra-red optical video sensor for detecting hand and body gestures: comntrasted with touchscreen and with Microsoft WII. [RaptTouch14a] Rapt Touch "Multi-touch Technology by RAPT", www.rapttouch.com, 2014 Optical touchscreen using radial pattern of internally-reflected light beams through glass layer. See Piot reference for details. Also see video file, same web site. [RheeTH14a] Rhee, Taik Heon; Byeon, Kwangmin; and Shin, Hochul "Contelli: A user-controllable intelligent keyboard for watch-sized small touchscreens", Proc. UIST '14, October 5..8, 2014, Honolulu, pp. 85..86 Very small touchpad keyboard for watch-sized devices: long-tapping vs. short-tapping: holding finger on keyboard for longer time with zooming UI display allows for more precise input of characters, system then knows not to do word-substituion/spelling correction. [Sabic14a] Sabic Innovative Plastics / Cema Nanotech "Transparent Conductive Polycarbonate Film", sabic-ip.com, sabic.eu, fetched 2015 ITO-replacement, streatched carbon monofilament film on polycarbonate substrate. Largr-format tabletop/touchscreen, multi-touch (up to 10 touches) projected capacitance. [SeifertJ14a] Seifert, Juilan; Boring, Sebastian; Winkler, Christian; Schaub, Florian; Schwab, Fabian; Herrdum, Steffen; Maier, Fabian; Mayer, Daniel; and Rukzio, Enrico "Hover Pad: Interacting with Autonomous and SelfActuated Displays in Space", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 139..147 3-D display (of sorts): flat display suspended from movable support arm on ceiling, user can move display in 5 DOF degrees of freedom (X,Y,Z,pitch,yaw -- no roll) to see views of volumetric images. More importantly, display can move autonomously without user's hands: e.g. user requests view of something (e.g. a particular portion by name of a 3D image of a brain), display moves autonomously to position and orientation to display view. Uses in-air gestures (e.g. in-air pinch gesture to "pick up" display so that it is moved in direction of hand motion. [SekitaniT14a] Sekitani, Tsuyoshi; Kaltenbrunner, Martin; Yokota, Tomoyuki; and Someya, Takao "Imperceptible Electronic Skin", SID Society for Information Display, January/February 2014, available at informationdisplay.org Long-term project review (back to 2001) for thin flexible electronic skin E-skin touch sensor, proposed uses are robotic sensing, medical instrumentation. Sensor skin applied to throat to recognize speech by muscular activity. Plastic substrate, TFT touch sensors. Shows pressure profile from array of sensors. [SenSeg15a] SenSeg.com "SenSeg Tixel Haptic Technology", SenSeg.com, fetched 2014 Tactile/haptic electrosensory display. AC-modulated signal coupled capacitively to fingertips at discreet pads, sensation is direct Coulomb force capacitive to Pacinian mechanoreceptor nerve cells in fingertip, not physical vibration or electrostatic friction. [SensorProducts14a] Sensor Products Inc., Madison NJ "Product information, Tactilus Contact Surface Pressure Mapping Sensors", sensorprod.com, fetched 2014 Matrix pressure-sensing tablet, piezoresistive grid, minimum sensing grid resolutation approx 1.6 mm. Matrix can be flexible, and also stretchable (elastomeric). Targeted as tactile sensor for mechanical testing, measurement of fit (of chairs, mattresses, etc.). [ShahparniaS14a] Sharparnia, Sharooz; Mullens, christopher Tenzin; and Grunthaner, Martin Paul "Display Touch and Stylus Synchronization", United States Patent Application 2014/0354555 A1, December 4, 2014 Parallel/simultaneous touch (multi-touch) and stylus sensors on a touchscreen display, syncronized to refresh signal for display. [SongJ14a] Song, Jie; Sörös, Gabor; Pece, Fabrizio; Fanello, Sean Ryan; Isadi, Shahram; Keskin, Cem; and Hilliges, Otmar "In-air Gestures Around Unmodified Mobile Devices", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 319..329 in-air/proximity gestures on smartphone using regular smartphone camera on front. States advantages for mobile use, hands-free, display screen not obscured. Continuous recognition of pinch/zoom gesture motions of hand (move in and out, etc.). Compare hand/gesture detection with Wellner? Mentions touch+gestures (see files). [TactualLabs15a] tactuallabs.com "Fast Multi-Touch product information", TactualLabs.com, fetched 2015



"Zero-latency" capacitive multi-touch: reduce display latency responding to touch input by up to 33 msec (two video frams). Inking, UI response done at interrupt level: states does not use predictive input. Founding staff from U. Toronto. [Tactus14a] Tactus.com "New Product Adds Physical Buttons to Touchscreens", IEEE Computer, March 2014, pp 20-21 micro-fluidics channels in thin layer, can be inflated to create physical buttons on transparent touchscreen as haptic display. [ThreeDconnexion14a] 3Dconnexion "SpaceMouse Pro: product information - three-dimensional mouse, 6DoF sensor", 3dconnexion.com, fetched 2014 3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick? ) for pan, zoom rotate. [ThreeDconnexion14b] 3Dconnexion "SpacePilot Pro: product information - three-dimensional mouse, 6DoF sensor", 3dconnexion.com, 2014 3D three-dimensional "mouse" input device: responds to push, pull, twist or tilt input (more like a joystick? ) for pan, zoom rotate. Five near-mouse function keys, use short press (tap) and long press (press-andhold) for 10 functions. Function keys automatically redefine function according to appliction (default function of OS?). Keyboard modifier keys (shift,alt,ctl) that are often used with mouse input. as qualifiers/modifiers. [ThreeDconnexion14c] 3Dconnexion "3Dconnexion - Navigate your 3D world - The CAD Professional's Guide to Lightning-Fast R&D", 3dconnexion.com, 2014 3D three-dimensional "mouse" input device Sales brochure (?) on CAD services and software: Third-party Product Data Management PDM tools for collaborative 3D CAD design work: GrabCAD Workbench, Tinkercad, Sketchfab, WikiSpaces, HackPad, etc. [Toshiba14a] Phys.org "Touch- and gesture-based input to support field work", Phys.org, Feburary 18, 2014 Data Glove for industrial applications with gyroscopic sensor, accelerometer, contact detection sensor on index finger: compare with Micorosoft WII controller? Motion sensors used for gestures, contact sensor and tag reader (RFID?) used to recognize labels on equipement. Intended as wearable data entry device. [TungYC14a] Tung, Ying-Chao; Cheng, Ta-Yang; Yu, Neng-Hao; Chen, Mike Y. "FlickBoard: enabling trackpad interaction with automatic mode switching on a capacitive-sensing keyboard", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 107..108 transparent capacitive-grid (multi-touch) touchpad film, overlaid on keyboard. "Machine learning" to distinguish moving touchpad input and keyboard typing input. Call capacitive sensing grid a capacitanceto-digital converter. Cites to Rekimoto ThumbSense, but not to multitouch tabletop. [VertheinW14a] Verthein, William; Watson, Josh; Lee, Stephen "Infrastructural haptics on wall scale interactive displays", United States Patent 8,830,188, September 9, 2014 Wall-sized display with tactile feedback (vibration) [VuT14a] Vu, Tam; Baid, Akash; Gao, Simon; Gruteser, Marco; Howard, Richard; Lindqvist, Janne; Spasojevic, Predrag; and Walling, Jeffrey "Capacitive Touch Communication A Technique to Input Data through Devices Touch Screen", IEEE Trans. on Mobile Computing, Vol 13 No 1, January 2014, pp. 4-19 Transmit additional information through existing capacitive/electrostatic touchscreen: circuit in signet ring on user's hand generates signal that can be picked up by existing touchscreen electronics at low data rate (5..10 bps). Can also be used to identify multiple styli/styluses, multiple users, etc. [WalkerG14a] Walker, Geoff "Display Week 2014 Review: Touch Technology", Information Display, Nov/Dec 2014, Vol 30 No 6 pp. 40-45 Review of produced/exhibitors at Display Week exposition: embedded-touch and in-cell touchscreens and sensors, integrated with display. Reverse-side electrostatic/P-CAP; single-layer mutual-capacitance using "caterpillar pattern"two-element grids with crossovers/bridges; "Pixel Eyes" (JDI vendor); optical fingerprint sensor, very high resolution, LED pixels used for touch sensing. Water resistance: does not affect selfcapacitance sensors, only mutual-capacitance. Electromagnetic resonance touch battery-less unpowered stylus from Hanvon: compare with Walcom? ITO replacements carbon nanotubes, silver nanowires, silicondoped zinc, PET PoliyIC film. photolithography printing of mesh. Resistive-film multi-touch using 12 mm sensor pads. [WalkerG14b] Walker, Geoff "SID Display Week 2014 Tutorial: Touch Technology", SID Display Week, June 1, 2014 Comprehensive tutorial of touchscreen technologies as of 2014. Emphasis on projected capacitance (electrostatic). Electrode patterns for grid, including single-layer grid geometries. Notes that pressure/force sensing is actually contact area. Optical coatings, ITO-Replacement materials, embedded sensors, Stylus. Cites to Buxton. Illustrations from Apple patent, no mention of FingerWorks or Rekimoto. Resistive multitouch by dividing into small finger-sized squares. Cites to JazzMutant as first multi-touch product. [Widevine14a] Widevine Technologies "www.widevine.com, fetched 2014", Widevine DRM, Common Encryption



(CENC), Dynamic Adaptive Streaming (DASH) and Encrypted Media Extensions (EME) Encryption/DRM application platform (with API) for protecting multimedia (especially Video) documents and content. Uses secure hardware for keyboxes to store private PKI keys. [WigdorD14a] Wigdor, Daniel; Sanders, Steven Leonard; Jota Costa, Ricardo Jorge; and Forlines, Clifton "Hybrid Systems and Methods for Low-Latency User Input Processing and Feedback", United States Patent Application 2014/0139456 A1, May 22, 2014 Reduce user interface latency on touchscreen input by having second (faster/low-latency) processing of user input to give immediate feedback based on static definitions of UI elements. Mentions low-latency system reducing events sent to high-latency application. Compare with PenWindows/PenPoint/Freestyle? [Wikipedia14a] Wikipedia.org "OpenGL Open Graphics Library", Wikipedia.org, fetched 2014 OpenGL: 2D and 3D vector graphics library, includes alpha blending. API specification for access to hardware. [Wikipedia14b] Wikipedia.org "BitBlt BitBlit", Wikipedia.org, fetched 2014 BitBlt binary block transfer operator/operation. [Wikipedia14c] Wikipedia.org "Halftone", Wikipedia.org, fetched 2014 halftone image typically 300 dpi or 65 lpi resolution on laser printer, lower on display [Wikipedia14d] Wikipedia.org "Application Framework", Wikipedia.org, fetched 2014 Standard structure of an application (look-and-feel / GUI). OS-specific examples (MacApp), Cocoa (iPad), also cross-platform (Qt, wxWidgets, etc.), NetBeans, KDE. [Wikipedia14e] Wikipedia.org "Multics", Wikipedia.org, fetched 2014 General information on Multics OS "Multiplexed Information and Computing Services": access control lists on files/objects, all files memory-mapped. [Wikipedia14f] Wikipedia.org "Software Framework", Wikipedia.org, fetched 2014 Generalized concept for application framework: includes inversion of control, default behavior, extensibility, non-modifiable framework code [Wikipedia15a] Wikipedia.org "UNIX System V", Wikipedia.org, fetched 2015 History of UNIX System V through 2014 [Wikipedia15b] Wikipedia.org "Microsoft Windows", Wikipedia.org, fetched 2015 History of Microsoft Windows releases and versions. Windows 9X/ME operating system. Early versions used MS/DOS for file system, Windows had own device drivers, (cooperative) multitasking, segment-based software virtual memory: compare Android on Linux. [Wikipedia15c] Wikipedia.org "Code Signing", Wikipedia.org, fetched 2015 PKI signatures and timestamps to authenticate software: Cites to Lotus Notes Release 1.0. Alternative to CA certificates is to embed public key in installer. [Wikipedia15e] Wikipedia.org "Remote Procedure Call", Wikipedia.org, fetched 2015 RPC / RMI: remote method invocation. Early history in Arpanet. Summary listing of protocols and standards. [Wikipedia15f] Wikipedia.org "Inkwell (Macintosh)", Wikipedia.org, fetched 2015 Inkwell/Ink handwriting recognition software on Mac OS X, originated in Newton. Not used in iPad/iPhone/iPod. File contains additional materials. Press-and-hold to switch from inking/handwriting to mouse mode on touchscreen. [WilsonG14a] Wilson, Graham; Carter, Tom; Subramanian, Sriram; and Brewster, Stephen "Perception of Ultrasonic Haptic Feedback on the Hand: Localisation and Apparent Motion", Proc. CHI 2014, April 26..May 1, 2014, Toronto CA, pp. 1133..1142 Study of resolution/accuracy of haptic feedback using ultrasonic acoustic radiation pressure. Discussion of various nerve sensors: Pacinian corpuscles (vibration); mechanoreceptors have higher acuity. Hand-gesture sensors and tactile feedback mounted on user's hand. [XiaH14a] Xia, Haijun; Jota, Ricardo; McCanny, Benjamin; Yu, Zhe; Forlines, Clifton; Singh, Karan; and Wigdor, Daniel "Zero-latency tapping: using hover information to predict touch locations and eliminate touchdown latency", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 205..214 Experimental system using 3D "hover" information on fingertip to predict point of contact, and signal touchdown with less latency. Microsoft Surface (Samsung SUR40 tabletop / PixelSense) with 3D camera tracking system of marked fingertips. See other Wigdor, Forlines, Jota references. [YamaokaJ14a] Yamaoka, Junichi and Kakehi, Yasuaki "A pen-based device for sketching with multi-directional traction forces", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 43..44 Haptic feedback (simulated traction/friction/repulsive forces) on stylus, using asymmetric vibrators for any X/Y direction. Cites to Traxion by Rekimoto. Suggests using force to guide/teach user, rather than to simulate feel of virtual objects (brush, etc.), due to limited angular accuracy of prototype. [YoonD14a] Yoon, Dongwook; Chen, Nicholas; Guimbretiere, Francois; and Sellen, Abigail "RichReview: Blending Ink, Speech, and Gesture to Support Collaborative Document Review", Proc. UIST '14, October 5..8, 2014,



Honolulu, pp 481..490 Real-time and asyncronous (recorded) document review system including electronic ink annotations, syncronized voice, pointing, gestures. Hovering places "highlighter" color for pointing. Cites to Wang Freestyle: Freestyle could also do pointing gestures via "hover". Users spontaneously allocated particular space for comments: bottom of page for general comments on page, to side of text for comments on particular text. Compare with Anoto SmartPen? Space can be expanded for additional ink (+ voice) annotations: compare with van Raamsdonck? [ZeaglerC14a] Zeagler, Clint; Gilliland, Scott; Freil, Larry; Starner, Thad; and Jackson, Melody Moore "Going to the Dogs: Towards an Interactive Touchscreen Interface for Working Dogs", Proc. UIST 2014, October 5..8, 2014, Honolulu, pp. 497..507 Interesting study of abilities of service dogs to learn to use a touchscreen interface: applications include calling emergency numbers, etc. Also found that ability to learn and perform this task ("drive") predictive of ability to focus and work as guide guide dog (to save training expense on dogs without enough "drive"). Authors appeared confused about multi-touch vs. single-touch touchscreens: switched to IR/LED optical touch screen (not identified) because saliva and moisture from dog's snout (one dog spontaneously used paw) affected capacitive touchscreen. [ZhaoC14a] Zhao, Chen; Chen, Ke-Yu; Aumi, Md Tanvir Islam; Patel, Shwetak; and Reynolds, Matthew S. "SideSwipe: Detecting In-air Gestures Around Mobile Devices Using Actual GSM Signals", Proc. UIST '14, October 5..8, 2014, Honolulu, pp 527..534 Proximity sensor used for hand gestures on smartphone: additional antennas pick up reflect GSM signal from smartphone itself, hand proximity affects signals received by multipath reflection/propagation. Recognize in-air swipe gestures, proximity on different edges and in front. biblio13 (Prev page)

Main Page

(Next page) biblioxx



((unindexed)..(uncategorized)) History of Pen and Gesture Computing

History of Pen and Gesture Computing: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers References from the approximate years (unindexed) to (uncategorized). This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2015. Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement: Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward

Most of these are in my personal collection, either as physical items or electronic files. I can be contacted at [email protected]. Copyright © Tue Jul 14 21:09:25 EDT 2015 . Mirrors of this page can be found at: http://www.ruetersward.com/pens/biblioxx.html http://users.rcn.com/rwservices/pens/biblioxx.html http://rwservices.no-ip.info:81/pens/biblioxx.html (Generally most up-to-date) biblio15 (Prev page)

Main Page

(Next page) penhist

biblio15 (Prev page)

Main Page

(Next page) penhist


http://rwservices.no-ip.info:81/pens/biblioxx.html[7/14/2015 9:47:02 PM]

Images of early pen-computing products

Videos and pictures of some interesting items in my collection. I have a lot of old pen-computing "junk". I have started to post pictures of a few things I think are particularly interesting. First, a few video files: ... I used to have a collection of links to where other people (with more bandwidth) had the video files, but some of them died from link rot. (Or in one case on YouTube, from file rot.) Now, I keep a copy of the videos here -- your access will be slower than YouTube, though.

1985: Pencept PenPad: Software Control at the Stroke of a Pen This is a commercial video that contains excerpts of the the demonstration at CHI '85 of the Pencept PenPad and keyboardless/mouseless operation of various applications. The actual CHI video is more extensive (and IMHO, more interesting), but I don't have a copy. The person demonstrating operation is Andrew Nilsson, Pencept's Director of Marketing. Andy and I were co-inventors on one of Pencept's UI patents. Pencept's specialty was a proprietary technology for robust on-line recognition of hand-printed (or scribbled) characters and symbols, combined with pointing input at the same time. Pencept was not a large enough company to have the resources to develop a custom operating system (like GO's PenPoint) or custom applications (like Wang's Freestyle), and instead developed innovative UIs for their technology to front-end existing keyboard and CAD/CAM graphic applications. Pencept Penpad 320: Software Control at the Stroke of a Pen (LARGE file) Pencept Penpad 320: Software Control at the Stroke of a Pen (smaller file)

1988: Wang Freestyle electronic ink / voice annotation application. This is demonstrated by Dr. Stephen Levine, my boss on the Freestyle project at Wang Laboratories. Freestyle was an application that would take a screen capture from a DOS application, and let you add handwriting and voice annotations to it. There was no handwriting recognition: as a whiteboard application, it was very sophisticated at the time. I think Wang found only one customer for the product. Freestyle demo part 1 Freestyle demo part 2

1989: IBM "Paper-like Interface", demonstrations of various prototype applications. I visited Dr. Charles Tappert at IBM at about the same time. Dr. Tappert developed the handwriting recognizer used in the application demonstrations. The small research group there had developed a few different pen-computing applications as prototypes. The operating system was DOS: only the applications used pen computing, you had to use a keyboard to run the operating system. IBM Paper-Like Interface: applications.

1991: Demonstration of PenPoint OS While at Slate, I worked on applications for both the PenPoint OS, and Windows for Pen Computing -- that was the historical predecessor to the Tablet PC software that came out about ten years later. Here is a video demonstration from GO Corporation of the PenPoint OS. PenPoint OS Demonstration.

Alan Kay and Dynabook Alan Kay is well-known for the Dynabook proposal and for his conceptual work in the 1960's and 1970's on personal use of computers, particularly for education. He has given a number of talks over the years about his view of the history behind the ideas for a personal computer. The videos are interesting, because they include segments from very early demonstrations of the RAND tablet, the GRAIL system, and other handwriting/gesture systems of the 1960's. As Alan Kay himself said (in the 2008 Video), history is a slippery subject: For example, Alan Kay has said that the Dynabook concept included a touchscreen or tablet, but this was evidently a proposal made by another person person associated with Alan's research group at the time, and never carried out. Also, the RAND tablet was not the first electronic tablet: that could have been either the Styalator from the 1950's, or Eliza Gray's Telautograph from the late 19th century. PenPoint91b_Penpoint_Demonstration.avi. KayA87a_Alan_Kay_Doing_With_Images_Makes_Symbols_1987.mp4. 2005_Alan_Kay_What_is_Squeak.mp4. 2008_Alan_Kay_40_Years_of_Dynabook.mp4.

1983: Pencept PenPad M200 Handwriting Terminal

http://rwservices.no-ip.info:81/pens/images.html[7/14/2015 9:47:05 PM]


Here are several views of a Pencept PenPad M200 model: this was a no-keyboard, no-mouse "handwriting terminal" that was a direct replacement for the VT-100 computer terminal that was the de facto standard terminal in the late 70's and early 80's. ... you may want to download the images from the link, to see them full-size.



1986: Casio PF-8000-s calculator. I have two of these in my personal collection: they still work. It is a one-of-a-kind design: rather than using a touchpad or digitizer, it uses a set of rubberized buttons on the right side: Casio got a patent on the use of buttons instead of a digitizer or touchscreen. Copy of the sales brochure for the PF-8000s



This is one of the PF-8000s units in my collection:

Excerpts from Casio user's manual, showing elements of character recognition:



... you may want to download the images from the link, to see them full-size.



Here are some pictures of a working PenPoint "Hobbit" model in my collection. (I also have a working Apple Newton, but they are relatively easy to find on eBay and elsewhere.)






[email protected]: Notes on the (relatively unknown) History of Pen-based Computing -- Jean Renard Ward

Notes on the (relatively unknown) History of Pen-based Computing Copyright © Jean Renard Ward, 1992, 1996, 2003, 2005, 2008, 2011, 2013 Contact: jrward_at_alum.mit.edu Permission is hereby given to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement: Source: Annotated Bibliography in Touch/Pen Computing and Handwriting Recognition by Jean Renard Ward". This is a short outline of a presentation on the history of "pen computing" I gave for the Pen Computing group of the Boston Computer Society, March 11, 1992. Unfortunately, I did not keep the full set of slides, which had pictures of the products and systems. However, Dan Bricklin (CTO at Slate Corporation at the time) has posted a video of the same presentation on youtube, which does show all the pictures and slides: http://www.youtube.com/watch?v=4xnqKdWMa_8.

Topic threads for tonight ... Pen Computing is not new: "A lot of it has happened before" (... with selected examples, mostly from commercial development) Some specific shortcomings of the current state of the Pen-Computing art (... they are not what you might think) What this implies ... or, "What should we do?"

Why this "historical view" might be of interest to this audience: Historical developments are relatively unknown, and quite voluminous. Present a broad range of case-study examples. Early examples were done by workers motivated by other interests, with a different perspective ... man-machine-interfaces (sic), human functionality and usability, cognitive concerns a study problem for pattern recognition ... not in modern "GUI" development. Current problems: There are neglected / overlooked issues, falling under the rubric of human factors, that affect and may threaten the usability and viability of pen-based systems. Especially: "Electronic Ink" (hardware and software) Topics not addressed tonight: No product demonstrations ... No applications demonstrations ... Other presentations coming up: Slate is presenting to BCS general meeting, May 21. Society for Information Display (S.I.D.) is meeting in Boston in May. Videotape demo of certain applications.

Historical overview: There is a repeated theme in commercial developments, namely the "Lief Ericsson in 1992" scenario --"I/we discovered/invented pen computing, pen computers, pen-based systems."

"The Archeology of Pen-based computing"

http://rwservices.no-ip.info:81/pens/penhist.html[7/14/2015 9:47:08 PM]


1914 Hyman Eli Goldberg, U.S. Patent 1,117,184, "Controller". ... on-line (not OCR) recognition of hand-written numerals to control a machine in real-time. 1938 George Hansel, U.S. Patent 2,143,875, machine recognition of handwriting. G.G. Neil Wright, "The Writing of Arabic Numerals"

"The Ancient History of Pen-based computing" 1956 RAND tablet ... invention of the digitizing tablet, for a handwriting recognition project. (Actualy the Stylator came first!) 1957 T.L. Dimond, "Reading Handwritten Characters", EJCC Claimed 97% recognition rate, and a better input device for humans than a keyboard 1962 Masterson and Hirsch, IRE Transactions on Human Factors and Electronics 99.79% correct recognition. 1962 J.C.R. Licklider, "On-line Man-computer Communication", SJCC 1961/3 Sitari, Harmon, Frishkopf, et al cursive script recognition at Bell Labs 1966 Groner et al, "GRAIL System", RAND Corporation Text/character recognition, gesture commands, sketch recognition

"The Early Commercial History of Pen-based computing" 1973 Applicon Corporation / Ledeen recognizer Gesture-based command input, with hot-points, in a widely-praised GUI for a commercial CAD system 1974 SRI / Xebec Systems Incorporated Text input to computer with a pen 1976 S-I Hanaki, NEC handwriting kana/romanji billing machine product 1981 Several commercial vendors, small units, some portable MicroPad ImageData Telepad WriteAway many Japanese (Application areas: data-capture, data-entry)

"The Modern History of Pen-based computing" 1983 PC-based or -oriented commercial products PenPad CIC Handwriter NestorWriter Included "front-end" interfaces to word-processing systems, CAD/Paint systems, spread-sheet input (Note: these companies are still around) Special note: even perfect recognition does not solve the problem! Gould et al, "Composing Letters with a Simulated Listening Typewriter", CACM April 1983. 1985 "Experimental" systems AEG (Germany) Word-processing IBM (Tappert) 1986 Linus Technologies Application areas: walking data capture (nurses), financing from Baxter Medical 1986/7 MAC-based products: Personal Writer / Anatex

"Current Events of Pen-based computing" 1989 GridPAD Application area: remote data capture (surveys, inventory, shipping/receiving) 1990/91/92 Too many to mention ... "Pen OS" - PenPoint, Windows for Pens "Pocket" - Poqet , Sharpe Wizard "Tablets" - Wang/Momenta/Grid/Eden Group/SuperScript/ScriptWriter/ ... Application areas: Highly-mobile data capture Note taking / capture Document capture and display "Imaging systems" NOTE: All of these application areas involve "electronic ink", data that is fundamentally in written form (sketches by insurance adjusters, for example), that does not need to be recognized as text.



There will be a short video of some representative applications and software (So there is some commercial content after all - Dang!)

Specific shortcomings of the current art: Electronic ink: State of the Art -- Special considerations: Existing, alternative technology for data capture, note capture, and documents is paper and pen, or paper and pencil, ... not any current computer technology. Paper-based systems are highly evolved --> equivalent to well designed. "Ink" APIs and functions are not highly evolved: editing scaling storage embedding Current hardware designs for electronic ink are NOT adequate, due partly to ignorance of physical effects of human use on part of designers. Displays (LCD): State of the Art Low contrast No backlight LCD parallax / shadowing Glare Coarse resolution Slow ("fuzzy motion"), especially when cold Greyscale mapping Digitizers: State of the Art Bad electronic ink vs. real ink Comparison: Teletype keyboard vs. typewriter keyboard action Tablet electronic effects: (especially at low power!) Tablets produce very distorted images ... "it's not my signature" "it doesn't go where I point/draw" Spiking Fixed line distortions Velocity errors Angle of stylus Poor tracking with height Tablet mechanical effects: Slickness Tip force Surface hardness Stylus design: corded / handed surface texture weight and handling There will be a short video showing digitizers

Summary: So ... What does this history mean ... to us? What applications have been shown to be inappropriate? Patentability? Why should we think there is a market? There is a long historical perspective Examples for human-factors, application, market, U/I, and human-machine design study Old (obvious?) applications were not successful. New applications are highly mobile, in unfamiliar areas, and characterized by functionality such as electronic ink or



mobile data access, not character recognition Note takers Daytimer (R) Pen Scheduler PenBook (R) Hardware for electronic ink requires a design much more to ergonomic and human factors goals than has been the case. Back / Zurück Comments:

[email protected]


Web pages hosted at http://rwservices.no-ip.info:81

Rueters-Ward Services homepage Web pages currently hosted at this site: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers http://users.rcn.com/rwservices/biblio.html http://www.ruetersward.com/biblio.html http://rwservices.no-ip.info:81/biblio.html http://146.115.170.218:81/biblio.html C.V. and information for intellectual property / patent consulting http://users.rcn.com/rwservices/JRW_cv.pdf http://www.ruetersward.com/JRW_cv.pdf Development/Engineering resume for Jean Renard Ward http://users.rcn.com/rwservices/jrw.html http://www.ruetersward.com/jrw.html Dr. Betty J. Ward's Doctoral Dissertation: http://www.ruetersward.com/drbettyjwardph/bwthesis.html http://rwservices.no-ip.info:81/drbettyjwardph/bwthesis.html http://146.115.170.218:81/drbettyjwardph/bwthesis.html Rueters-Ward family photo album (under construction): http://ruetersward.com/RuetersWardFamily/Picture_index.htm http://rwservices.no-ip.info:81/RuetersWardFamily/Picture_index.htm http://146.115.170.218:81/RuetersWardFamily/Picture_index.htm RHS '69 Re-union and Yearbook pages: http://ruetersward.com/RHS/RHS_index.htm http://rwservices.no-ip.info:81/RHS/RHS_index.htm http://146.115.170.218:81/RHS/RHS_index.htm Information about Rueters-Ward Services: http://rwservices.no-ip.info:81/jrw.html http://146.115.170.218:81/jrw.html Rueters-Ward Intranet access: http://192.168.1.196:81 http://192.168.1.196:81/biblio.html http://192.168.1.196:81/drbettyjwardph/bwthesis.html http://192.168.1.196:81/RuetersWardFamily/Picture_index.htm http://192.168.1.196:81/RHS/RHS_index.htm http://192.168.1.196:81/jrw.html (Copyright © 20150711 21:31:38 EDT)

http://rwservices.no-ip.info:81/index.html[7/14/2015 9:47:09 PM]


Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers (DOI: 10.13140/2.1.3018.8322) This is a posting of a bibliography on gesture user interfaces, on-line character recognition (a.k.a. dynamic character recognition, a.k.a. pen computing), and related topics, including both hardware and software. I am posting it as a service to those with interest in the field. It may also be of special interest to anyone investigating any of the flood of patents in the areas of digitizer tablets, character recognition, tablet-PC GUIs, and multi-touch computing (such as the iPhone and Touch iPod). It covers the time period from approximately 1891 (first electronic tablet) through 1914 (first handwriting-recognition input system) to the present day. This compilation is copyright © Jean Renard Ward, 1992, 1996, 2003, 2005, 2008, 2009, 2011, 2013, 2015. Permission is hereby given to link to this list, or to cite it provided that notice of the source is given as stated below, that the full URL of this link is included, and also that you use my correct full name "Jean Renard Ward" is included with any such reference or link. Permission is hereby given to reproduce any diagrams or photographs in this collection for any purpose (to the extent that the materials are not subject to copyright by persons other than myself), including for the purposes of confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement: "Source: Annotated Bibliography in Touch/Pen Computing and Handwriting Recognition by Jean Renard Ward". The bibliography is broken up into several web pages, for convenience in printing: 1891 to 1984 to 1991 to 2001 to 2011 to Pictures

1970 1971 to 1975 1976 to 1980 1981 to 1983 1985 1985 to 1986 1987 to 1988 1989 to 1990 1993 1994 to 1995 1996 to 1998 1999 to 2000 2003 2004 to 2005 2006 to 2007 2008 to 2010 2013 2014 to 2015 not yet catalogued and Videos

When was Pen Computing invented? Check this out: Notes on the "unknown" history of Pen Computing (from a talk given to the Boston Computer Society in 1992) Dan Bricklin has posted a video of the presentation on youtube.com: http://www.youtube.com/watch?v=4xnqKdWMa_8. Note that there is a tendency for major vendors to re-name Pen Computing ("Tablet PCs", "Touch PCs", "Organic User Interfaces" are a few such names), which tends to obscure the historical record a bit. Who invented Pen Computing? You may have read something somewhere that indicated some organization like Microsoft Nestor Apple Newton MicroPad IBM PenCept GO Corporation Rand Corporation Linus PenPoint CIC Wacom GRAIL Casio Slate Graffiti ... had invented pen computing / tablet computing. After skimming through some of the references above, you might start to think some of those claims - particularly those by large corporations - could be slightly exaggerated: for

http://rwservices.no-ip.info:81/pens/..\biblio.html[7/14/2015 9:47:09 PM]


example, inventing an improvement on spelling correction [Viterbi67] is not the same as inventing the first graphical word-processing program. Which is not to say that a particular system did not do some very interesting new things within the context of pen computing, such as [CarrR91a] and [PenCept85f]. Note: I find the very early references to technology interesting. For example, Handwriting recognition by a machine in real time, with a user writing with a stylus, goes back to before World War I: United States Patent 1,117,184, November 17, 1914 [GoldbergHE1914] United States Patent 1,311,384, July 29, 1919 [DrewWF1919] Optical character recognition (from a printed page) goes back to before the days of computers: United States Patent 1,838,398, December 29, 1931 [GoldbergE31a] Electronic Tablets were invented in the 19th century: United States Patent 461,472, October 20, 1891 [Gray1891b] United States Patent 491,347, February 7, 1893 [Gray1893b] (Note: Elisha Grey is best known in history as the person who may have invented the telephone before Alexander Graham Bell, but lost the patent dispute in a famous and controversial court decision) "Electronic ink" (the pen-computing kind, not the electronic display) is older than you think: "The RAND Tablet: A Man-Machine Graphical Communication Device", 1964 [DavisMR64b] (Note: This is the earliest published use of this term in pen-computing, AFAIK). Language knowledge being used in character recognition: The early references to the "Viterbi" algorithm are interesting to compare with more recent inventions on glyphs and context. [Viterbi67] So you think your recognition software is pretty good? ... look at this. Another interesting slant is the many claims of "near perfect" handwriting recognition algorithms in the earliest days of pen computing, such as [Simek67] and [DimondTL57a] (Note: At one of my pen-computing jobs in the late 1980s, we joked that Dimond had achieved 97% "perfect" handwriting recognition in 1957, and that the technology had been getting about 5% better every decade since: the trend seems to have continued unbroken, like some odd form of Moore's Law.) You can find almost anything using one of the major search engines on the Internet these days. However, here are some links to other pages with talking about the history of pen computing. (Please note that links expire, so these may not all be up-to-date.) Images of early PenComputing products (a work in progress -- I have lots more in my attic). A talk on the "unknown" history of Pen Computing given to the Boston Computer Society in 1992. Schomaker's MIAMI group in the "Esprit" project includes a detailed taxonomy and another set of references relating to pen computing. Andreas Hennig has posted another (shorter) bibliography relating to pen computing. Nottingham Trent University has a research group in this area. Gary Perlman set up an extensive on-line bibliography on human-computer interaction. Bill Buxton has been working on haptic user-interfaces at various institutions for a long time. You can find his most recent work at www.billbuxton.com The "Pen Report" has references to many of the commercial developments in one of the more recent pencomputing "crazes". It is listed as part of the AMUG Newton page, one of many user-group pages on the Web. There have been some original papers posted in odd places, such as this one by Scattolin Patrice. Of course, there is information on Wikipedia, but you should be cautious that the information can be biased towards a particular organization. Thank you, Jean Renard Ward

EMail: [email protected]



Mirrors of this page can be found at: http://ruetersward/biblio.html http://users.rcn.com/rwservices/biblio.html http://rwservices.no-ip.info:81/biblio.html Back to homepage for Rueters-Ward / Zurück zur Heimatseite Rüters-Ward (Copyright © 20150711 21:31:40 EDT)
