Software Applications for Cultural Diversity Rod Davis, SIL International

About SIL International faith-based International NGO faith-based International NGO language-based development language-based development research, translation and literacy research, translation and literacy active in 20% of the world’s languages active in 20% of the world’s languages

SIL Language Software Development 53 software development personnel 53 software development personnel Member of the Unicode Consortium and very active on behalf of minority languages Member of the Unicode Consortium and very active on behalf of minority languages 60 software titles to support the work of language fieldworkers 60 software titles to support the work of language fieldworkers

One of the Biggest Challenges: Dealing with Complex Scripts Encoding Encoding –Transition to Unicode Input (Keyboarding) Input (Keyboarding) –Complexity –Extensibility Type Design Type Design –Unicode based –Smart-font compiler technology Rendering Rendering –Diacritic placement, Contextual shaping, Ligatures, Reordering/splitting, Bi-directionality

Solutions in Dealing with Complex Scripts Encoding Encoding –TechKit, a utility to convert legacy encodings to Unicode – SIL resource site with information, tutorials, utilities, etc. for making the transition to Unicode Input Input –Keyman (Windows); –KMFL (a Linux ‘Keyman’ still under development) –IMEs (Input Method Editors) Type Design Type Design –Graphite Compiler adds Graphite rendering tables to a TrueType font, giving it ‘smart-font’ capability Rendering Rendering –Graphite rendering engine: Can handle basic display of any complex script in use today. NOTE: These technologies can benefit not only ‘complex scripts’, but also orthographies with special diacritics or a few non-Roman characters. NOTE: These technologies can benefit not only ‘complex scripts’, but also orthographies with special diacritics or a few non-Roman characters.

What is a Smart Font? A font containing data describing how the glyphs are to displayed. A font containing data describing how the glyphs are to displayed. The smart font data is in the form of tables inside the font file itself. The smart font data is in the form of tables inside the font file itself. A rendering engine interprets the font tables to appropriately render the glyphs. A rendering engine interprets the font tables to appropriately render the glyphs. In contrast, a ‘dumb font’ has only a direct correspondence between the data characters and the displayed glyphs. In contrast, a ‘dumb font’ has only a direct correspondence between the data characters and the displayed glyphs.

What is Graphite? A package, developed by SIL International, that can be embedded into other applications, adding "smart font behavior" to that application. A package, developed by SIL International, that can be embedded into other applications, adding "smart font behavior" to that application. This package includes the following: This package includes the following: –A programming language (GDL) for specifying the font's behavior –A compiler for building the smart font –A rendering engine for displaying text using the smart font Note that it is open-source: Note that it is open-source:

What Makes Graphite Special? It differs from other complex script technologies: It differs from other complex script technologies: –Unlike OpenType, it does NOT assume that script-specific information is incorporated at the application or operating system level (as in Uniscribe). –It is extensible, and provides support for characters in Unicode’s Private Use Area ranges … unlike Uniscribe, which handles only script behaviors already part of the Unicode standard. –It is an open-source solution versus proprietary. –It can handle basic display of any complex script in use today. –The following slides demonstrate the types of rendering difficulties that Graphite was designed to handle:

SIL’s Work with UNESCO In 2003, SIL International and UNESCO engaged in a cooperative project as part of UNESCO’s Initiative effort. In 2003, SIL International and UNESCO engaged in a cooperative project as part of UNESCO’s Initiative effort. Initiative Initiative Goal: Enable the development of complex script support in information and communication technologies (ICTs). Goal: Enable the development of complex script support in information and communication technologies (ICTs). Project SILA: A Graphite-enabled version of Mozilla Project SILA: A Graphite-enabled version of Mozilla –Project Goal: To enable minority language communities to publish on the Internet … Graphite-enabled Edit Control Version 0.9 Graphite-enabled Edit Control Version 0.9 –basic Graphite edit control for data input applications –SDK for Windows 2000/XP, developer’s guide and well-commented source code Modified version of WorldPad -- Graphite-Enabled Text Processor Modified version of WorldPad -- Graphite-Enabled Text Processor –Simple text processor for Windows 2000/XP

Graphite-enabled Mozilla Contrast Internet Explorer on the left with Graphite-enabled Mozilla on the right. The Graphite-enabled version of Mozilla displays the multiple diacritics separately as intended.

Graphite-enabled Mozilla

Graphite-enabled Mozilla Instant Messaging

Graphite-enabled WorldPad

Lessons learned We are participating in or have created several open source development efforts. We have learned that… –FLOSS (Free/Libre Open Source Software) Linux is very attractive to low income groups we work with. –Setup and maintenance of open source OS and software is too complex for most of our end users. –Open source development is complex. –Getting support for complex minority scripts into the “core” builds of open source software is hard. –Localization of software is complex and time consuming. –Despite the difficulties, the benefits are great and we are heading in this direction on a number of fronts.

What is Keyman? Keyman allows you to enter text in Windows®-based applications in other languages without changing your physical keyboard (or system keyboard). Keyman allows you to enter text in Windows®-based applications in other languages without changing your physical keyboard (or system keyboard). It does this by remapping the character keys according to the font for the language you wish to use. It does this by remapping the character keys according to the font for the language you wish to use. It works with both ANSI” and Unicode. It is in wide-use and well tested. It works with both ANSI” and Unicode. It is in wide-use and well tested. Keyman 6.1 is a commercial program available at a reasonable price: Keyman 6.1 is a commercial program available at a reasonable price: The development tool called ‘Keyman Developer’ is sold separately, and uses a rule-based programming language. The development tool called ‘Keyman Developer’ is sold separately, and uses a rule-based programming language. Note: There are some issues with very large character sets that are handled better with IMEs (Input Method Editors). However, the IMEs are more complicated and are not really user modifiable or extensible. Note: There are some issues with very large character sets that are handled better with IMEs (Input Method Editors). However, the IMEs are more complicated and are not really user modifiable or extensible.

Technical overview of Keyman TIKE Compiler Keyman Program (.KMN) Keyman Executable (.KMX) Driver Operating system Application software Config- uration Program keystrokes characters

SIL Unicode-based Fonts Doulos SIL 4.0: A linguist’s general use font … This is a comprehensive inventory of glyphs needed for almost any Roman- or Cyrillic-based writing system, whether used for phonetic or orthographic needs. Status: Released but not a ‘full-family’ font (regular type-face only) Doulos SIL 4.0: A linguist’s general use font … This is a comprehensive inventory of glyphs needed for almost any Roman- or Cyrillic-based writing system, whether used for phonetic or orthographic needs. Status: Released but not a ‘full-family’ font (regular type-face only) Gentium is a typeface family designed to enable the diverse ethnic groups around the world who use the Latin script to produce readable, high-quality publications. It supports a wide range of Latin-based alphabets and includes glyphs that correspond to all the Latin ranges of Unicode. Status: Released and in wide use; Linux installation also released. Gentium is a typeface family designed to enable the diverse ethnic groups around the world who use the Latin script to produce readable, high-quality publications. It supports a wide range of Latin-based alphabets and includes glyphs that correspond to all the Latin ranges of Unicode. Status: Released and in wide use; Linux installation also released. Additional Unicode-based fonts, including one specifically tuned for literacy use, are under development. Additional Unicode-based fonts, including one specifically tuned for literacy use, are under development. (Just remember ) (Just remember )

Language and Culture tools SIL International has also developed software tools that serve the following needs: Lexical database programs that facilitate: Lexical database programs that facilitate: –Grammatical/morphological analysis –Dictionary creation and printing –Web publication of lexicons –Entry and analysis of cultural data Language Survey tools for doing statistical comparison of related languages/dialects Language Survey tools for doing statistical comparison of related languages/dialects Speech Tools for doing phonetic analysis of speech (including tonal analysis) Speech Tools for doing phonetic analysis of speech (including tonal analysis) An extensive CD-Rom based reference tool that covers all aspects of language and cultural field work An extensive CD-Rom based reference tool that covers all aspects of language and cultural field work

FieldWorks Data Notebook for analyzing cultural data

ToolBox for language data management

Lexique Pro for publishing interactive lexicons

WordSurv and PalmSurv to compare language survey data

Speech Tools for analyzing speech data See

LinguaLinks Library for electronic reference materials to support language and culture fieldwork Anthropology Anthropology Consulting Consulting General Reference Works General Reference Works Language Learning Language Learning Linguistics Linguistics Literacy Literacy Sociolinguistics Sociolinguistics Translation Translation

Partnering in Script Development Things SIL International could potentially offer: – –Support script development for languages currently unsupported – –Partner in the development of keyboarding solutions – –Continued development of the Graphite rendering technology – –Support incorporating these technologies into current open source or commercial tools, e.g., Open Offiice Things SIL International would benefit from: – –Resources necessary to actually develop the scripts (people, money, hardware/software) – –Community cooperation in the standardization of scripts – –Help getting open source software groups to incorporate these technologies into their core software efforts

Web-sites of Interest SIL International: SIL International: Multilingual Computing: Content: Here are Links to some SIL contributions to research and development in the area of multilingual computing, links to additional resources, and a glossary of Character Encoding and Rendering. Multilingual Computing: Content: Here are Links to some SIL contributions to research and development in the area of multilingual computing, links to additional resources, and a glossary of Character Encoding and Rendering. Fonts in Cyberspace: Content : This is a guide to finding language fonts on the Internet. Containing more than 400 sources for 123 languages. Fonts in Cyberspace: Content : This is a guide to finding language fonts on the Internet. Containing more than 400 sources for 123 languages. NRSI: Computers & Writing Systems: Content: Character Encoding, Understanding Unicode, Keyboard Design And Keyboarding Utilities, Type Design, and Script Rendering Technologies NRSI: Computers & Writing Systems: Content: Character Encoding, Understanding Unicode, Keyboard Design And Keyboarding Utilities, Type Design, and Script Rendering Technologieshttp://scripts.sil.org SIL Software Catalog: Content: SIL has developed more than 60 pieces of software to support the work of its fieldworkers; most available to the public for free download. SIL Software Catalog: Content: SIL has developed more than 60 pieces of software to support the work of its fieldworkers; most available to the public for free download. Keyman 6.1: Content: Keyman is a commercial Windows keyboard mapping software that, with Keyman Developer, is used to design your own keyboard layouts. Keyman 6.1: Content: Keyman is a commercial Windows keyboard mapping software that, with Keyman Developer, is used to design your own keyboard layouts. SIL FieldWorks 2.0: SIL FieldWorks is a suite of software tools that work together to help language teams worldwide. The current released suite of programs includes WorldPad and the SIL FieldWorks Data Notebook. Additional applications are under development. SIL FieldWorks 2.0: SIL FieldWorks is a suite of software tools that work together to help language teams worldwide. The current released suite of programs includes WorldPad and the SIL FieldWorks Data Notebook. Additional applications are under development. Field Linguist’s Toolbox: Toolbox is a data management and analysis tool for field linguists. It is especially useful for maintaining lexical data, and for parsing and interlinearizing text, but it can be used to manage virtually any kind of data. Field Linguist’s Toolbox: Toolbox is a data management and analysis tool for field linguists. It is especially useful for maintaining lexical data, and for parsing and interlinearizing text, but it can be used to manage virtually any kind of data. Lexique Pro: Lexique Pro is an interactive lexicon viewer, with hyperlinks between entries, category views, dictionary reversal, search, and export tools. It's designed to display your data in a user-friendly format so you can distribute it to others. Lexique Pro: Lexique Pro is an interactive lexicon viewer, with hyperlinks between entries, category views, dictionary reversal, search, and export tools. It's designed to display your data in a user-friendly format so you can distribute it to others.