The Unicode cookbook for linguists
This text is a practical guide for linguists, and programmers, who work with data in multilingual computational environments. We introduce the basic concepts needed to understand how writing systems and character encodings function, and how they work together at the intersection between the Unicode...
Сохранить в:
| Главные авторы: | , |
|---|---|
| Формат: | Online |
| Язык: | английский |
| Опубликовано: |
Language Science Press
2021
|
| Предметы: | |
| Online-ссылка: | 1001685 |
| Метки: |
Нет меток, Требуется 1-ая метка записи!
|
| _version_ | 1869531645583818752 |
|---|---|
| author | Moran, Steven Cysouw, Michael |
| author_browse | Cysouw, Michael Moran, Steven |
| author_facet | Moran, Steven Cysouw, Michael |
| author_sort | Moran, Steven |
| collection | Directory of Open Access Books |
| description | This text is a practical guide for linguists, and programmers, who work with data in multilingual computational environments. We introduce the basic concepts needed to understand how writing systems and character encodings function, and how they work together at the intersection between the Unicode Standard and the International Phonetic Alphabet. Although these standards are often met with frustration by users, they nevertheless provide language researchers and programmers with a consistent computational architecture needed to process, publish and analyze lexical data from the world's languages. Thus we bring to light common, but not always transparent, pitfalls which researchers face when working with Unicode and IPA. Having identified and overcome these pitfalls involved in making writing systems and character encodings syntactically and semantically interoperable (to the extent that they can be), we created a suite of open-source Python and R tools to work with languages using orthography profiles that describe author- or document-specific orthographic conventions. In this cookbook we describe a formal specification of orthography profiles and provide recipes using open source tools to show how users can segment text, analyze it, identify errors, and to transform it into different written forms for comparative linguistics research. |
| format | Online |
| id | doab-20.500.12854ir-34976 |
| institution | Directory of Open Access Books |
| language | eng |
| publishDate | 2021 |
| publishDateRange | 2021 |
| publishDateSort | 2021 |
| publisher | Language Science Press |
| publisherStr | Language Science Press |
| record_format | ojs |
| spelling | doab-20.500.12854ir-349762025-07-31T07:11:26Z The Unicode cookbook for linguists Moran, Steven Cysouw, Michael Linguistics This text is a practical guide for linguists, and programmers, who work with data in multilingual computational environments. We introduce the basic concepts needed to understand how writing systems and character encodings function, and how they work together at the intersection between the Unicode Standard and the International Phonetic Alphabet. Although these standards are often met with frustration by users, they nevertheless provide language researchers and programmers with a consistent computational architecture needed to process, publish and analyze lexical data from the world's languages. Thus we bring to light common, but not always transparent, pitfalls which researchers face when working with Unicode and IPA. Having identified and overcome these pitfalls involved in making writing systems and character encodings syntactically and semantically interoperable (to the extent that they can be), we created a suite of open-source Python and R tools to work with languages using orthography profiles that describe author- or document-specific orthographic conventions. In this cookbook we describe a formal specification of orthography profiles and provide recipes using open source tools to show how users can segment text, analyze it, identify errors, and to transform it into different written forms for comparative linguistics research. 2021-02-10T12:58:18Z 2018-10-11 23:55 2020-03-10 03:00:39 2020-04-01T12:19:47Z 2018-07-11 book 1001685 OCN: 1076699025 2364-8899 http://library.oapen.org/handle/20.500.12657/28277 9783961100903 https://directory.doabooks.org/handle/20.500.12854/34976 eng Translation and Multilingual Natural Language Processing open access image/jpeg image/jpeg image/jpeg image/jpeg n/a n/a n/a n/a https://library.oapen.org/bitstream/20.500.12657/28277/1/1001685.pdf https://library.oapen.org/bitstream/20.500.12657/28277/1/1001685.pdf https://library.oapen.org/bitstream/20.500.12657/28277/1/1001685.pdf https://library.oapen.org/bitstream/20.500.12657/28277/1/1001685.pdf Language Science Press 10.5281/zenodo.1296780 10.5281/zenodo.1296780 ed03121b-b998-4b50-8d58-1d0745565558 Knowledge Unlatched b818ba9d-2dd9-4fd7-a364-7f305aef7ee9 9783961100903 Knowledge Unlatched (KU) Language Science Press 2018-2020 Berlin 103595 Language Science Press 2018 - 2020 open access |
| spellingShingle | Linguistics Moran, Steven Cysouw, Michael The Unicode cookbook for linguists |
| title | The Unicode cookbook for linguists |
| title_full | The Unicode cookbook for linguists |
| title_fullStr | The Unicode cookbook for linguists |
| title_full_unstemmed | The Unicode cookbook for linguists |
| title_short | The Unicode cookbook for linguists |
| title_sort | unicode cookbook for linguists |
| topic | Linguistics |
| topic_facet | Linguistics |
| url | 1001685 |
| work_keys_str_mv | AT moransteven theunicodecookbookforlinguists AT cysouwmichael theunicodecookbookforlinguists AT moransteven unicodecookbookforlinguists AT cysouwmichael unicodecookbookforlinguists |