Previous section.

Portable Layout Services: Context-dependent and Directional Text

Portable Layout Services:
Context-dependent and Directional Text
Copyright © 1997 The Open Group

Complex-Text Languages - An Overview

This chapter contains text which is reprinted with the permission of the author, Israel Gidali.

This chapter presents an overview of aspects of the writing systems of a large family of languages that are collectively called complex-text languages. Complex-text Languages is an introduction to complex-text languages. Bidirectional Languages discusses bidirectional languages, in particular Arabic and Hebrew, including their use in a data processing environment. Thai Language and its Writing System and Korean Language and its Writing System describe other typical complex-text languages (Thai and Korean). Conclusions and Guidelines summarises the main principles and suggests a few guidelines for application developers.

Complex-text Languages

In the languages of the western world based on the Latin, Cyrillic and Greek scripts, there is no difference between how text is stored for data processing and how it is presented on a display or a printer. The text is read on horizontal lines from left to right, the lines progress from top to bottom and the characters are stored in a manner identical to how they are presented.

Not all the languages of the world have these characteristics.

In this document, complex-text languages are defined as those languages for which the text has a different layout when presented from when it is stored for data processing. The term layout, which is equivalent, in this context, to the term format, refers to the shape of the characters and the direction of portions of the text.

An additional characteristic of complex-text languages (with the exception of Vietnamese) is the fact that they do not have upper-case or lower-case characters.

Typical complex-text languages are those with a bidirectional script. Usually they are written from right to left, with some portions of text, such as numbers and embedded Latin-based text, written from left to right. Bidirectional languages include the languages of the Middle East and Africa (Arabic, Hebrew, Urdu, Farsi, Yiddish, and so on). Other complex-text languages include some languages of Asia that do not limit their encoding to a double-byte scheme (Thai, Lao, Vietnamese, Korean, and so on).

There is nothing in these languages themselves that is more complex than in the Latin-based languages; they are special only in that the presented text does not necessarily look identical to the text as stored.

Though the term complex is used to describe the text of the bidirectional and some other Asian languages, enabling a program to work in these languages is relatively simple, once the peculiarities of these languages are understood.

Layout Transformations and Related Attributes

To enter, process and present a text in a complex-text language, it is necessary to perform transformations between the processing layouts and the presentation layouts. The processing layout is the layout of text when stored or processed. The presentation layout is the layout of text when presented on a display or a printer.

These transformations have to take into account specific text attributes, including directionality, shaping, composition of characters and national numbers. Text attributes that describe bidirectional writing systems are defined in Bidirectional Languages .

An internationalised application must be designed to deal automatically with this kind of transformation and related attributes.

Bidirectional Languages

The bidirectional languages are used mainly in the Middle East. They include Arabic, Urdu, Farsi, Hebrew and Yiddish.1

In a bidirectional language, the general flow of text proceeds horizontally from right to left, but numbers are written from left to right, the same way as they are written in English. In addition, if an English or another left-to-right language text (addresses, acronyms or quotations) is embedded, it is also written from left to right.

The Arabic Languages and their Writing System

Arabic is a Semitic language that originated with the Arabs of the Hejaz and Nejd regions of Saudi Arabia. There are several spoken dialects of Arabic, but all are derived from the same root: the classical Arabic, which is taught at school in all Arab countries, and is used in all these countries for writing. The written form of the language has different levels of sophistication, depending on the use. These levels range from newspaper style to literary style, passing through technical, business and administration styles.

Figure: Arabic, Farsi and Urdu Characters
Arabic Alphabet Characteristics
The basic Arabic alphabet consists of:

The following characters are added to the basic set, in a way similar to the accented characters in Latin-based languages:

In addition, two special characters (Aleph Wasla and superscripted Aleph) are sometimes used in language education and linguistics.

The Arabic alphabet is also the root from which several other alphabets, such as Farsi and Urdu, are derived. In addition to the characters of the Arabic alphabet, the Farsi and Urdu alphabets have a few more specific characters. Farsi adds four consonants to the Arabic alphabet, and Urdu adds eight.

Character Set Considerations
The Arabic character set has the following characteristics:

Shapes of the Arabic Characters
Arabic code pages can contain from one to four shapes for each character or ligature, depending on the implementation. The four possible shapes of an Arabic character are:

Isolated
The character is not linked to either the preceding or the following character.

Final
The character is linked to the preceding character but not to the following one.

Initial
The character is linked to the following character but not to the preceding one.

Middle
The character is linked to both the preceding and following characters.

Figure: The Arabic Character Ghayn in its Different Shapes

In a text string, both the connection capabilities of a character and its neighbours, and its position in the word determine its actual presentation shape.

Numerals Used in Arabic

In countries using Arabic script, the decimal system is in use. In addition to the "Arabic" digits used in the Western world, national digit shapes, known as Hindi shapes, are in use. The equivalent of the Arabic digits 1, 2, 3, 4, 5, 6, 7, 8, 9 and 0 are:

Figure: Shape of the National Numbers in Arabic, Farsi and Urdu

The Hebrew Language and its Writing System

The Hebrew language dates back to biblical times. It remained relatively unchanged for about 2000 years until the end of the nineteenth century, when the birth of the modern Hebrew language took place. Since that time, the Academy of the Hebrew Language has extended traditional Hebrew to include Hebrew words for modern objects and concepts.

Hebrew is used mainly in Israel.

Hebrew Alphabet Characteristics
The Hebrew alphabet uses 27 characters to represent 22 consonants. This is because five consonants have different shapes when they appear at the end of a word.
Figure: Hebrew Alphabet

Vowels are represented in two ways:

Written Hebrew has no equivalent to capital letters. Hebrew does not have a cursive script: the letters are not connected. Unlike Arabic, Hebrew letters do not take on different shapes depending on the surrounding letters. The five final shape letters are considered additional, separate letters of the alphabet.

Figure: Hebrew Final Letters

Aspects of Bidirectional Language Writing Systems

This section discusses aspects of bidirectional texts, related to directionality, shaping and national numbers as well as keyboard input and compliance with common user access guidelines. The text attributes described here also pertain to some degree to other complex-text languages such as the languages of Asia (for example, Thai, Lao, Korean).

Bidirectionality

In the context of bidirectionality, the following are key concepts:

These attributes are described below.

Segments

A bidirectional text may consist of a main part that has one directionality (for example, an Arabic text written from right to left), and portions that have an opposite directionality (for example, an English address written from left to right.) The portion of text with a different directionality is called a segment. A bidirectional text thus might have a body bgcolor="#FFFFFF" of right-to-left text with embedded left-to right segments. Sometimes a segment with one directionality might itself have embedded or nested within it an additional segment with an opposite directionality. It is conceptually possible to have many levels of nesting; in most cases, however, there are no more than two levels.

One level of nesting is necessary for the entry of numbers within Arabic or Hebrew text. To simulate bidirectional scripts in the following examples, Hebrew and Arabic text is represented by lower-case English letters, while English text is represented by upper-case letters.

In Hebrew, it is customary to write the name of the street before the number of the house, as shown below:

b ecnartne 25 teerts elpam
<--------- -> <-----------

The street name is entered from right to left. The flow then has to be reversed to allow correct entry of the number from left to right (this being the nested left-to-right segment.) Then the flow must be reversed again to allow the entry of the entrance information from right to left.

Imagine somebody bgcolor="#FFFFFF" writing a letter in English to somebody who can read Hebrew too, and writing his or her address in Hebrew. In this case, the address in Hebrew is actually a nested segment of the English text.

            MY ADDRESS IS b ecnartne 25 teerts elpam THIS MONTH.
             ------------><--------- -> <----------- --------->
NEST LEVEL: 0000000000000011111111111221111111111111000000000000

Because the nested segment of the address has itself a nested segment (the street number), there are two levels of nesting.
Global Orientation

Bidirectional text may consist of mainly right-to-left text with some left-to-right nested segments (such as an Arabic text with some information in English), or mainly left-to-right with some right-to-left segments (such as an English letter with a Hebrew address nested within it). The predominant direction is called the global orientation; it cannot always be quickly deduced from the general context.

FRED DOES NOT BELIEVE taht yas syawla i

This sentence has one meaning when the reading is from left to right (Fred does not believe I always say that), and another meaning when read from right to left (I always say that Fred does not believe). In the first half of the above example, the global orientation of the text is left-to-right and in the second half it is right-to-left.

Because the global orientation is not always obvious from the context2 it must be known to the application developer whose product is processing the bidirectional data.

Note:
Not to be confused with the global orientation of the text is the physical orientation of the presentation device. A display terminal has, for example, a right-to-left physical orientation if the first character on the screen is the one in the upper right-hand corner and the general cursor movement is from right to left (and top to bottom.)
Text-types
In a bidirectional text a programmer must clearly distinguish between the physical order in which the text is presented, and the logical order in which its segments are processed (or pronounced if read aloud). Some segments may need to be reordered to a logical or physical order.

There are different approaches to how bidirectional text is to be reordered, and at present none can be said to be prevalent. The concept text-type is used to point to which approach is applicable for a specific text. The physical and logical order and the different text-types are discussed further below.

MY WIFE'S NAME IS ilin

The global orientation is left-to-right. The first letter in the text is M, followed by Y and so forth. In the physical order, after the letters I and S comes the letter i of the segment containing my wife's name in Hebrew. Note, however, that my wife's name is pronounced "nili". In the logical order the first letter of the name segment is thus the letter n, followed by i, l and i.

Sometimes, for example in on-line help, it is convenient to store the bidirectional text exactly as presented - that is, in the physical order. But if there is an intent to process the text (for example, to sort it), the segments must be stored in their logical order. There is no meaning, in the above example, to sort the name "ilin". It makes sense to reorder the text, so the directional segment containing the name "nili" is inverted, before being stored for further processing. The logical order is the preferred sequence for entering text and for processing. Conceptually, any storage device can be seen as storing the data from left to right. If a programmer wants to perform straightforward processing on the stored text (sorting, collating, indexing) without the need to preprocess each segment, the bidirectional data has to be stored in its logical sequence. This means reversing segments whose direction is opposite to the global orientation.

Text-types and Reordering Techniques

Different text-types require different approaches to reordering:

visual text-type

The oldest approach, dating from the time when there was no processing capability at the workstation, is simply to copy the entire screen to storage, and storage to screen (possibly inverting every row, depending on the physical orientation of the screen). It is up to each application programmer to know where the embedded segments are located and to process them accordingly. This text-type is called visual because it is a replication of the presented form. Many legacy applications3 and their files have this type of text.

implicit text-type

In the implicit text-type it is assumed that the letters of the Latin alphabet have a strong inherent left-to-right directionality, and those of the Arabic, Farsi, Urdu and Hebrew alphabet have a strong right-to-left inherent directionality. An algorithm of implicit text processing recognises segments based on their inherent directional characteristics, and segment inversion is performed automatically. The concept of an implicit algorithm is simple to understand. Its main limitation is that it cannot correctly handle some strings that have numbers and intermixed left-to-right and right-to-left letters.

explicit text-type

The explicit text-type assumes that there are additional control characters, embedded in the text, that instruct an explicit algorithm to perform segment inversions, shaping or numeral selections, and other transformations.

Thus, a text with visual text-type is stored in its physical order, and a text with an implicit text-type is stored in its logical order, which is better suited for automatic processing. A text with an explicit text-type is usually stored in logical order, but because of the embedded controls in the text, the automatic processing is not always straightforward.

There is no one type of text that can be used in all cases. The implicit techniques are usually heuristic and thus have some limitations as noted previously. The explicit techniques, while alleviating the limitations of implicit techniques, introduce other limitations such as the need for automatic processes to cope with embedded controls.

One specific technique, the Basic Display Algorithm,4 tries to be a bridge between the implicit and explicit techniques. In principle it is an implicit reordering algorithm, but it can deal with a few specific directional controls embedded in the text.

There are applications and related databases for all three text-types. It is possible for bidirectional text that is presented one way to be stored in a different layout. A programmer need only know what text-type or reordering algorithm was used, to correctly transform or process the bidirectional text.

Symmetrical Swapping

Some characters, such as the greater-than sign, have an implied directional meaning and have a complementary symmetric character with an opposite directional meaning (the less-than sign). When used within a segment that is presented right-to-left but is inverted (left-to-right) when stored for processing, such a character might have to be replaced by its symmetric sibling to ensure that the correct meaning of the text is preserved. The replacement of such a character by its complement during transformation of a bidirectional text is called symmetrical swapping.

Example: Example of Symmetrical Swapping
On a right-to-left window of the screen, the expression:
     b < a

is read as a is greater than b. In storage the orientation is always left-to-right; the first character in storage is thus a, followed by < and then b. So the result in storage is:
     a < b

which is of course incorrect. In this case, to preserve the correct meaning of the expression, the < character must be exchanged in storage with >.

Other graphic characters that require symmetrical swapping include the parentheses, square brackets, braces, and so on.

Although symmetrical swapping is a characteristic of bidirectional languages, it is not always mandatory for the software functions that transform different bidirectional-language text layouts. Sometimes this function is performed automatically by the workstation hardware or microcode.

Shaping

Shaping is the process by which characters are rendered in the appropriate presentation forms. This might involve the presentation of characters in a form different from the one in which they are stored. In general, to simplify processing, an unshaped (abstract or basic) representation is used internally. Shaping takes into account the character being shaped and the characters in its vicinity, and replaces its abstract representation (or that of its parts) with the proper shape. Shaping is a characteristic of many complex text languages, in particular the languages of the Middle East.

The Arabic scripts are cursive. A writing system is cursive if it is suited to handwriting rather than printing, with adjacent characters in a word connected to each other. Some letters can only connect to the letter on their right. This is the only way in which Arabic script is used, whether in books, newspapers, signs, or workstation displays. (English can be handwritten in a cursive style, for personal communications, but is seldom published or displayed that way. Thus English is not considered a cursive script.)

Shaping in Cursive Script Languages
In cursive scripts, letters might assume different shapes according to their position in the word and to the connectivity properties they and the adjacent letters have. There are as many as four shapes for each letter. As described in Shapes of the Arabic Characters characters may have initial, middle, final, and isolated forms (not all characters have all forms). Only one shape per letter is represented on Arabic keyboards, but all shapes must be available for presentation. Similarly, in most cases, a cursive language text is not stored with full shapes. Each character has a base form, which is an abstraction to allow selection of a cursive character without specifying its shape.

The proper shape can be selected by a shape determination routine, which allows for automatic (algorithmic) selection of the appropriate shape according to the context as directed by the software or the user. It may allow for user or software controlled selection of any of the four shapes mentioned above. Alternatively, it may allow transparent throughput of data: that is, it may become temporarily deactivated under software or user control. Whenever cursive-language characters are folded by processing to one shape, they must be reshaped using the same algorithm prior to presentation. In some very specific cases, data may be corrupted by this processing, as the algorithm may not be perfectly reversible. As an analogy, in English, converting 12Ab2 to upper case would result in 12AB2; the return to lower case would result in 12ab2, which is not the same as the original.

Though in most cases a cursive language text would be stored in basic shapes only, there are cases where it may be stored with characters shaped as presented, as in the case of messages or on-line help text.

Character Composition, Ligatures and Diacritics
In complex-text languages, it is possible that there is not a one-to-one correspondence between the number of characters of text stored for processing and the number of characters of the presented text. Sometimes two or more characters might be represented by a single glyph occupying one presentation cell:

ligatures

In the cursive languages, ligatures use one glyph to represent two or more specific letters. For example, the ligature Lamalif is used to represent the frequently used pair of letters Lam and Alif.

diacritics

These are marks above, near, within or below a consonant. They are used in bidirectional languages, among other functions, to represent vowels. When kept in storage for processing, these marks occupy physical positions, but if used for representation, they might occupy the same cell as the associated consonants.

As a compromise, given existing limitations (in the graphical capabilities and resolution of the display devices and the number of code points available), bidirectional languages such as Hebrew have in many implementations given up the ability to represent vowels by diacritics. The vowels sounds have to be surmised by readers based on their knowledge of the language and according to the semantics of the text.

However this guesswork is not acceptable for specific applications, such as poetry or processing of a classical text, which requires the use of diacritics. In some complex-text languages, such as Thai, the use of vowel symbols and tone marks is mandatory.

In Arabic, spacing diacritics are currently used as a compromise. In the present Arabic systems, some or all of the Arabic diacritics are implemented as separate characters to be rendered following the character to which the diacritics belong.

National Numbers

In both Latin-based languages and Hebrew, numbers are represented using the so-called Arabic digits (1, 2, 3, 4, 5, 6, 7, 8, 9 and 0). However, the cursive languages, (Arabic, Farsi, and Urdu), as well as many other complex-text languages, have their own national glyphs for digits.5 The local name for numbers used in the cursive languages is not ``Arabic numbers", but Hindi or sometimes Arabic-Indic numbers. The direction of the numbers is always left-to-right. Mathematical formulae in Arabic are written from right to left and in Farsi they are written from left to right.

It is important to understand that in most cases, the text stored for processing has numbers encoded in their Arabic (western) code. When it comes to presentation, these numbers might be presented using either national glyphs for digits or ordinary Arabic digits, according to the intent of the user or application developer.

Bidirectional Data Entry

To those unfamiliar with bidirectional languages, understanding how segments of text with different directionality can actually be entered from a keyboard is somewhat of a puzzle.
Keying Order
The order in which bidirectional data is typed into a workstation is the order in which the text is meant to be read - the logical order.
Bidirectional Keyboards
The keyboards used for bidirectional languages are similar to those used for English, but on the same keytops on which Latin characters and symbols are engraved, character symbols specific to the other language are added. In the case of the cursive languages, such as Arabic, the character symbols engraved are the basic characters only. Special key combinations are used to switch between the English keyboard layer and the national-language keyboard layer. For example, in some cases, the Hebrew layer is made active, on a Hebrew keyboard, by simultaneously pressing the <Alt> and <Right Shift> keys. Such key combinations are also used to enter appropriate input modes. For example, in some environments, push mode is entered by simultaneously pressing <Shift> and <Num Lock>. Push mode is a keyboard input mode in which characters are pushed in the direction opposite to the base direction of the segment and the cursor does not move, in the same way the digits behave on the screen of a pocket calculator.
Bidirectional Typing Interfaces
To allow for bidirectional text entry from a keyboard, the interfaces must be able to intercept and process each keystroke. These interfaces can be part of the terminal and associated controller's hardware or microcode, or they can be a specific routine that is added to the operating system.

There are two typing interfaces to consider:

Manual Typing Method
In the manual typing method the user informs the system in which direction the characters are to be typed. For mixed-direction typing, the user makes extensive use of the Push and End Push keyboard functions.

The manual method also supports an Automatic Push (Auto Push) mode. When the Auto Push setting is active, the Push Mode is started and terminated automatically, according to the actual characters being typed.

When the manual typing method is active, the keyboard language group and cursor direction are handled separately by the system. This means that the user has separate control for:

Automatic (Logical) Typing Method
This convention provides some automatic handling of directionality. When this method is active, the system determines the directionality of each part of the text (each segment) based on the actual characters being typed, using a set of predefined rules. The method is called logical because the direction of the text is logically deduced based on the language of the characters.

With this method, the system automatically determines how to display characters in the correct order when the user switches keyboard language groups.

Another feature of this method is that it handles text in typing order; that is, the system remembers the order in which the characters were initially typed. It then uses this knowledge along with a set of predefined rules, to determine how the text is displayed, processed and deleted by the application.

If the cursor is in the Home position (the first logical position in the field or window) and a character of a language other than the default language of the current orientation is entered, the screen or window orientation is reversed automatically. That is, if the character entered is Hebrew, the window orientation is right-to-left; if the character is English, the window orientation is left-to-right.

Common User Access and Bidirectional Languages

The basic rule for applications that are to conform to Common User Access guidelines is that ``... All pieces of data must be displayed in the orientation that is correct for the application user. Data input must be supported in the orientation that is natural for users".

Thai Language and its Writing System

The Thai language is representative of a class of complex-text languages whose characters are composed of a number of symbols or elements. Thai belongs to the Sino-Tibetan family of languages. Like the Chinese languages, which also belong to this family, Thai is a monosyllabic tone language. While it resembles Chinese structurally and though much of its basic vocabulary is of Chinese origin, it has also been greatly influenced by both Pali and Sanskrit.

The Thai writing system was developed from the Devanagari system, which originated in India and came to Thailand from Cambodia. A major difference between the Chinese and Thai writing systems is that while Chinese makes use of a large number of pictorial symbols, Thai uses an alphabet of consonants, vowels, tone marks, diacritics and special symbols. With some exceptions, a Thai word can be pronounced correctly on sight, in a similar manner to Italian or French.

Writing Thai Characters - Graphic Representation

Thai is written from left to right, without spaces between words. Each word is represented by one or more syllables; each syllable consists of a consonant, a vowel, a tone and a final consonant or a final diacritic. Spaces in the text indicate the ends of phrases or sentences, and are thus used as a form of punctuation. Thus, individual words are recognised only by scanning the text for syllable boundaries. Compared to western writing systems, the composed characters tend to be taller and thinner.
Figure: Thai Text Example

A line of Thai text can be considered to be logically divided into four parallel lines:

Thai Written Symbols

Generally speaking, the more than 2,000 characters in the Thai writing system can be categorised into 20 types of written symbols, with 88 basic symbols:

Normally, Thai data is encoded using a single-byte code page, where each symbol has an adequate code point. The symbols are used to enter Thai data on a Thai keyboard. Thus the Thai data is stored, for processing purposes, as symbol elements. These elements have to be combined into characters for rendering purposes.

Writing Order

In the most common writing order, first a base line symbol is written, and then optionally, an upper vowel or lower vowel symbol is written above or below it. A tone mark symbol may then optionally be written either above the base line symbol, or above the upper vowel symbol, if present.

This order of writing is taught in Thai elementary schools. However, writing-order inconsistencies exist between individuals. The valid combinations of symbols for Thai composed characters are:

Any other combinations would be considered invalid.

What is a Thai Character?

From a linguistic or phonetic point of view, the Thai writing system is actually more complex than that described above. Consonants are written on the base line. A middle vowel can be written either before, after or straddling the related consonant. Upper-vowels are written above, and lower vowels below, their related consonant. Vowels are always pronounced and collated after the consonant. The tone mark is usually written after the upper vowel or lower vowel, but some people might write it after the consonant. The left and right pieces of a middle vowel, which straddle a consonant, are included as separate components in some encoding schemes.

To prevent confusion, the term composed character is used here for the representation of one syllable at a writing position, and the term symbol is used for the components of a composed character.

Thai Numbers

Although Western numerals (Arabic numbers) are now widely used in Thai writing, there are also ten Thai glyphs for numbers. In Thai, the equivalent of the Arabic digits 1, 2, 3, 4, 5, 6, 7, 8, 9 and 0 are respectively:

Figure: Shape of the National Numbers in Thai

Character Composition

According to the rules for writing Thai, only certain combinations of symbols are possible. When someone fluent in Thai is writing or reading a line, a process of composition is taking place. In about 74 percent of cases a character is formed from a single symbol; in about 22 percent of cases, it is formed from two symbols; and in 4 percent of cases it is formed from three symbols.

A Thai speaker does not think of a composed character as, for example, an accented character in French. This difference in thinking is reflected in the difference between European and Thai keyboards. In European keyboards, dead keys are used to place accents on characters. The dead key is pressed first to show the accent, and then the character key is pressed. The cursor moves only after the character has been entered. All character manipulation is done at the cursor position.

In Thai the consonant or middle vowel is entered first. It is displayed, and the cursor then moves one position to the right. The upper and lower (dead key) vowels and tone marks are then added to the character to the left of the cursor. The rightmost column of positions on the screen is used to display the cursor only, and data is not allowed in this column. Usually vowels and tone marks are stacked on the consonants to compose syllables. The exception is middle vowels, which stay independently at the same level as the consonants.

Thai Character Rendering

Quality font rendering (for example, for desktop publishing), requires additional changes to be made to a Thai composite character form, and sometimes to other characters in its vicinity.

Examples

It is thus possible to recognise a similarity between character composition in Thai, and ligatures composition and shaping in bidirectional languages. The character presented is not identical with the symbols stored, so a shaping or composing algorithm must be applied.

Similarly, there are cases where the shaping transformation must not to be performed at rendering, but at a previous stage. When using the high-quality printers adapted for double-byte character set (DBCS), a shaping of characters (maximum three-symbol), is performed as part of the transformation of text to a double-byte encoding scheme. In this case, the text can be considered stored in a shaped form for higher-efficiency printing. This resembles the case in which Arabic message text is kept in storage in a shaped layout.

Korean Language and its Writing System

Korean is the official language of Korea (both south and north) and is spoken by more than 60 million people. The Korean language contains not only Korean words, but also borrows a small number of Chinese-language words. A small number of words are also borrowed from other foreign languages, such as Japanese, English, German, French, and so on.

The official writing system of Korean is known as Hangul, which means "the Korean letter" in Korean. It was created and announced by King Sejong and his scholars in 1446. The first standardisation of Hangul was published by the Korean Language Association in 1933.

Hanja, which means "Chinese letter", is a term sometimes used to describe the ideograms used to express words borrowed from Chinese. Likewise, the English alphabet is used to express some English terminology. Nevertheless, contemporary Koreans regard Hangul as the writing system for the Korean language, and as such the Korean writing system is not ideographic.

Korean script uses Arabic figures to represent numbers in most cases. The pronunciation of figures can be expressed using either Hangul or Hanja.

Hangul Writing System

In Hangul, a one-syllable letter may be composed of a cluster of two or three elements, the first of which is always an initial consonant (this can be a null consonant, which is not pronounced). The initial consonant is placed at the left, at the upper side or at the upper-left side of the syllable cluster, depending on the second element of the cluster. This second element is always a vowel. The vowels whose representation is based on a vertical line, are positioned to the right of the initial consonant. The vowels whose representation is based on a horizontal line are placed under it. In the case of double vowels with both horizontal and vertical line representation, the initial consonant is placed at the upper-left side and the double vowel is usually placed at the right side and under the consonant. If present, the third element of the cluster is a final consonant, and is always placed under the other two elements of the cluster.

These cluster elements are called Jamo in Korean, where Ja means consonant and mo means vowel.

In standard Hangul, there are 24 basic Jamo elements, of which 14 are used for consonants. In the case of sounds that are not representable by the basic Jamo elements, Hangul grammar allows them to be represented by combining two or more elements. About 27 additional Jamo elements, of which 16 are used for consonants, belong to such combined Jamo elements.

There are 19 permissible initial consonants, 21 vowels and 28 final consonants. The total number of possible combinations of Hangul Consonant-Vowel-Consonant (CVC) is thus 11,172 (19x21x28). Everyday Korean, however, makes do with approximately 2,500 combinations.

Writing Hangul Syllables - Graphic Representation

The major graphic distinction in Hangul is between vowels and consonants. Vowels are based on long horizontal or vertical lines that have distinguishing marks. The basic vowel Jamo elements are as follows:

Consonants are represented by more compact, two-dimensional signs; the basic consonant Jamo elements are as follows:

The shapes of the consonants were apparently chosen by King Sejong to represent highly stylised pictures of the tongue and mouth when the equivalent sounds are pronounced. For example, the Hangul sign:

which represents a sound equivalent to the English letter n has a shape that suggests the tongue-tip raised to touch the front of the palate.
Word Grouping and Direction

In Korean, words are made up of syllables, and the words are separated by spaces. There are two ways in which syllables are juxtaposed to create words and text: vertically (top to bottom and right to left) and horizontally (left to right and top to bottom.)

In South Korea, although newspapers are printed from top to bottom in vertical columns that shift from right to left, horizontal writing is very much in use. The script is Hangul, with occasional Chinese (Hanja) characters as well as Latin-based characters for English text.

In North Korea, newspapers are printed horizontally with Korean script (Hangul only, without Hanja characters).

If, hypothetically, English were written like Hangul, and the vowel O belonged to the set of vowels written under the initial consonant while the vowels A and E were written to the right of the initial consonant, then an English term such as common market would be presented in the following manner:

Horizontal direction:

C  M     M A  K E
O  O      R    T
M  N

Vertical direction of writing:

                               C
                               O
                               M

                               M
                               O
                               N


                               M A
                                R

                               K E
                                T

Figure: Korean Writing Direction

The following figure shows actual Hangul Jamo elements and their corresponding composition. The right-most column is an example of vertical writing; the bottom line is an example of horizontal writing.

Figure: Syllable Composition and Writing Direction in Hangul

Character Set Considerations

Korean uses two character sets:

Conclusions and Guidelines

Though so different in their appearances, all complex-text languages - the bidirectional ones such as Arabic, Farsi, Urdu, Hebrew and Yiddish, or the languages such as Thai, Lao, or Korean - have a distinct common characteristic: the form of the rendered text is different from that of the stored text. The transformation functions needed to perform the changes between rendered and stored text depend on descriptive information pertaining to the attributes of complex-text languages: global orientation, text-type, symmetrical swapping, shaping and national numbers.

Application developers should be aware of the fact that in the complex-text languages there is a need for transformations between the different text layouts. They should allow for user or system exits to facilitate invoking these transformations, in those places where a transformation might be expected (at input, before output, before a collating process, and so on). Programs must be able to identify the location and content of the complex-text attributes, and be able to change their content if needed.

Just as for any other language, an application meant to be used for complex-text languages should utilise the appropriate language code page and cultural data (date and time layout, collating sequence, monetary layout, and so on).

Application developers should design their products in such a way that they use, as much as possible, the standard functions and controls provided by the operating system services or toolkits for these languages. They might choose to use the APIs offered in the national language versions of the operating system services or toolkits to perform such transformations (when available).

It would be good practice to concentrate all the functions related to National Language in a specific program area for easy maintainability and change support.


Footnotes

1.
Arabic is spoken mainly in Algeria, Bahrain, Egypt, Iraq, Israel, Jordan, Kuwait, Lebanon, Libya, Morocco, Oman, Qatar, Saudi Arabia, Sudan, Syria, Tunisia, United Arab Emirates, and Yemen. Urdu is spoken mainly in Pakistan. Farsi is spoken mainly in Iran. Hebrew is spoken mainly in Israel, and Yiddish is spoken mainly in Israel, Europe, and North America.

2.
Sometimes it is possible to have a contextual global orientation, where the global orientation is set according to the directional characteristic of the first character in the data stream that has a distinct directionality.

3.
Legacy applications are those which have been inherited from a prior era. They may be obsolete, but must be supported.

4.
The Basic Display Algorithm was initially published in the Directionality Appendix A of the Unicode standard.

5.
Other complex-text languages that have their own national glyphs for decimal digits are Devanagari, Gurmukhi, Gujarati, Oriya, Bengali, Tamil, Telugu, Malayalam, Sinhalese, Khmer (Cambodian), Lao, Mongolian, Chinese, Tibetan and Thai.


Why not acquire a nicely bound hard copy?
Click here to return to the publication details or order a copy of this publication.

Contents Next section Index