Scripts and Languages

Scripts and Languages — Identifying writing systems and languages

Synopsis




enum        PangoScript;
#define     PANGO_TYPE_SCRIPT
            PangoScriptIter;
PangoScript pango_script_for_unichar        (gunichar ch);
PangoLanguage* pango_script_get_sample_language
                                            (PangoScript script);
PangoScriptIter* pango_script_iter_new      (const char *text,
                                             int length);
void        pango_script_iter_get_range     (PangoScriptIter *iter,
                                             G_CONST_RETURN char **start,
                                             G_CONST_RETURN char **end,
                                             PangoScript *script);
gboolean    pango_script_iter_next          (PangoScriptIter *iter);
void        pango_script_iter_free          (PangoScriptIter *iter);
            PangoLanguage;
#define     PANGO_TYPE_LANGUAGE
PangoLanguage* pango_language_from_string   (const char *language);
const char* pango_language_to_string        (PangoLanguage *language);
gboolean    pango_language_matches          (PangoLanguage *language,
                                             const char *range_list);
gboolean    pango_language_includes_script  (PangoLanguage *language,
                                             PangoScript script);
const PangoScript* pango_language_get_scripts
                                            (PangoLanguage *language,
                                             int *num_scripts);
PangoLanguage* pango_language_get_default   (void);
const char* pango_language_get_sample_string
                                            (PangoLanguage *language);

Description

The functions in this section are used to identify the writing system, or script of individual characters and of ranges within a larger text string.

Details

enum PangoScript

typedef enum {                         /* ISO 15924 code */
      PANGO_SCRIPT_INVALID_CODE = -1,
      PANGO_SCRIPT_COMMON       = 0,   /* Zyyy */
      PANGO_SCRIPT_INHERITED,          /* Qaai */
      PANGO_SCRIPT_ARABIC,             /* Arab */
      PANGO_SCRIPT_ARMENIAN,           /* Armn */
      PANGO_SCRIPT_BENGALI,            /* Beng */
      PANGO_SCRIPT_BOPOMOFO,           /* Bopo */
      PANGO_SCRIPT_CHEROKEE,           /* Cher */
      PANGO_SCRIPT_COPTIC,             /* Qaac */
      PANGO_SCRIPT_CYRILLIC,           /* Cyrl (Cyrs) */
      PANGO_SCRIPT_DESERET,            /* Dsrt */
      PANGO_SCRIPT_DEVANAGARI,         /* Deva */
      PANGO_SCRIPT_ETHIOPIC,           /* Ethi */
      PANGO_SCRIPT_GEORGIAN,           /* Geor (Geon, Geoa) */
      PANGO_SCRIPT_GOTHIC,             /* Goth */
      PANGO_SCRIPT_GREEK,              /* Grek */
      PANGO_SCRIPT_GUJARATI,           /* Gujr */
      PANGO_SCRIPT_GURMUKHI,           /* Guru */
      PANGO_SCRIPT_HAN,                /* Hani */
      PANGO_SCRIPT_HANGUL,             /* Hang */
      PANGO_SCRIPT_HEBREW,             /* Hebr */
      PANGO_SCRIPT_HIRAGANA,           /* Hira */
      PANGO_SCRIPT_KANNADA,            /* Knda */
      PANGO_SCRIPT_KATAKANA,           /* Kana */
      PANGO_SCRIPT_KHMER,              /* Khmr */
      PANGO_SCRIPT_LAO,                /* Laoo */
      PANGO_SCRIPT_LATIN,              /* Latn (Latf, Latg) */
      PANGO_SCRIPT_MALAYALAM,          /* Mlym */
      PANGO_SCRIPT_MONGOLIAN,          /* Mong */
      PANGO_SCRIPT_MYANMAR,            /* Mymr */
      PANGO_SCRIPT_OGHAM,              /* Ogam */
      PANGO_SCRIPT_OLD_ITALIC,         /* Ital */
      PANGO_SCRIPT_ORIYA,              /* Orya */
      PANGO_SCRIPT_RUNIC,              /* Runr */
      PANGO_SCRIPT_SINHALA,            /* Sinh */
      PANGO_SCRIPT_SYRIAC,             /* Syrc (Syrj, Syrn, Syre) */
      PANGO_SCRIPT_TAMIL,              /* Taml */
      PANGO_SCRIPT_TELUGU,             /* Telu */
      PANGO_SCRIPT_THAANA,             /* Thaa */
      PANGO_SCRIPT_THAI,               /* Thai */
      PANGO_SCRIPT_TIBETAN,            /* Tibt */
      PANGO_SCRIPT_CANADIAN_ABORIGINAL, /* Cans */
      PANGO_SCRIPT_YI,                 /* Yiii */
      PANGO_SCRIPT_TAGALOG,            /* Tglg */
      PANGO_SCRIPT_HANUNOO,            /* Hano */
      PANGO_SCRIPT_BUHID,              /* Buhd */
      PANGO_SCRIPT_TAGBANWA,           /* Tagb */

      /* Unicode-4.0 additions */
      PANGO_SCRIPT_BRAILLE,            /* Brai */
      PANGO_SCRIPT_CYPRIOT,            /* Cprt */
      PANGO_SCRIPT_LIMBU,              /* Limb */
      PANGO_SCRIPT_OSMANYA,            /* Osma */
      PANGO_SCRIPT_SHAVIAN,            /* Shaw */
      PANGO_SCRIPT_LINEAR_B,           /* Linb */
      PANGO_SCRIPT_TAI_LE,             /* Tale */
      PANGO_SCRIPT_UGARITIC,           /* Ugar */

      /* Unicode-4.1 additions */
      PANGO_SCRIPT_NEW_TAI_LUE,        /* Talu */
      PANGO_SCRIPT_BUGINESE,           /* Bugi */
      PANGO_SCRIPT_GLAGOLITIC,         /* Glag */
      PANGO_SCRIPT_TIFINAGH,           /* Tfng */
      PANGO_SCRIPT_SYLOTI_NAGRI,       /* Sylo */
      PANGO_SCRIPT_OLD_PERSIAN,        /* Xpeo */
      PANGO_SCRIPT_KHAROSHTHI,         /* Khar */

      /* Unicode-5.0 additions */
      PANGO_SCRIPT_UNKNOWN,            /* Zzzz */
      PANGO_SCRIPT_BALINESE,           /* Bali */
      PANGO_SCRIPT_CUNEIFORM,          /* Xsux */
      PANGO_SCRIPT_PHOENICIAN,         /* Phnx */
      PANGO_SCRIPT_PHAGS_PA,           /* Phag */
      PANGO_SCRIPT_NKO,                /* Nkoo */

      /* Unicode-5.1 additions */
      PANGO_SCRIPT_KAYAH_LI,           /* Kali */
      PANGO_SCRIPT_LEPCHA,             /* Lepc */
      PANGO_SCRIPT_REJANG,             /* Rjng */
      PANGO_SCRIPT_SUNDANESE,          /* Sund */
      PANGO_SCRIPT_SAURASHTRA,         /* Saur */
      PANGO_SCRIPT_CHAM,               /* Cham */
      PANGO_SCRIPT_OL_CHIKI,           /* Olck */
      PANGO_SCRIPT_VAI,                /* Vaii */
      PANGO_SCRIPT_CARIAN,             /* Cari */
      PANGO_SCRIPT_LYCIAN,             /* Lyci */
      PANGO_SCRIPT_LYDIAN              /* Lydi */
} PangoScript;

The PangoScript enumeration identifies different writing systems. The values correspond to the names as defined in the Unicode standard. Note that new types may be added in the future. Applications should be ready to handle unknown values. This enumeration is interchangeable with GUnicodeScript. See Unicode Standard Annex 24: Script names.

PANGO_SCRIPT_INVALID_CODE a value never returned from pango_script_for_unichar()
PANGO_SCRIPT_COMMON a character used by multiple different scripts
PANGO_SCRIPT_INHERITED a mark glyph that takes its script from the base glyph to which it is attached
PANGO_SCRIPT_ARABIC Arabic
PANGO_SCRIPT_ARMENIAN Armenian
PANGO_SCRIPT_BENGALI Bengali
PANGO_SCRIPT_BOPOMOFO Bopomofo
PANGO_SCRIPT_CHEROKEE Cherokee
PANGO_SCRIPT_COPTIC Coptic
PANGO_SCRIPT_CYRILLIC Cyrillic
PANGO_SCRIPT_DESERET Deseret
PANGO_SCRIPT_DEVANAGARI Devanagari
PANGO_SCRIPT_ETHIOPIC Ethiopic
PANGO_SCRIPT_GEORGIAN Georgian
PANGO_SCRIPT_GOTHIC Gothic
PANGO_SCRIPT_GREEK Greek
PANGO_SCRIPT_GUJARATI Gujarati
PANGO_SCRIPT_GURMUKHI Gurmukhi
PANGO_SCRIPT_HAN Han
PANGO_SCRIPT_HANGUL Hangul
PANGO_SCRIPT_HEBREW Hebrew
PANGO_SCRIPT_HIRAGANA Hiragana
PANGO_SCRIPT_KANNADA Kannada
PANGO_SCRIPT_KATAKANA Katakana
PANGO_SCRIPT_KHMER Khmer
PANGO_SCRIPT_LAO Lao
PANGO_SCRIPT_LATIN Latin
PANGO_SCRIPT_MALAYALAM Malayalam
PANGO_SCRIPT_MONGOLIAN Mongolian
PANGO_SCRIPT_MYANMAR Myanmar
PANGO_SCRIPT_OGHAM Ogham
PANGO_SCRIPT_OLD_ITALIC Old Italic
PANGO_SCRIPT_ORIYA Oriya
PANGO_SCRIPT_RUNIC Runic
PANGO_SCRIPT_SINHALA Sinhala
PANGO_SCRIPT_SYRIAC Syriac
PANGO_SCRIPT_TAMIL Tamil
PANGO_SCRIPT_TELUGU Telugu
PANGO_SCRIPT_THAANA Thaana
PANGO_SCRIPT_THAI Thai
PANGO_SCRIPT_TIBETAN Tibetan
PANGO_SCRIPT_CANADIAN_ABORIGINAL Canadian Aboriginal
PANGO_SCRIPT_YI Yi
PANGO_SCRIPT_TAGALOG Tagalog
PANGO_SCRIPT_HANUNOO Hanunoo
PANGO_SCRIPT_BUHID Buhid
PANGO_SCRIPT_TAGBANWA Tagbanwa
PANGO_SCRIPT_BRAILLE Braille
PANGO_SCRIPT_CYPRIOT Cypriot
PANGO_SCRIPT_LIMBU Limbu
PANGO_SCRIPT_OSMANYA Osmanya
PANGO_SCRIPT_SHAVIAN Shavian
PANGO_SCRIPT_LINEAR_B Linear B
PANGO_SCRIPT_TAI_LE Tai Le
PANGO_SCRIPT_UGARITIC Ugaritic
PANGO_SCRIPT_NEW_TAI_LUE New Tai Lue. Since 1.10
PANGO_SCRIPT_BUGINESE Buginese. Since 1.10
PANGO_SCRIPT_GLAGOLITIC Glagolitic. Since 1.10
PANGO_SCRIPT_TIFINAGH Tifinagh. Since 1.10
PANGO_SCRIPT_SYLOTI_NAGRI Syloti Nagri. Since 1.10
PANGO_SCRIPT_OLD_PERSIAN Old Persian. Since 1.10
PANGO_SCRIPT_KHAROSHTHI Kharoshthi. Since 1.10
PANGO_SCRIPT_UNKNOWN an unassigned code point. Since 1.14
PANGO_SCRIPT_BALINESE Balinese. Since 1.14
PANGO_SCRIPT_CUNEIFORM Cuneiform. Since 1.14
PANGO_SCRIPT_PHOENICIAN Phoenician. Since 1.14
PANGO_SCRIPT_PHAGS_PA Phags-pa. Since 1.14
PANGO_SCRIPT_NKO N'Ko. Since 1.14
PANGO_SCRIPT_KAYAH_LI Kayah Li. Since 1.20.1
PANGO_SCRIPT_LEPCHA Lepcha. Since 1.20.1
PANGO_SCRIPT_REJANG Rejang. Since 1.20.1
PANGO_SCRIPT_SUNDANESE Sundanese. Since 1.20.1
PANGO_SCRIPT_SAURASHTRA Saurashtra. Since 1.20.1
PANGO_SCRIPT_CHAM Cham. Since 1.20.1
PANGO_SCRIPT_OL_CHIKI Ol Chiki. Since 1.20.1
PANGO_SCRIPT_VAI Vai. Since 1.20.1
PANGO_SCRIPT_CARIAN Carian. Since 1.20.1
PANGO_SCRIPT_LYCIAN Lycian. Since 1.20.1
PANGO_SCRIPT_LYDIAN Lydian. Since 1.20.1

PANGO_TYPE_SCRIPT

#define PANGO_TYPE_SCRIPT (pango_script_get_type())

The GObject type for PangoScript


PangoScriptIter

typedef struct _PangoScriptIter PangoScriptIter;

A PangoScriptIter is used to iterate through a string and identify ranges in different scripts.


pango_script_for_unichar ()

PangoScript pango_script_for_unichar        (gunichar ch);

Looks up the PangoScript for a particular character (as defined by Unicode Standard Annex 24). No check is made for ch being a valid Unicode character; if you pass in invalid character, the result is undefined.

As of Pango 1.18, this function simply returns the return value of g_unichar_get_script().

ch : a Unicode character
Returns : the PangoScript for the character.

Since 1.4


pango_script_get_sample_language ()

PangoLanguage* pango_script_get_sample_language
                                            (PangoScript script);

Given a script, finds a language tag that is reasonably representative of that script. This will usually be the most widely spoken or used language written in that script: for instance, the sample language for PANGO_SCRIPT_CYRILLIC is ru (Russian), the sample language for PANGO_SCRIPT_ARABIC is ar.

For some scripts, no sample language will be returned because there is no language that is sufficiently representative. The best example of this is PANGO_SCRIPT_HAN, where various different variants of written Chinese, Japanese, and Korean all use significantly different sets of Han characters and forms of shared characters. No sample language can be provided for many historical scripts as well.

As of 1.18, this function checks the environment variables PANGO_LANGUAGE and LANGUAGE (checked in that order) first. If one of them is set, it is parsed as a list of language tags separated by colons or other separators. This function will return the first language in the parsed list that Pango believes may use script for writing. This last predicate is tested using pango_language_includes_script(). This can be used to control Pango's font selection for non-primary languages. For example, a PANGO_LANGUAGE enviroment variable set to "en:fa" makes Pango choose fonts suitable for Persian (fa) instead of Arabic (ar) when a segment of Arabic text is found in an otherwise non-Arabic text. The same trick can be used to choose a default language for PANGO_SCRIPT_HAN when setting context language is not feasible.

script : a PangoScript
Returns : a PangoLanguage that is representative of the script, or NULL if no such language exists.

Since 1.4


pango_script_iter_new ()

PangoScriptIter* pango_script_iter_new      (const char *text,
                                             int length);

Create a new PangoScriptIter, used to break a string of Unicode into runs by text. No copy is made of text, so the caller needs to make sure it remains valid until the iterator is freed with pango_script_iter_free().x

text : a UTF-8 string
length : length of text, or -1 if text is nul-terminated.
Returns : the new script iterator, initialized to point at the first range in the text, which should be freed with pango_script_iter_free(). If the string is empty, it will point at an empty range.

Since 1.4


pango_script_iter_get_range ()

void        pango_script_iter_get_range     (PangoScriptIter *iter,
                                             G_CONST_RETURN char **start,
                                             G_CONST_RETURN char **end,
                                             PangoScript *script);

Gets information about the range to which iter currently points. The range is the set of locations p where *start <= p < *end. (That is, it doesn't include the character stored at *end)

iter : a PangoScriptIter
start : location to store start position of the range, or NULL
end : location to store end position of the range, or NULL
script : location to store script for range, or NULL

Since 1.4


pango_script_iter_next ()

gboolean    pango_script_iter_next          (PangoScriptIter *iter);

Advances a PangoScriptIter to the next range. If iter is already at the end, it is left unchanged and FALSE is returned.

iter : a PangoScriptIter
Returns : TRUE if iter was successfully advanced.

Since 1.4


pango_script_iter_free ()

void        pango_script_iter_free          (PangoScriptIter *iter);

Frees a PangoScriptIter created with pango_script_iter_new().

iter : a PangoScriptIter

Since 1.4


PangoLanguage

typedef struct _PangoLanguage PangoLanguage;

The PangoLanguage structure is used to represent a language.

PangoLanguage pointers can be efficiently copied and compared with each other.


PANGO_TYPE_LANGUAGE

#define PANGO_TYPE_LANGUAGE (pango_language_get_type ())

The GObject type for PangoLanguage.


pango_language_from_string ()

PangoLanguage* pango_language_from_string   (const char *language);

Take a RFC-3066 format language tag as a string and convert it to a PangoLanguage pointer that can be efficiently copied (copy the pointer) and compared with other language tags (compare the pointer.)

This function first canonicalizes the string by converting it to lowercase, mapping '_' to '-', and stripping all characters other than letters and '-'.

Use pango_language_get_default() if you want to get the PangoLanguage for the current locale of the process.

language : a string representing a language tag, or NULL
Returns : an opaque pointer to a PangoLanguage structure, or NULL if language was NULL. The returned pointer will be valid forever after, and should not be freed.

pango_language_to_string ()

const char* pango_language_to_string        (PangoLanguage *language);

Gets the RFC-3066 format string representing the given language tag.

language : a language tag.
Returns : a string representing the language tag. This is owned by Pango and should not be freed.

pango_language_matches ()

gboolean    pango_language_matches          (PangoLanguage *language,
                                             const char *range_list);

Checks if a language tag matches one of the elements in a list of language ranges. A language tag is considered to match a range in the list if the range is '*', the range is exactly the tag, or the range is a prefix of the tag, and the character after it in the tag is '-'.

language : a language tag (see pango_language_from_string()), NULL is allowed and matches nothing but '*'
range_list : a list of language ranges, separated by ';', ':', ',', or space characters. Each element must either be '*', or a RFC 3066 language range canonicalized as by pango_language_from_string()
Returns : TRUE if a match was found.

pango_language_includes_script ()

gboolean    pango_language_includes_script  (PangoLanguage *language,
                                             PangoScript script);

Determines if script is one of the scripts used to write language. The returned value is conservative; if nothing is known about the language tag language, TRUE will be returned, since, as far as Pango knows, script might be used to write language.

This routine is used in Pango's itemization process when determining if a supplied language tag is relevant to a particular section of text. It probably is not useful for applications in most circumstances.

This function uses pango_language_get_scripts() internally.

language : a PangoLanguage, or NULL
script : a PangoScript
Returns : TRUE if script is one of the scripts used to write language or if nothing is known about language (including the case that language is NULL), FALSE otherwise.

Since 1.4


pango_language_get_scripts ()

const PangoScript* pango_language_get_scripts
                                            (PangoLanguage *language,
                                             int *num_scripts);

Determines the scripts used to to write language. If nothing is known about the language tag language, or if language is NULL, then NULL is returned. The list of scripts returned starts with the script that the language uses most and continues to the one it uses least.

The value num_script points at will be set to the number of scripts in the returned array (or zero if NULL is returned).

Most languages use only one script for writing, but there are some that use two (Latin and Cyrillic for example), and a few use three (Japanese for example). Applications should not make any assumptions on the maximum number of scripts returned though, except that it is positive if the return value is not NULL, and it is a small number.

The pango_language_includes_script() function uses this function internally.

language : a PangoLanguage, or NULL
num_scripts : location to return number of scripts, or NULL
Returns : An array of PangoScript values, with the number of entries in the array stored in num_scripts, or NULL if Pango does not have any information about this particular language tag (also the case if language is NULL). The returned array is owned by Pango and should not be modified or freed.

Since 1.22


pango_language_get_default ()

PangoLanguage* pango_language_get_default   (void);

Returns the PangoLanguage for the current locale of the process. Note that this can change over the life of an application.

On Unix systems, this is the return value is derived from setlocale(LC_CTYPE, NULL), and the user can affect this through the environment variables LC_ALL, LC_CTYPE or LANG (checked in that order). The locale string typically is in the form lang_COUNTRY, where lang is an ISO-639 language code, and COUNTRY is an ISO-3166 country code. For instance, sv_FI for Swedish as written in Finland or pt_BR for Portuguese as written in Brazil.

On Windows, the C library does not use any such environment variables, and setting them won't affect the behavior of functions like ctime(). The user sets the locale through the Regional Options in the Control Panel. The C library (in the setlocale() function) does not use country and language codes, but country and language names spelled out in English. However, this function does check the above environment variables, and does return a Unix-style locale string based on either said environment variables or the thread's current locale.

Your application should call setlocale(LC_ALL, ""); for the user settings to take effect. Gtk+ does this in its initialization functions automatically (by calling gtk_set_locale()). See man setlocale for more details.

Returns : the default language as a PangoLanguage, must not be freed.

Since 1.16


pango_language_get_sample_string ()

const char* pango_language_get_sample_string
                                            (PangoLanguage *language);

Get a string that is representative of the characters needed to render a particular language.

The sample text may be a pangram, but is not necessarily. It is chosen to be demonstrative of normal text in the language, as well as exposing font feature requirements unique to the language. It is suitable for use as sample text in a font selection dialog.

If language is NULL, the default language as found by pango_language_get_default() is used.

If Pango does not have a sample string for language, the classic "The quick brown fox..." is returned. This can be detected by comparing the returned pointer value to that returned for (non-existent) language code "xx". That is, compare to:

pango_language_get_sample_string (pango_language_from_string ("xx"))

language : a PangoLanguage, or NULL
Returns : the sample string. This value is owned by Pango and should not be freed.