tr59-1.html
631 lines<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head><base href="https://www.unicode.org/reports/tr59/tr59-1.html">
<title>UTR #59: East Asian Spacing</title>
<link rel="stylesheet" type="text/css" href="https://www.unicode.org/reports/reports-v2.css">
<style type="text/css">
th {
background-color: #CCFFCC;
}
ul.inline {
display: inline;
margin: 0;
padding: 0;
li {
display: inline;
list-style: none;
}
li::after {
content: ',';
}
li:last-of-type::after {
content: '.';
}
}
</style>
</head>
<body>
<table class="header">
<tr>
<td class="icon" style="width:38px; height:35px">
<a href="https://www.unicode.org/">
<img border="0" src="https://www.unicode.org/webscripts/logo60s2.gif" align="middle"
alt="[Unicode]" width="34" height="33"></a>
</td>
<td class="icon" style="vertical-align:middle">
<a class="bar"> </a>
<a class="bar" href="https://www.unicode.org/reports/"><font size="3">Technical Reports</font></a>
</td>
</tr>
<tr>
<td colspan="2" class="gray"> </td>
</tr>
</table>
<div class="body">
<h2 class="uaxtitle"><span class="changed">Proposed Draft</span> Unicode® Technical Report #59</h2>
<h1>East Asian Spacing</h1>
<table class="simple" width="90%">
<tr>
<td valign="top">Editors</td>
<td valign="top">Koji Ishii 石井宏治 (Google Inc.)</td>
</tr>
<tr>
<td valign="top">Date</td>
<td valign="top">2024-12-16</td>
</tr>
<tr>
<td valign="top">This Version</td>
<td valign="top">
<a href="https://www.unicode.org/reports/tr59/tr59-1.html">https://www.unicode.org/reports/tr59/tr59-1.html</a>
</td>
</tr>
<tr>
<td valign="top">Previous Version</td>
<td valign="top">
n/a
</td>
</tr>
<tr>
<td valign="top">Latest Version</td>
<td valign="top"><a href="https://www.unicode.org/reports/tr59/">https://www.unicode.org/reports/tr59/</a></td>
</tr>
<!--
<tr>
<td valign="top">Latest Proposed Update</td>
<td valign="top"><a href="https://www.unicode.org/reports/tr59/proposed.html">https://www.unicode.org/reports/tr59/proposed.html</a></td>
</tr>
-->
<tr>
<td valign="top">Revision</td>
<td valign="top"><a href="#Modifications">1</a></td>
</tr>
</table>
<h4 class="summary">Summary</h4>
<blockquote><p><i>
East Asian established typography defines that a small amount of visible space
between East Asian scripts and other scripts improves readability.
This report describes the algorithm and the data
which can be used to automatically add visible space.
</i></p></blockquote>
<h4 class="status">Status</h4>
<!-- NOT YET APPROVED -->
<p><i><span class="changed">This is a<b><font color="#ff3333"> draft </font></b>document which
may be updated, replaced, or superseded by other documents at any time.
Publication does not imply endorsement by the Unicode Consortium. This is
not a stable document; it is inappropriate to cite this document as other
than a work in progress.</span></i></p>
<!-- END NOT YET APPROVED -->
<!-- APPROVED
<p><i>This document has been reviewed by Unicode members and other interested
parties, and has been approved for publication by the Unicode Consortium.
This is a stable document and may be used as reference material or cited as
a normative reference by other specifications.</i></p>
END APPROVED -->
<blockquote>
<p><i><b>A Unicode Technical Report (UTR)</b> contains informative material.
Conformance to the Unicode Standard does not imply conformance to any UTR.
Other specifications, however, are free to make normative references to a UTR.
</i></p>
</blockquote>
<p><i>Please submit corrigenda and other comments with the online reporting
form [<a href="https://www.unicode.org/reporting.html">Feedback</a>].
Related information that is useful in understanding this annex is found in Unicode Standard Annex #41,
“<a href="https://www.unicode.org/reports/tr41/tr41-34.html">Common References for Unicode Standard Annexes</a>.”
For the latest version of the Unicode Standard, see [<a href="https://www.unicode.org/versions/latest/">Unicode</a>].
For a list of current Unicode Technical Reports, see [<a href="https://www.unicode.org/reports/">Reports</a>].
For more information about versions of the Unicode Standard, see [<a href="https://www.unicode.org/versions/">Versions</a>].
</i></p>
<h4 class="contents">Contents</h4>
<ul class="toc">
<li>1 <a href="#overview">Overview and Scope</a></li>
<li>2 <a href="#conformance">Conformance</a>
<li>3 <a href="#property">The East_Asian_Spacing Property</a>
<li><ul class="toc">
<li>3.1 <a href="#definitions">Definitions</a></li>
<li>3.2 <a href="#values">Property Values</a></li>
<li>3.3 <a href="#spacing">Spacing Algorithm</a></li>
<li><ul class="toc">
<li>3.3.1 <a href="#algorithm-steps">Algorithm Steps</a></li>
</ul></li>
<li>3.4 <a href="#scope">Scope of the Property</a></li>
<li><ul class="toc">
<li>3.4.1 <a href="#grapheme-cluster">Grapheme Cluster</a></li>
<li>3.4.2 <a href="#space-char">Space Characters</a></li>
<li>3.4.3 <a href="#symbols-punctuation">Symbols and Punctuation Characters</a></li>
<li>3.4.4 <a href="#vertical">Vertical Text Layout</a></li>
<li>3.4.5 <a href="#rtl">Right-to-Left Scripts</a></li>
</ul></li>
</ul></li>
<li>4 <a href="#data">Data File</a>
<li><a href="#Acknowledgments">Acknowledgments</a></li>
<li><a href="#References">References</a></li>
<li><a href="#Modifications">Modifications</a></li>
</ul>
<hr>
<h2>1 <a name="overview" href="#overview">Overview and Scope</a></h2>
<p>
East Asian text usually consists of multiple scripts, such as Han ideographs,
Kana syllables, and Hangul syllables, along with Latin letters and numeric
characters.
East Asian established typography conventions define that
a thin spacing between East Asian scripts and other scripts
improves readability.
This spacing should be represented by
adjusting glyph spacings similar to kerning,
rather than by using space characters.
</p><p>
While detailed rules of the spacing can vary across documents, it is important
that the choice made by an author for a specific document be clearly
established, so that a rendering system can display what the author intended.
It is also important that this choice be established independently of the font
resources, as the rendering systems may have to use other fonts than those
intended or specified in the document. Finally, the expression of the author’s
choice should be relatively concise, to facilitate document authoring and
minimize document size.
</p><p>
This report describes an algorithm, together with a character property,
which can serve as a default for
inserting spacing for the purpose of reliable document
interchange.
</p><p>
The character property is independent of external information,
such as provided in the tables of the selected font,
so that implementations could reliably render a
character stream based solely on the property values.
</p><p>
The intent is that an author should be able to specify where they want to
override, and that in the absence of an explicit specification, the spacing is
applied according to this specification.
</p><p>
The actual choice for the property values should result in a reasonable or
legible default, but it may not be publishing-material quality, and it may not
be a good choice if used in a specific style or context.
</p><p>
The property values are chosen first to match existing practice in East Asian
contexts in their respective environments. For characters that are not generally
used in such environments, similarity to existing characters has been taken into
consideration. It also takes East Asian characters in non-East Asian texts into
account.
</p>
<h2>2 <a name="conformance" href="#conformance">Conformance</a></h2>
<p>
The property defined in this report is informative. The intent of this report is
to provide, in the absence of other information, a reasonable way to determine
the correct automatic spacing, but this behavior can be overridden by inserting
space characters
as described in <a href="#space-char">Space Characters</a>,
or by a higher-level protocol, such as through markup or by
the preferences of a layout application. This default determination is defined
in the accompanying data file <a href="#data">Data</a>,
but in no way implies that the spacing is
inserted only by this rule.
</p><p>
For more information on the conformance implications, see
[<a href="https://www.unicode.org/versions/latest/">Unicode</a>],
Section 3.5, Properties,
in particular the definition (D35) of an informative property.
</p>
<h2>3 <a name="property" href="#property">The East_Asian_Spacing Property</a></h2>
<p class="reviewnote">
The name of this specification and of this property reflect well-known East
Asian typographic tradition.
UTC members are not currently aware of similar traditions
for other writing systems.
It would be possible to widen the set of scripts
considered by this algorithm in the future,
and rename this specification accordingly, but once established,
the name of the property would have to remain stable (at least as an alias).
If there is a reasonable expectation that this may happen,
then the property could be renamed without the "East Asian" reference.
</p>
<p class="reviewnote">There is currently no proposed short property name.</p>
<h3>3.1 <a name="definitions" href="#definitions">Definitions</a></h3>
<p>
<i><b><a name="UTR59-D1" href="#UTR59-D1">UTR59-D1</a></b></i>.
<i>Auto spacing:</i>
The addition of a small amount of visible space in rendered text
as determined by an algorithm like the one defined in this document.
Depending on the implementation,
a visible space may be added as a glyph space in a text layout process,
or by adding an appropriate Unicode space character.
</p>
<p>
<i><b><a name="UTR59-D2" href="#UTR59-D2">UTR59-D2</a></b></i>.
<i>East Asian scripts:</i>
Scripts in the following list:
</p>
<ul>
<li>Bopomofo (Bopo)</li>
<li>Han (Hani)</li>
<li>Hangul (Hang)</li>
<li>Hiragana (Hira)</li>
<li>Katakana (Kana)</li>
<li>Khitan_Small_Script (Kits)</li>
<li>Nushu (Nshu)</li>
<li>Tangut (Tang)</li>
<li>Yi (Yiii)</li>
</ul>
<p>
<i><b><a name="UTR59-D3" href="#UTR59-D3">UTR59-D3</a></b></i>.
<i>Language context:</i>
The language of the input string,
which may be detected,
or determined by higher-level protocols,
the preferences of the application or the operating system,
or other appropriate methods.
</p>
<h3>3.2 <a name="values" href="#values">Property Values</a></h3>
<p>
The possible property values are given in Table 1.
</p>
<table class="simple">
<caption>Table 1. Property Values</caption>
<tr class="heading">
<th>Value</th>
<th>Short Name</th>
<th>Description</th>
<th>Examples</th>
</tr>
<tr>
<th>Wide</th>
<td>W</td>
<td>
Characters that are considered as East Asian script.
</td>
<td>Han ideographic characters and Kana syllables.</td>
</tr>
<tr>
<th>Narrow</th>
<td>N</td>
<td>Characters that need auto-spacing with adjacent “W” characters.</td>
<td>Latin letters and digits.</td>
</tr>
<tr>
<th>Other</th>
<td>O</td>
<td>Characters that don’t need auto-spacing.</td>
<td>
Most symbols and punctuation characters
such as COMMA and FULL STOP.
</td>
</tr>
<tr>
<th>Conditional</th>
<td>C</td>
<td>Characters that are “N” for Chinese languages,
and “O” for other languages
or when the language is unknown.</td>
<td>
Characters that can appear as prefix or suffix of Latin letters or digits,
such as U+0025 PERCENT SIGN.
</td>
</tr>
</table>
<p>
Characters that have the property value “N” are
similar to “East_Asian_Width=Narrow” characters in
[<a href="https://unicode.org/reports/tr11/">UAX11</a>],
but most punctuation characters and symbols are excluded.
Similarly, characters that have the property value “W” are similar to
“East_Asian_Width=Wide” characters,
but most punctuation characters and symbols are excluded.
Also, to follow existing practice, circled characters,
square characters, and Emoji are defined as “O”.
</p><p>
For the value “C”, please refer to
<a href="#symbols-punctuation">Symbols and Punctuation Characters</a>
for more details.
</p>
<h3>3.3 <a name="spacing" href="#spacing">Spacing Algorithm</a></h3>
<p>
The East Asian Spacing should be inserted between “W” and “N”, and between “N” and “W”,
after resolving the conditional value “C” to “N” or “O”.
</p><p>
The exact amount of the spacing can vary across documents. This property doesn’t
define the exact amount. Instead, it should be defined by higher-level protocols
or applications such as through markup or by the preferences of a layout
application.
</p><p>
There are two ways to represent a space:
As a Unicode space character,
or as a glyph space (similar to kerning,
adjusting the metrics of adjacent glyphs on the device).
Higher-level protocols or applications should use glyph spaces where possible.
</p>
<h4>3.3.1 <a name="algorithm-steps" href="#algorithm-steps">Algorithm Steps</a></h4>
<p>Inputs:</p>
<ol>
<li>An input string.</li>
<li>Whether the <a href="#UTR59-D3">language context</a>
is a Chinese language or not.
<ol>
<li>If the language is unknown, it is not a Chinese language.</li>
<li>It is a Chinese language if the language subtag is the
macrolanguage "zh" (Chinese, widely used for tagging Mandarin)
or one of the language subtags that are encompassed by "zh";
for example "cmn" (Mandarin), "yue" (Cantonese), "hak" (Hakka).
(See “Macrolanguage: zh” in the
<a href="https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry">IANA language-subtag-registry</a>.)</li>
</ol></li>
<li>If the string is to be displayed as vertical text,
then this algorithm also needs information for each grapheme cluster for
whether it is to be displayed upright or rotated;
this can be determined by performing the algorithm in [<a href="https://unicode.org/reports/tr50/">UAX50</a>],
or by a higher-level protocol.</li>
</ol>
<p>Process the input string as follows.</p>
<ol>
<li>Iterate over each grapheme cluster in the input string according to
[<a href="https://www.unicode.org/reports/tr29/">UAX29</a>].</li>
<li>In each grapheme cluster, fetch the first code point.</li>
<li>Look up the East_Asian_Spacing property value for this code point.</li>
<li>If the string is displayed in vertical text layout,
then determine if the character is displayed upright:
The character is displayed upright if
[<a href="https://unicode.org/reports/tr50/">UAX50</a>]
resolves the character's Vertical_Orientation property value
to U, Tu, or Tr; or if a higher-level protocol overrides it to be displayed upright.</li>
<li>If the string is displayed in vertical text layout,
and if the character is displayed upright,
and if the property value is Narrow (N),
then change the property value to Other (O).</li>
<li>If the property value is Conditional (C):
If the <a href="#UTR59-D3">language context</a> is a Chinese language,
then change the property value to Narrow (N);
otherwise change the property value to Other (O).</li>
<li>Apply the property value to the entire grapheme cluster.</li>
<li>If a grapheme cluster G1 is followed by a grapheme cluster G2,
and the property value of G1 is Wide (W) and the property value of G2 is Narrow (N),
or the value of G1 is Narrow (N) and the value of G2 is Wide (W),
then insert a visible space between G1 and G2.</li>
</ol>
<h3>3.4 <a name="scope" href="#scope">Scope of the Property</a></h3>
<h4>3.4.1 <a name="grapheme-cluster" href="#grapheme-cluster">Grapheme Cluster</a></h4>
<p>
As in all matters of typography, the interesting unit of text is not a
character, but a grapheme cluster: it does not make sense to insert the auto
spacing between a base character and a combining mark attached to it.
Implementations should insert the auto-spacing before or after each grapheme
cluster.
</p><p>
A possible choice for the notion of grapheme cluster is either that of legacy
grapheme cluster or that of extended grapheme cluster, as defined in
[<a href="https://www.unicode.org/reports/tr29/">UAX29</a>].
</p>
<p class="reviewnote">Note that UAX #29 by default specifies extended grapheme clusters.
We should probably settle on those, without option.</p>
<p>
The property value for a grapheme cluster as a whole is then determined by
taking the property value of the first character in the cluster, with the
following exception:
</p>
<ul>
<li>
If the cluster contains an enclosing combining mark (General_Category=Me),
then the whole cluster has the East_Asian_Spacing property value “O”.
</li>
</ul>
<h4>3.4.2 <a name="space-char" href="#space-char">Space Characters</a></h4>
<p>
The property values for space characters (General_Category=Zs) are “O”.
This is to avoid inserting the auto-spacing around space characters,
which can lead to undesirable double spacing.
</p><p>
It also allows authors to override the algorithm
when higher-level protocols or applications don’t provide a way
for authors to express their intent to override this algorithm,
such as plain text files.
</p><p>
U+0020 SPACE indicates a semantic boundary,
which is stronger than the spacing for readability.
Using the code point for the East Asian Spacing purpose
can make distinguishing semantic boundaries
from the spacing for readability difficult,
and therefore it’s discouraged.
</p><p>
U+2009 THIN SPACE should usually represent a thin space,
which is suitable to represent the auto-spacing for readability.
Inserting the code point
to where the algorithm doesn’t insert the auto-spacing
should indicate that the auto-spacing is desired there.
</p><p>
Likewise, inserting U+200B ZERO WIDTH SPACE
to where the algorithm inserts the auto-spacing
should prevent the auto-spacing from being inserted by rendering systems.
</p>
<h4>3.4.3 <a name="symbols-punctuation" href="#symbols-punctuation">Symbols and Punctuation Characters</a></h4>
<p>
In some existing practices,
symbols and punctuation characters insert the spacing,
while they don’t in other existing practices.
</p><p>
For example, when one side of a word is a letter or a digit
and the other side is a punctuation character,
such as “20%”, “$20”, or “C#”,
and they appear adjacent to “W” characters,
some existing practices
prefer inserting the spacing to both sides of them,
considering that not doing so look unbalanced.
</p><p>
On the other hand, some other existing practices prefer
not inserting the spacing
between punctuation characters and “W” characters
even in such cases.
They view the spacing as a way to secure legibility
by preventing East Asian letters from being too close
to other letters and numeral digits,
rather than to highlight words as parentheses do.
</p><p>
The conditional value “C” is assigned to such characters.
Chinese languages content often prefer the spacing in such cases,
and thus they should resolve “C” to “N”,
while other content should resolve “C” to “O”.
</p><p>
If the author is uncertain
whether their content is used in a Chinese language context or not,
and they want to express their intentions,
they can override the algorithm as described in
<a href="#space-char">Space Characters</a>.
</p>
<h4>3.4.4 <a name="vertical" href="#vertical">Vertical Text Layout</a></h4>
<p>
In vertical text layout, a character may be displayed upright or sideways
rotated.
A character is displayed upright
if the resolved Vertical_Orientation property value
by performing the algorithm in [<a href="https://unicode.org/reports/tr50/">UAX50</a>]
is U, Tu, or Tr,
or if higher-level protocols override to display it upright.
</p><p>
If a character that has the East_Asian_Spacing property value “N” is displayed upright,
the rendering system should handle it
as if it has the property value “O” instead.
</p>
<h4>3.4.5 <a name="rtl" href="#rtl">Right-to-Left Scripts</a></h4>
<p>
This specification has a current limitation in that the handling of right-to-left
scripts is not specified. This includes scripts that are predominantly written
right to left, such as Arabic, along with right-to-left scripts that are meant
to be written vertically, such as Chorasmian.
</p>
<h2>4 <a name="data" href="#data">Data File</a></h2>
<p>
This property is derived by the following algorithm:
</p>
<ol>
<li>
Assign the property value “W” if it’s in the following set:
<ol>
<li>
Include if the Script property is one of the
<a href="#UTR59-D2">East Asian scripts</a>.
</li>
<li>
Include if the Script_Extensions property is one of the
<a href="#UTR59-D2">East Asian scripts</a>,
except when the East_Asian_Width property is
“Neutral (N)” or “Narrow (Na)”.
</li>
<li>Exclude if the East_Asian_Width property is
“East Asian Halfwidth (H)”.</li>
<li>Exclude if the General_Category property is
“Punctuation (P)” or “Other_Number (No)”.</li>
<li>Exclude if the General_Category property is
“Symbol (S)” except “Modifier_Symbol (Sk)”.</li>
<li>Include the following code point:
<ul class="inline">
<li>U+3013 GETA MARK</li>
</ul>
</li>
</ol>
</li>
<li>
Otherwise, assign the property value “C” if it’s in the following set:
<ol>
<li>Include if the General_Category property is
“Other_Punctuation (Po)”.</li>
<li>Exclude if the East_Asian_Width property is
“East Asian Fullwidth (F)”, “East Asian Halfwidth (H)”,
or “East Asian Wide (W)”.</li>
<li>Exclude the following code points:
<ul class="inline">
<li>U+0022 QUOTATION MARK</li>
<li>U+0027 APOSTROPHE</li>
<li>U+002A ASTERISK</li>
<li>U+002F SOLIDUS</li>
<li>U+00B7 MIDDLE DOT</li>
<li>U+2020 DAGGER</li>
<li>U+2021 DOUBLE DAGGER</li>
<li>U+2026 HORIZONTAL ELLIPSIS</li>
</ul>
</li>
</ol>
</li>
<li>
Otherwise, assign the property value “N” if it’s in the following set:
<ol>
<li>Include if the General_Category property is
“Letter (L)”, “Mark (M)”, or “Decimal_Number (Nd)”.</li>
<li>Exclude if the East_Asian_Width property is
“East Asian Fullwidth (F)", “East Asian Halfwidth (H)”,
or “East Asian Wide (W)”.</li>
</ol>
</li>
<li>Otherwise, assign the property value “O”.</li>
</ol>
<p>
The
<a href="east-asian-spacing.txt">
derived data file</a>
is available for reference.
</p>
<h2 class="nonumber"><a name="Acknowledgments" href="#Acknowledgments">Acknowledgments</a></h2>
<p>
The initial version is based on a proposal
authored by Koji Ishii, Yasuo Kida, and Fuqiao Xue.
The proposal is originated from a discussion at W3C
primarily driven by Elika Etemad and Florian Rivoal.
Markus Scherer assisted in the documentation.
Peter Constable,
Asmus Freytag,
Ken Lunde,
Nat McCully,
Taro Yamamoto,
and many members of the UTC have participated in the review of the data and the documentation.
</p>
<h2 class="nonumber"><a name="References" href="#References">References</a></h2>
<p>For references for this report, see Unicode Standard Annex #41, "<a href="../tr41/tr41-34.html">Common
References for Unicode Standard Annexes</a>."</p>
<h2 class="nonumber"><a name="Modifications" href="#Modifications">Modifications</a></h2>
<p>The following summarizes modifications from previous revisions of this
report.</p>
<h3>Revision 1</h3>
<ul>
<li>First working draft.</li>
</ul>
<hr width="50%">
<p class="copyright">© 2024 Unicode, Inc. This publication is protected by copyright, and permission must be obtained from Unicode, Inc. prior to any reproduction, modification, or other use not permitted by the <a href="https://www.unicode.org/copyright.html">Terms of Use</a>. Specifically, you may make copies of this publication and may annotate and translate it solely for personal or internal business purposes and not for public distribution, provided that any such permitted copies and modifications fully reproduce all copyright and other legal notices contained in the original. You may not make copies of or modifications to this publication for public distribution, or incorporate it in whole or in part into any product or publication without the express written permission of Unicode.</p>
<p class="copyright">Use of all Unicode Products, including this publication, is governed by the Unicode <a href="https://www.unicode.org/copyright.html">Terms of Use</a>. The authors, contributors, and publishers have taken care in the preparation of this publication, but make no express or implied representation or warranty of any kind and assume no responsibility or liability for errors or omissions or for consequential or incidental damages that may arise therefrom. This publication is provided “AS-IS” without charge as a convenience to users.</p>
<p class="copyright">Unicode and the Unicode Logo are registered trademarks of Unicode, Inc., in the United States and other countries.</p>
</div> <!-- body -->
</body>
</html>Rendered documentLive HTML preview