词法格式

字符

文本格式为源代码赋予含义，源代码由一系列字符组成。假设字符表示为有效的Unicode（第 2.4 节）标量值。

\[\begin{split}\begin{array}{llll} \def\mathdef3317#1{{}}\mathdef3317{source} & \href{../text/lexical.html#text-source}{\mathtt{source}} &::=& \href{../text/lexical.html#text-char}{\mathtt{char}}^\ast \\ \def\mathdef3317#1{{}}\mathdef3317{character} & \href{../text/lexical.html#text-char}{\mathtt{char}} &::=& \def\mathdef3356#1{\mathrm{U{+}#1}}\mathdef3356{00} ~|~ \dots ~|~ \def\mathdef3357#1{\mathrm{U{+}#1}}\mathdef3357{D7FF} ~|~ \def\mathdef3358#1{\mathrm{U{+}#1}}\mathdef3358{E000} ~|~ \dots ~|~ \def\mathdef3359#1{\mathrm{U{+}#1}}\mathdef3359{10FFFF} \\ \end{array}\end{split}\]

注意

虽然源代码可以在注释或字符串字面量中包含任何 Unicode 字符，但语法规则的其余部分仅由 7 位ASCII Unicode 子集支持的字符构成。

标记

源代码中的字符流从左到右被划分为一系列标记，如下面的语法定义所示。

\[\begin{split}\begin{array}{llll} \def\mathdef3317#1{{}}\mathdef3317{token} & \href{../text/lexical.html#text-token}{\mathtt{token}} &::=& \href{../text/lexical.html#text-keyword}{\mathtt{keyword}} ~|~ \href{../text/values.html#text-int}{\def\mathdef3338#1{{\mathtt{u}#1}}\mathdef3338{N}} ~|~ \href{../text/values.html#text-int}{\def\mathdef3344#1{{\mathtt{s}#1}}\mathdef3344{N}} ~|~ \href{../text/values.html#text-float}{\def\mathdef3352#1{{\mathtt{f}#1}}\mathdef3352{N}} ~|~ \href{../text/values.html#text-string}{\mathtt{string}} ~|~ \href{../text/values.html#text-id}{\mathtt{id}} ~|~ \def\mathdef3360#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3360{(} ~|~ \def\mathdef3361#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3361{)} ~|~ \href{../text/lexical.html#text-reserved}{\mathtt{reserved}} \\ \def\mathdef3317#1{{}}\mathdef3317{keyword} & \href{../text/lexical.html#text-keyword}{\mathtt{keyword}} &::=& (\def\mathdef3362#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3362{a} ~|~ \dots ~|~ \def\mathdef3363#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3363{z})~\href{../text/values.html#text-idchar}{\mathtt{idchar}}^\ast \qquad (\mathrel{\mbox{if}}~\mbox{occurring as a literal terminal in the grammar}) \\ \def\mathdef3317#1{{}}\mathdef3317{reserved} & \href{../text/lexical.html#text-reserved}{\mathtt{reserved}} &::=& (\href{../text/values.html#text-idchar}{\mathtt{idchar}} ~|~ \href{../text/values.html#text-string}{\mathtt{string}})^+ \\ \end{array}\end{split}\]

标记根据最长匹配规则从输入字符流中形成。也就是说，下一个标记始终由词法语法识别的最长可能的字符序列组成。标记可以由空白分隔，但除了字符串之外，它们自身不能包含空白。

关键字标记是隐式定义的，即在终结符的字面形式中出现，例如 \(\def\mathdef3364#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3364{keyword}\)，在本章的语法产生式中，或者在本章中出现的显式情况下。

任何不属于其他类别的标记都被视为保留标记，并且不能出现在源代码中。

注意

定义保留标记集的效果是，所有标记都必须由括号、空白或注释分隔。例如，\(\def\mathdef3365#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3365{0\$x}\) 是一个保留标记，\(\def\mathdef3366#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3366{"a""b"}\) 也是一个保留标记。因此，它们不被识别为两个单独的标记 \(\def\mathdef3367#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3367{0}\) 和 \(\def\mathdef3368#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3368{\$x}\)，或分别为 \("a"\) 和 \("b"\)，而是被禁止。标记化的这种属性不受保留标记的定义与其他标记类重叠这一事实的影响。

空白

空白是任何字面空格字符、格式字符或注释的序列。允许的格式字符对应于ASCII格式效果的子集，即水平制表符 (\(\def\mathdef3369#1{\mathrm{U{+}#1}}\mathdef3369{09}\))、换行符 (\(\def\mathdef3370#1{\mathrm{U{+}#1}}\mathdef3370{0A}\)) 和回车符 (\(\def\mathdef3371#1{\mathrm{U{+}#1}}\mathdef3371{0D}\))。

\[\begin{split}\begin{array}{llclll@{\qquad\qquad}l} \def\mathdef3317#1{{}}\mathdef3317{white space} & \href{../text/lexical.html#text-space}{\mathtt{space}} &::=& (\def\mathdef3372#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3372{~~} ~|~ \href{../text/lexical.html#text-format}{\mathtt{format}} ~|~ \href{../text/lexical.html#text-comment}{\mathtt{comment}})^\ast \\ \def\mathdef3317#1{{}}\mathdef3317{format} & \href{../text/lexical.html#text-format}{\mathtt{format}} &::=& \href{../text/lexical.html#text-newline}{\mathtt{newline}} ~|~ \def\mathdef3373#1{\mathrm{U{+}#1}}\mathdef3373{09} \\ \def\mathdef3317#1{{}}\mathdef3317{newline} & \href{../text/lexical.html#text-newline}{\mathtt{newline}} &::=& \def\mathdef3374#1{\mathrm{U{+}#1}}\mathdef3374{0A} ~|~ \def\mathdef3375#1{\mathrm{U{+}#1}}\mathdef3375{0D} ~|~ \def\mathdef3376#1{\mathrm{U{+}#1}}\mathdef3376{0D}~\def\mathdef3377#1{\mathrm{U{+}#1}}\mathdef3377{0A} \\ \end{array}\end{split}\]

空白的唯一作用是分隔标记。否则会被忽略。

注释

注释可以是行注释，以双分号 \(\def\mathdef3337#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3337{{;}{;}}\) 开头，一直延伸到行末，也可以是块注释，用分隔符 \(\def\mathdef3335#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3335{{(}{;}} \dots \def\mathdef3336#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3336{{;}{)}}\) 括起来。块注释可以嵌套。

\[\begin{split}\begin{array}{llclll@{\qquad\qquad}l} \def\mathdef3317#1{{}}\mathdef3317{comment} & \href{../text/lexical.html#text-comment}{\mathtt{comment}} &::=& \href{../text/lexical.html#text-comment}{\mathtt{linecomment}} ~|~ \href{../text/lexical.html#text-comment}{\mathtt{blockcomment}} \\ \def\mathdef3317#1{{}}\mathdef3317{line comment} & \href{../text/lexical.html#text-comment}{\mathtt{linecomment}} &::=& \def\mathdef3337#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3337{{;}{;}}~~\href{../text/lexical.html#text-comment}{\mathtt{linechar}}^\ast~~(\href{../text/lexical.html#text-newline}{\mathtt{newline}} ~|~ \mathtt{eof}) \\ \def\mathdef3317#1{{}}\mathdef3317{line character} & \href{../text/lexical.html#text-comment}{\mathtt{linechar}} &::=& c{:}\href{../text/lexical.html#text-char}{\mathtt{char}} & (\mathrel{\mbox{if}} c \neq \def\mathdef3378#1{\mathrm{U{+}#1}}\mathdef3378{0A} \land c \neq \def\mathdef3379#1{\mathrm{U{+}#1}}\mathdef3379{0D}) \\ \def\mathdef3317#1{{}}\mathdef3317{block comment} & \href{../text/lexical.html#text-comment}{\mathtt{blockcomment}} &::=& \def\mathdef3335#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3335{{(}{;}}~~\href{../text/lexical.html#text-comment}{\mathtt{blockchar}}^\ast~~\def\mathdef3336#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3336{{;}{)}} \\ \def\mathdef3317#1{{}}\mathdef3317{block character} & \href{../text/lexical.html#text-comment}{\mathtt{blockchar}} &::=& c{:}\href{../text/lexical.html#text-char}{\mathtt{char}} & (\mathrel{\mbox{if}} c \neq \def\mathdef3380#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3380{;} \land c \neq \def\mathdef3381#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3381{(}) \\ &&|& \def\mathdef3382#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3382{;} & (\mathrel{\mbox{if}}~\mbox{the next character is not}~\def\mathdef3383#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3383{)}) \\ &&|& \def\mathdef3384#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3384{(} & (\mathrel{\mbox{if}}~\mbox{the next character is not}~\def\mathdef3385#1{\mbox{‘}\mathtt{#1}\mbox{’}}\mathdef3385{;}) \\ &&|& \href{../text/lexical.html#text-comment}{\mathtt{blockcomment}} \\ \end{array}\end{split}\]

这里，伪标记 \(\mathtt{eof}\) 表示输入的结束。对 \(\href{../text/lexical.html#text-comment}{\mathtt{blockchar}}\) 产生式的前瞻限制使语法消歧，以便只允许块注释分隔符的良好括号用法。

注意

任何格式和控制字符都可以在注释中使用。