<!DOCTYPE html><htmllang="en"><head><metacharset="utf-8"><metaname="viewport"content="width=device-width, initial-scale=1.0"><metaname="generator"content="rustdoc"><metaname="description"content="List of all items in this crate"><title>List of all items in this crate</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2" crossorigin href="../static.files/${f}">`).join(""))</script><linkrel="stylesheet"href="../static.files/normalize-9960930a.css"><linkrel="stylesheet"href="../static.files/rustdoc-46132b98.css"><metaname="rustdoc-vars"data-root-path="../"data-static-root-path="../static.files/"data-current-crate="unicode_ident"data-themes=""data-resource-suffix=""data-rustdoc-version="1.85.1 (4eb161250 2025-03-15)"data-channel="1.85.1"data-search-js="search-75f5ac3e.js"data-settings-js="settings-0f613d39.js"><scriptsrc="../static.files/storage-59e33391.js"></script><scriptdefersrc="../static.files/main-5f194d8c.js"></script><noscript><linkrel="stylesheet"href="../static.files/noscript-893ab5e7.css"></noscript><linkrel="alternate icon"type="image/png"href="../static.files/favicon-32x32-6580c154.png"><linkrel="icon"type="image/svg+xml"href="../static.files/favicon-044be391.svg"></head><bodyclass="rustdoc mod sys"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><navclass="mobile-topbar"><buttonclass="sidebar-menu-toggle"title="show sidebar"></button></nav><navclass="sidebar"><divclass="sidebar-crate"><h2><ahref="../unicode_ident/index.html">unicode_<wbr>ident</a><spanclass="version">1.0.18</span></h2></div><divclass="sidebar-elems"><sectionid="rustdoc-toc"><h3><ahref="#functions">Crate Items</a></h3><ulclass="block"><li><ahref="#functions"title="Functions">Functions</a></li></ul></section><divid="rustdoc-modnav"></div></div></nav><divclass="sidebar-resizer"></div><main><divclass="width-limiter"><rustdoc-search></rustdoc-search><sectionid="main-content"class="content"><h1>List of all items</h1><h3id="functions">Functions</h3><ulclass="all-items"><li><ahref="fn.is_xid_continue.html">is_xid_continue</a></li><li><ahref="fn.is_xid_start.html">is_xid_start</a></li></ul></section></div></main></body></html>
<!DOCTYPE html><htmllang="en"><head><metacharset="utf-8"><metaname="viewport"content="width=device-width, initial-scale=1.0"><metaname="generator"content="rustdoc"><metaname="description"content="Whether the character has the Unicode property XID_Continue."><title>is_xid_continue in unicode_ident - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2" crossorigin href="../static.files/${f}">`).join(""))</script><linkrel="stylesheet"href="../static.files/normalize-9960930a.css"><linkrel="stylesheet"href="../static.files/rustdoc-46132b98.css"><metaname="rustdoc-vars"data-root-path="../"data-static-root-path="../static.files/"data-current-crate="unicode_ident"data-themes=""data-resource-suffix=""data-rustdoc-version="1.85.1 (4eb161250 2025-03-15)"data-channel="1.85.1"data-search-js="search-75f5ac3e.js"data-settings-js="settings-0f613d39.js"><scriptsrc="../static.files/storage-59e33391.js"></script><scriptdefersrc="sidebar-items.js"></script><scriptdefersrc="../static.files/main-5f194d8c.js"></script><noscript><linkrel="stylesheet"href="../static.files/noscript-893ab5e7.css"></noscript><linkrel="alternate icon"type="image/png"href="../static.files/favicon-32x32-6580c154.png"><linkrel="icon"type="image/svg+xml"href="../static.files/favicon-044be391.svg"></head><bodyclass="rustdoc fn"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><navclass="mobile-topbar"><buttonclass="sidebar-menu-toggle"title="show sidebar"></button></nav><navclass="sidebar"><divclass="sidebar-crate"><h2><ahref="../unicode_ident/index.html">unicode_<wbr>ident</a><spanclass="version">1.0.18</span></h2></div><divclass="sidebar-elems"><divid="rustdoc-modnav"></div></div></nav><divclass="sidebar-resizer"></div><main><divclass="width-limiter"><rustdoc-search></rustdoc-search><sectionid="main-content"class="content"><divclass="main-heading"><spanclass="rustdoc-breadcrumbs"><ahref="index.html">unicode_ident</a></span><h1>Function <spanclass="fn">is_xid_continue</span><buttonid="copy-path"title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><spanclass="sub-heading"><aclass="src"href="../src/unicode_ident/lib.rs.html#264-271">Source</a></span></div><preclass="rust item-decl"><code>pub fn is_xid_continue(ch: <aclass="primitive"href="https://doc.rust-lang.org/1.85.1/core/primitive.char.html">char</a>) -><aclass="primitive"href="https://doc.rust-lang.org/1.85.1/core/primitive.bool.html">bool</a></code></pre><detailsclass="toggle top-doc"open><summaryclass="hideme"><span>Expand description</span></summary><divclass="docblock"><p>Whether the character has the Unicode property XID_Continue.</p>
<!DOCTYPE html><htmllang="en"><head><metacharset="utf-8"><metaname="viewport"content="width=device-width, initial-scale=1.0"><metaname="generator"content="rustdoc"><metaname="description"content="Whether the character has the Unicode property XID_Start."><title>is_xid_start in unicode_ident - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2" crossorigin href="../static.files/${f}">`).join(""))</script><linkrel="stylesheet"href="../static.files/normalize-9960930a.css"><linkrel="stylesheet"href="../static.files/rustdoc-46132b98.css"><metaname="rustdoc-vars"data-root-path="../"data-static-root-path="../static.files/"data-current-crate="unicode_ident"data-themes=""data-resource-suffix=""data-rustdoc-version="1.85.1 (4eb161250 2025-03-15)"data-channel="1.85.1"data-search-js="search-75f5ac3e.js"data-settings-js="settings-0f613d39.js"><scriptsrc="../static.files/storage-59e33391.js"></script><scriptdefersrc="sidebar-items.js"></script><scriptdefersrc="../static.files/main-5f194d8c.js"></script><noscript><linkrel="stylesheet"href="../static.files/noscript-893ab5e7.css"></noscript><linkrel="alternate icon"type="image/png"href="../static.files/favicon-32x32-6580c154.png"><linkrel="icon"type="image/svg+xml"href="../static.files/favicon-044be391.svg"></head><bodyclass="rustdoc fn"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><navclass="mobile-topbar"><buttonclass="sidebar-menu-toggle"title="show sidebar"></button></nav><navclass="sidebar"><divclass="sidebar-crate"><h2><ahref="../unicode_ident/index.html">unicode_<wbr>ident</a><spanclass="version">1.0.18</span></h2></div><divclass="sidebar-elems"><divid="rustdoc-modnav"></div></div></nav><divclass="sidebar-resizer"></div><main><divclass="width-limiter"><rustdoc-search></rustdoc-search><sectionid="main-content"class="content"><divclass="main-heading"><spanclass="rustdoc-breadcrumbs"><ahref="index.html">unicode_ident</a></span><h1>Function <spanclass="fn">is_xid_start</span><buttonid="copy-path"title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><spanclass="sub-heading"><aclass="src"href="../src/unicode_ident/lib.rs.html#254-261">Source</a></span></div><preclass="rust item-decl"><code>pub fn is_xid_start(ch: <aclass="primitive"href="https://doc.rust-lang.org/1.85.1/core/primitive.char.html">char</a>) -><aclass="primitive"href="https://doc.rust-lang.org/1.85.1/core/primitive.bool.html">bool</a></code></pre><detailsclass="toggle top-doc"open><summaryclass="hideme"><span>Expand description</span></summary><divclass="docblock"><p>Whether the character has the Unicode property XID_Start.</p>
<!DOCTYPE html><htmllang="en"><head><metacharset="utf-8"><metaname="viewport"content="width=device-width, initial-scale=1.0"><metaname="generator"content="rustdoc"><metaname="description"content="githubcrates-iodocs-rs"><title>unicode_ident - Rust</title><script>if(window.location.protocol!=="file:")document.head.insertAdjacentHTML("beforeend","SourceSerif4-Regular-6b053e98.ttf.woff2,FiraSans-Regular-0fe48ade.woff2,FiraSans-Medium-e1aa3f0a.woff2,SourceCodePro-Regular-8badfe75.ttf.woff2,SourceCodePro-Semibold-aa29a496.ttf.woff2".split(",").map(f=>`<link rel="preload" as="font" type="font/woff2" crossorigin href="../static.files/${f}">`).join(""))</script><linkrel="stylesheet"href="../static.files/normalize-9960930a.css"><linkrel="stylesheet"href="../static.files/rustdoc-46132b98.css"><metaname="rustdoc-vars"data-root-path="../"data-static-root-path="../static.files/"data-current-crate="unicode_ident"data-themes=""data-resource-suffix=""data-rustdoc-version="1.85.1 (4eb161250 2025-03-15)"data-channel="1.85.1"data-search-js="search-75f5ac3e.js"data-settings-js="settings-0f613d39.js"><scriptsrc="../static.files/storage-59e33391.js"></script><scriptdefersrc="../crates.js"></script><scriptdefersrc="../static.files/main-5f194d8c.js"></script><noscript><linkrel="stylesheet"href="../static.files/noscript-893ab5e7.css"></noscript><linkrel="alternate icon"type="image/png"href="../static.files/favicon-32x32-6580c154.png"><linkrel="icon"type="image/svg+xml"href="../static.files/favicon-044be391.svg"></head><bodyclass="rustdoc mod crate"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><navclass="mobile-topbar"><buttonclass="sidebar-menu-toggle"title="show sidebar"></button></nav><navclass="sidebar"><divclass="sidebar-crate"><h2><ahref="../unicode_ident/index.html">unicode_<wbr>ident</a><spanclass="version">1.0.18</span></h2></div><divclass="sidebar-elems"><ulclass="block"><li><aid="all-types"href="all.html">All Items</a></li></ul><sectionid="rustdoc-toc"><h3><ahref="#">Sections</a></h3><ulclass="block top-toc"><li><ahref="#comparison-of-performance"title="Comparison of performance">Comparison of performance</a></li><li><ahref="#comparison-of-data-structures"title="Comparison of data structures">Comparison of data structures</a><ul><li><ahref="#unicode-xid"title="unicode-xid">unicode-xid</a></li><li><ahref="#ucd-trie"title="ucd-trie">ucd-trie</a></li><li><ahref="#fst"title="fst">fst</a></li><li><ahref="#roaring"title="roaring">roaring</a></li><li><ahref="#unicode-ident"title="unicode-ident">unicode-ident</a></li></ul></li></ul><h3><ahref="#functions">Crate Items</a></h3><ulclass="block"><li><ahref="#functions"title="Functions">Functions</a></li></ul></section><divid="rustdoc-modnav"></div></div></nav><divclass="sidebar-resizer"></div><main><divclass="width-limiter"><rustdoc-search></rustdoc-search><sectionid="main-content"class="content"><divclass="main-heading"><h1>Crate <span>unicode_ident</span><buttonid="copy-path"title="Copy item path to clipboard">Copy item path</button></h1><rustdoc-toolbar></rustdoc-toolbar><spanclass="sub-heading"><aclass="src"href="../src/unicode_ident/lib.rs.html#1-271">Source</a></span></div><detailsclass="toggle top-doc"open><summaryclass="hideme"><span>Expand description</span></summary><divclass="docblock"><p><ahref="https://github.com/dtolnay/unicode-ident"><imgsrc="https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github"alt="github"/></a><ahref="https://crates.io/crates/unicode-ident"><imgsrc="https://img.shields.io/badge/crates.io-fc8d62?style=for-the-badge&labelColor=555555&logo=rust"alt="crates-io"/></a><ahref="https://docs.rs/unicode-ident"><imgsrc="https://img.shields.io/badge/docs.rs-66c2a5?style=for-the-badge&labelColor=555555&logo=docs.rs"alt="docs-rs"/></a></p>
<br>
<p>Implementation of <ahref="https://www.unicode.org/reports/tr31/">Unicode Standard Annex #31</a> for determining which
<code>char</code> values are valid in programming language identifiers.</p>
<p>This crate is a better optimized implementation of the older <code>unicode-xid</code>
crate. This crate uses less static storage, and is able to classify both
ASCII and non-ASCII codepoints with better performance, 2–6×
faster than <code>unicode-xid</code>.</p>
<br>
<h3id="comparison-of-performance"><aclass="doc-anchor"href="#comparison-of-performance">§</a>Comparison of performance</h3>
<p>The following table shows a comparison between five Unicode identifier
implementations.</p>
<ul>
<li><code>unicode-ident</code> is this crate;</li>
<li><ahref="https://github.com/unicode-rs/unicode-xid"><code>unicode-xid</code></a> is a widely used crate run by the “unicode-rs” org;</li>
<li><code>ucd-trie</code> and <code>fst</code> are two data structures supported by the
<p>Source code for the benchmark is provided in the <em>bench</em> directory of this
repo and may be repeated by running <code>cargo criterion</code>.</p>
<br>
<h3id="comparison-of-data-structures"><aclass="doc-anchor"href="#comparison-of-data-structures">§</a>Comparison of data structures</h3><h5id="unicode-xid"><aclass="doc-anchor"href="#unicode-xid">§</a>unicode-xid</h5>
<p>They use a sorted array of character ranges, and do a binary search to look
up whether a given character lands inside one of those ranges.</p>
<p>Uncompressed, these would take 140 K to store, which is beyond what would be
reasonable. However, as you can see there is a large degree of similarity
between the two bitmaps and across the rows, which lends well to
compression.</p>
<p>This crate stores one 512-bit “row” of the above bitmaps in the leaf level
of a trie, and a single additional level to index into the leafs. It turns
out there are 124 unique 512-bit chunks across the two bitmaps so 7 bits are
sufficient to index them.</p>
<p>The chunk size of 512 bits is selected as the size that minimizes the total
size of the data structure. A smaller chunk, like 256 or 128 bits, would
achieve better deduplication but require a larger index. A larger chunk
would increase redundancy in the leaf bitmaps. 512 bit chunks are the
optimum for total size of the index plus leaf bitmaps.</p>
<p>In fact since there are only 124 unique chunks, we can use an 8-bit index
with a spare bit to index at the half-chunk level. This achieves an
additional 8.5% compression by eliminating redundancies between the second
half of any chunk and the first half of any other chunk. Note that this is
not the same as using chunks which are half the size, because it does not
necessitate raising the size of the trie’s first level.</p>
<p>In contrast to binary search or the <code>ucd-trie</code> crate, performing lookups in
this data structure is straight-line code with no need for branching.</p>
</div></details><h2id="functions"class="section-header">Functions<ahref="#functions"class="anchor">§</a></h2><ulclass="item-table"><li><divclass="item-name"><aclass="fn"href="fn.is_xid_continue.html"title="fn unicode_ident::is_xid_continue">is_<wbr>xid_<wbr>continue</a></div><divclass="desc docblock-short">Whether the character has the Unicode property XID_Continue.</div></li><li><divclass="item-name"><aclass="fn"href="fn.is_xid_start.html"title="fn unicode_ident::is_xid_start">is_<wbr>xid_<wbr>start</a></div><divclass="desc docblock-short">Whether the character has the Unicode property XID_Start.</div></li></ul></section></div></main></body></html>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
github-actions[bot]