Package index • tidycjk

Summarise a text column

Verbs taking (data, column) and returning a tibble, for a dplyr pipeline.

cjk_summary(): Summarise CJK content in a text column
cjk_char_counts(): Count the CJK characters in a text column
cjk_tokens(): One row per token

Segment

Splitting CJK text into words. Where a word ends is a fact about a language rather than about Unicode, so no segmenter is bundled and the engine is a required, pluggable choice.

cjk_segment(): Split CJK text into words
cjk_segmenters() register_cjk_segmenter(): Segmentation engines

Detect & classify

has_cjk(): Does the text contain CJK characters?
cjk_script(): Which CJK script dominates the text?
cjk_detect_language(): Which language is the text written in?
cjk_ratio(): What share of the text is CJK?

Display width

Terminal columns rather than characters, so CJK tables line up.

cjk_width(): Display width in terminal columns
cjk_pad(): Pad text to a display width
cjk_truncate(): Truncate text to a display width

Normalise width variants

to_halfwidth() to_fullwidth(): Normalise fullwidth and halfwidth forms

Reference data

cjk_blocks(): The Unicode blocks 'tidycjk' recognises

Package

tidycjk tidycjk-package: tidycjk: Tidy Tools for Chinese, Japanese and Korean Text