text-generation-webui/grammars/japanese.gbnf

mirror of https://github.com/oobabooga/text-generation-webui.git synced 2024-11-23 16:38:21 +01:00
# A probably incorrect grammar for Japanese
root        ::= jp-char+ ([ \t\n] jp-char+)*
jp-char     ::= hiragana | katakana | punctuation | cjk
hiragana    ::= [ぁ-ゟ]
katakana    ::= [ァ-ヿ]
punctuation ::= [、-〾]
cjk         ::= [一-鿿]
Add grammar to llama.cpp loader (closes #4019) 2023-09-24 16:08:41 +02:00			`# A probably incorrect grammar for Japanese`
			`root ::= jp-char+ ([ \t\n] jp-char+)*`
			`jp-char ::= hiragana \| katakana \| punctuation \| cjk`
			`hiragana ::= [ぁ-ゟ]`
			`katakana ::= [ァ-ヿ]`
			`punctuation ::= [、-〾]`
			`cjk ::= [一-鿿]`