What writing system in the Man'yōshū helped give kana its name?

The writing system in the Man'yōshū that helped give kana its name was Man'yōgana.

Man'yōgana used Chinese characters for their sounds rather than primarily for their meanings. Japanese writers selected kanji to represent the sounds of Old Japanese, creating a phonetic system that could record the language. The Man'yōshū, an eighth-century anthology, is the best-known major work associated with this practice.

The name Man'yōgana derives from Man'yōshū, not from either modern kana script. Over time, abbreviated or simplified forms of the characters developed into hiragana and katakana. Hiragana evolved mainly from cursive transformations of whole characters, while katakana developed from selected parts of characters used as reading aids.

Kanbun is different: it is a method for reading or adapting Classical Chinese texts in Japanese contexts. The modern scripts are descendants of the older phonetic practice, but they should not be treated as synonyms for Man'yōgana. Man'yōgana was a historical system based on kanji; hiragana and katakana are the standardized kana scripts used in modern Japanese.