Lognormality of the distribution of Japanese sentence lengths

Sho Furuhashi, Yoshinori Hayakawa

    研究成果: Article査読

    4 被引用数 (Scopus)

    抄録

    The lengths of sentences in written texts have been reported to exhibit characteristic distributions that resemble lognormal distributions. However, the mechanism responsible for such lognormality is unclear. In this quantitative study, we analyze over 10,000 Japanese sentences from out-of-copyright Japanese texts stored on Aozora Bunko. We first confirm that sentence length distributions can be better represented by the lognormal function than by other functions (e.g., the gamma distribution). Next, under the assumption that each sentence is generated by a hierarchical branching process in terms of dependency trees, we test whether the composition of sentences can be explained by a simple multiplicative process by utilizing the Japanese dependency analyzer CaboCha. The results imply that the lognormality of sentence length distributions originates from the dependency tree depth and that a simple multiplicative model cannot accurately model the processes involved in generating sentences.

    本文言語English
    論文番号034004
    ジャーナルjournal of the physical society of japan
    81
    3
    DOI
    出版ステータスPublished - 2012 3月

    ASJC Scopus subject areas

    • 物理学および天文学(全般)

    フィンガープリント

    「Lognormality of the distribution of Japanese sentence lengths」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

    引用スタイル