Neural Information Processing - Letters and Reviews
Vol. 12, Nos. 1-3, January-March 2008
RNN with a Recurrent Output Layer for Learning of Naturalness
Ján Dolinský and Hideyuki Takagi
Kyushu University, 4-9-1 Shiobaru, Minami-ku, Fukuoka, 815-8540 JAPAN
E-mail: email@example.com, firstname.lastname@example.org
The behavior of recurrent neural networks with a recurrent output layer (ROL) is described mathematically and it is shown that using ROL is not only advantageous, but is in fact crucial to obtaining satisfactory performance for the proposed naturalness learning. Conventional belief holds that employing ROL often substantially decreases the performance of a network or renders the network unstable, and ROL is consequently rarely used. The objective of this paper is to demonstrate that there are cases where it is necessary to use ROL. The concrete example shown models naturalness in handwritten letters.
Keywords – recurrent output layer, RNN, ESN, naturalness learning, handwritten letters