Dissertation committee:
Γεώργιος Κουρουπέτρογλου, Αναπληρωτής Καθηγητής, ΕΚΠΑ
Στέλλα Βοσνιάδου, Ομότιμη Καθηγήτρια, ΕΚΠΑ
Αθηνά Ζώνιου Σιδέρη, Ομότιμη Καθηγήτρια, ΕΚΠΑ
Κωνσταντίνος Παπαδόπουλος, Καθηγητής, Πανεπιστήμιο Μακεδονίας
Κωνσταντίνος Μουτούσης, Αναπληρωτής Καθηγητής, ΕΚΠΑ
Βασίλειος Αργυρόπουλος, Αναπληρωτής Καθηγητής, Πανεπιστήμιο Θεσσαλίας
Μαρία Ρούσσου, Επίκουρη Καθηγήτρια, ΕΚΠΑ
Summary:
In this thesis, a framework is provided for the auditory representation of visual signals, such as different size, weight and style of characters, used in textbooks and generally in any structured text.
The writing signals (optical nature labels), which define the structure, the organization and the important points of a structured document, when it is presented visually, transfer information (metadata) in addition to the meaning of the text and affect the process of understanding and memorizing.
Previous studies in the field of auditory representation of structured texts have not been carried out in a systematic method based on the principles of psychology, by considering the perceptual capabilities, limitations, and the cognitive factors that impact on people during the auditory representation of a structured text. The choice of acoustic characteristics by the software developers for the performance of these signals made in an ad hoc manner, have thus far led to inefficient efforts.
In order to create perceptually acceptable acoustic versions, we had to answer two questions: First, which is better to be rendered in audio form, the optical characteristics or the meanings they carry and secondly, what are the appropriate sounds or prosodic features for the auditory display of the signals, which will be easily memorized, will not distract the listener’s attention and will not create cognitive overload?
A series of studies in Greek and American textbooks, and the largest Greek publishing houses, showed that besides the size of the font, there are significant differences in the meanings carried by different weights and style characters.
We investigated the way in which these characteristics are used by publishing houses, as well as the type of metadata that readers perceive as they read a structured text. The results of these investigations showed that there were no clear rules and consistency in the use of different weights and styles of characters in the texts of schoolbooks.
In order to attribute the visual writing signals into acoustic form, we explored, through a series of psychoacoustic experiments, the existence of relational similarity between these signals and the features of speech, such as the intensity, the pitch and the rhythm.
Our method was evaluated on a sample of blind and sighted students on a schoolbook text and the results showed that the visual writing signals significantly improved the participants’ understanding and memorizing of information which were in the text as opposed to the acoustic version of the text of the control group, where there was no auditory representation of visual writing signals.
Keywords:
Text to Speech, Expressive speech synthesis, Visual Writing Signals, Accessibility of Textbooks
