Listening Effort in Cochlear Implant Users

General methods:

Speech recognition testing: The main experimental task is listening to speech in the free field in a sound-attenuated booth. Traditional speech recognition consists of hearing and repeating sentences, and these studies will vary on that general theme with modifications to suit the specific aims of the project. For example, some stimuli are repeated twice before verbal response, and some are questions and answers rather than declarative sentences. The investigators have developed the stimuli specifically for each experiment, always avoiding emotionally evocative stimuli since they can have large effects on pupil dilation.

Intelligibility: The investigators will track percent-correct scores for whole sentences and words within each sentence. The investigators also track the position of mistakes during the sentence (e.g. first word, last word), and whether the participant's response could feasibly be a sensible utterance, or whether it lacks any coherent meaning, because this factor has such a large effect on listening effort.

The investigators will test normal-hearing (NH, or TH Typical Hearing) listeners' perception of both regular speech and vocoded speech, allowing us to separately examine the influence of signal degradation separately from the chronic experience of hearing difficulty.

Pupillometry: During speech recognition tests, changes in the participant's pupil size will be monitored by an Eyelink eye tracker placed 50 cm from the eyes. Phasic pupil dilations are linked with experiment timing landmarks interpreted as changes in listening effort, based on numerous published studies.

Microsaccade tracking: small eye movements will be tracked concurrently with pupil size (described above) as an additional measurement of attentional effort.

Blink tracking: blinks will be tracked concurrently with pupil dilation and microsaccades (described above).

Experiment 1A: The investigators will test the ability to prepare extra effort in a study where the listener knows in advance whether an upcoming stimulus will be easier or harder to understand. Each trial begins with a cue of "easy" or "hard" displayed in text on the screen. Following a four-second delay to let the pupils return to baseline, the sentence is played. Pupil dilation is tracked throughout the entire trial, with the crucial differences being examined between the text signal and the onset of the speech, reflecting the listener's momentary preparation of extra effort. Importantly, the easy and hard trials are randomly intermixed rather than blocked, so that the elicited effort specifically reflects momentary preparation rather than a long-term disposition.

Experiment 2A: This task simulates a misperception of a word during a sentence, with an opportunity to hear a repetition of the sentence to clarify that word. In each trial, a sentence is played twice. In the first presentation, the sentence is either fully intact (normal), or has a single word replaced by noise. The sentences are constructed so that the word is essential to the meaning of the sentence, but cannot be recovered by context. In the second presentation, the sentence is always fully intact, enabling restoration of the missing word.

Experiment 2B: This task simulates a situation where a listener aims effort at specific points in time based on their listening needs. Participants will complete a standard sentence repetition task, but with stimuli where key information is placed at earlier or later parts of the trial, and which have different levels of importance to the task.

Experiment 3A: Judging answers to questions. In each trial, the participant hears a question (Q) and an answer (A) spoken by a different talker, as for a conversational dyad. The participant is prompted to respond with "yes" (the answer was correct) or "no" (the answer was not correct).The answer can be either correct, incorrect, or completely unexpected (nonsense), randomly varying trial to trial. The nonsense answers invite a moment of second-guessing the perception since they are so unexpected.

Experiment 3B: This is similar to experiment 3A, but instead of completely unexpected answers, the listener will hear an answer that is related to the question, inviting a reassessment of whether the question was heard correctly. For example, "What color is a basketball? Round" would invite the listener to momentarily doubt whether the question was understood correctly.

Experiment 3C: This experiment requires the listener to use prosody to determine whether a talker is continuing to speak or if they are done speaking. Effort devoted to listening will be tracked using a combination of reaction time and eye-tracking measures such as pupillometry.

Experiment 3D: The reliance on post-sentence effort will be examined in a behavioral study where listeners hear a continuous stream of sentences and can manually adjust the rate of speech (manipulated using the pitch-synchronous overlap-add PSOLA algorithm in Praat) until arriving at a comfortable pace.