1. Neural decoding of Visual Imagery during sleep PRESENTED BY: Sandesh Chopade Kaviti Sai Saurab T. Horikawa, M. Tamaki et al.

2. The ‘unreal’ reality Dreaming is a highly subjective experience. Research linking physiological states with dreams. Activation in sensorimotor cortex due to dream movements – lucid dreaming How is visual content of dreams represented in brain activity?

3. Logical workflow : Training > Map onto neural signals > Evaluate spontaneous data with maps Stimulus – induced brain activity > machine learning Goal : Identifying natural images from human brain activity

4. The current Study Hypothesis: Contents of visual imagery during sleep are represented at least partly by visual cortical activity patterns shared by stimulus representation. Objective: Training decoders on brain activity to identify and resolve visual imagery in dreams. Focus on visual imagery during hypnogogic phase.

5. The experiment Three subjects were chosen for the study Made to sleep in an fMRI machine along with other polysomnography apparatus like EEG,EOG etc., Subjects woken when an EEG signature was detected. Asked to give a verbal report freely describing their visual experience before waking up.

6. Repeated this procedure to attain at least 200 awakenings for each participant. Words describing visual objects or scenes manually extracted and mapped to WordNet (lexical database).

7. The fMRI data obtained before each awakening were labeled with a visual content vector. Images were taken from ImageNet (image database) for decoder training. The ML module was trained on fMRI data measured while each person viewed Web images for each base synset.

8. Next, a multilabel decoder was trained which allotted a score based on the probability of the presence of a base synset in a particular sample. First, a binary classifier trained using pairs of synsets. Base synsets that appeared in atleast 10 reports (without co-occurrence) were chosen.

9. The mean decoding accuracy was 60.0% which was significantly higher than that of the label-shuffled decoders Synset pairs that produced content-specific patterns had higher accuracies(mean = 70%) Results

10. In the multilabel decoding, the synset detector provided a continuous score indicating how likely the synset is to be present in each report. ROC curves were plotted by shifting the detection threshold of output scores.

11. 18 out of the total 60 synsets were detected with above-chance levels. Decoding performance for individual synsets grouped into metacategories was even better.

12. Further Analysis The output scores (score vector) were used to identify the true visual content vector among a variable number of candidates (true vector + random vectors). The output scores (score vector) were used to identify the true visual content vector among a variable number of candidates (true vector + random vectors) The performance exceeded chance level across all set sizes.

13. The same analysis was performed with extended visual content vectors in which unreported synsets having a high co-occurrence with reported synsets (top 15% conditional probability) were assumed to be present. The results showed that extended visual content vectors were better identified, suggesting that multilabel decoding outputs may represent both reported and unreported contents.

14. Conclusion The results suggest that the principle of perceptual equivalence generalizes to spontaneously generated visual experience during sleep. Similar techniques may be extended to study REM sleep also. High level semantic decoding has been demonstrated here. Lower level decoding in different areas of the brain still has to be studied.

15. "Well, there were persons, about 3 persons, inside some sort of hall. There was a male, a female, and maybe like a child. Ah, it was like a boy, a girl, and a mother. I don't think that there was any color."

16. THANK YOU!