Singing as the optimal educational medium - Part II Neuroscience

I want to analyze songs and signing from a neuroscience perspective and show that it has all the qualities to make an optimal educational medium. Songs has several unique cognitive aspects. The first is its sequence-memory aspect and by this I mean that once you know a song, even hearing the first few notes, you can recite the song completely. Even more than that, if you hear a note in the middle of a song, you can continue with no problem reciting it. However, it is almost impossible to recite the song backward in time, i.e. from start to finish. In order to do that, you need to re-sing the song from the beginning each time. The second aspect is song's catchiness and by this I mean our extraordinary ability to remember songs from first hearing. This field has been studied, but I'm not certain as to the degree we fully understand why (more on this below). Songs, in this aspect, are very different from poems or any other non-musical verbal medium. It is the rhythm and fluency of the song that makes them so memorable, in my opinion. I propose a project to find out the exact border between songs and poems, in that how much rhythm is required for us to learn and remember them.

The next aspect is the neurobiological one. Memories are created and recalled by patterns of activations of neuronal cell firing. There is a vast network of neurons that form an intricate and complex connectivity pattern and the sequence of activation determines the memory being recalled. Note that a specific neuron spiking may be enough to cause a recall, or other behavioral outcomes, but the memory itself is stored across a plethora of neurons. I believe that singing has that unique aspect of optimal recall due to their rhythm and fluency, mentioned above. I believe that opposed to a story or a poem, the temporal aspect of songs somehow resonate with the neuronal activation patterns. Furthermore, as opposed to music, which may also be unique in the former aspect, we have quite a large chunk of brain associated with language. I believe that this unique combination of a verbal and a rhythmic component is what makes songs so memorable. I propose a project in which activation of neuronal cells resonate with their inherent network activation and show that in the human auditory cortex and language areas, these activations occur only, or mostly, during singing.

To sum up the neuroscience aspect of singing as the optimal educational medium, I believe that for some bizarre reason, our brain is wired such that songs are optimal in the encoding and retrieving of memory. This grants us the opportunity to use them as an educational tool, if memory is required.

Singing as the optimal educational medium - Part I Engineering

Audio signal can be presented as what is called a spectrogram, which is a way to show the distribution of audio frequencies over time. Along the x-axis is time, and y-axis is frequency, where color codes for intensity. Think of it as the equalizer bars you have while playing your favorite mp3, smeared over a page. This representation is very common in speech recognition and other audio-based analysis.

Music

Speech

Now think of different types of audio signals that humans use to convey messages. The first is speech. Speech is compose of short utterances, called words (dah!). However, when viewed in the spectrogram they look like vertical stripes, i.e. short in time, but complex in frequency. On the other hand, music without words are the complete opposite. They are long in time but narrow in frequency, since they represent something closer to "pure tones". Singing, i.e. speech with melody, is just in the middle.

What all of this have to do with "optimal" and "education"? In signal processing there is a notion called a "compact set", which is the minimal set of features you need to have in order to convey information. In other words, with these features you can code the most information and then send it to other people. It is "compact" because just a few features can generate a lot of information. It is "optimal" in the sense that you cannot have the same number of different features that can convey more information; any change in the features will result in loss of information. What are you talking about???

The project I'm suggesting is to analyze singing in the context of a compact set of auditory human information. If information is coded via the spectrogram, i.e. the information content is frequency over time, then I believe that singing, which contains both long sequences and high frequency content, can serve as the optimal compact set to convey that information. I think that the reason is that complex frequency conveys meaning, e.g. words, whereas the long temporal domain conveys sentiment, e.g. emotion. In human communication both are important and I believe it can be quantified.

One such suggested experiment is to have a questionnaire on the information conveyed via several types of communications, for example: "what did this person think?" "what did this person feel?" "what did this person try to convey to you?" The three conditions will be: (i) speaking; (ii) musical instrument and (iii) singing. Each condition will be of equal duration. My hypothesis is that the best answers will be with singing.

This type of experiments suggest that the optimal way to communicate is not talking, but rather singing. Can this be used in education? Wait for the next blogpost…

Creating mythical creatures

Today's genetic manipulations and understanding of embryonic development is quite astounding. You can actually write the genetic code on your computer, send it over the web and get to your home/lab a vial with the DNA you've written. On the other end, our understanding of biological genetic code is increasing. Scientists know the exact codes of several thousands of proteins, and most of their functions.

A more difficult field is embryonic development, in which the genetic code unfolds into creating the beings that are then born. It is a delicate balance between the genetic code, transcription factors that regulate the production of proteins, differentiation of cells and local and global chemical gradients. Nevertheless, a lot is known about the sequence of organ developments and how to influence them.

The last piece of the puzzle is genetic manipulations and creation of chimeras, in which a gene sequence from one species is inserted into another. You probably all know of the glowing mouse, where genes from a glowing bacteria has been inserted to the mouse genome and it glows in the dark.

Putting it all together, I think it is time to start bringing mythical creatures into life. While dragons are still out of the question, minor variations can be made. The first such animal that is the easiest to create, from my understanding, is the unicorn. The only unique thing about it is a single horn in the forehead. It has been shown that grafting an organ is possible, but I'm talking about a genetic/embryonic manipulation, such that a unicorn will be born. I believe it should be made similar to the rhinoceros horn, since it can be made pointy and doesn't require a bony structure. Hence, the genetic manipulation should be not major. Furthermore, the structure of the horn is not dissimilar from that of the horse's hooves, hence there is no need to insert new proteins into the pool, only regulating its expression. I admit, the way there is not easy in the design, implementation and regulation, but… a unicorn.

Other mythical creatures are probably harder, although I'll bet that Cerberus should not be that hard. It's simply having a siamese-triplet with a single body. Directly controlling this, so as to make it reproducible is probably not easy, but if Harry Potter has one, why shouldn't we? Pegasus is altogether another problem, since while I believe it would not be too difficult to have a horse with wings, to have a flying horse is, to the best of my knowledge, against the laws of physics. The same is true with a fire-breathing dragon, but that is for another post…

Why do this? Except for scientific curiosity, engineering challenge and pure geekiness, the market of such creatures could be huge. Which zoo would pass an opportunity to show its unicorn? Think of the extra marketing a circus can have with a Cerberus at its gate. The opportunities are endless. The only thing remaining is the curios scientist/engineer/geek with enough money to start this crazy project. Good luck!

Laser Tombstone Preserver

If you walk in old graveyards, the thing you notice most is that the words on the tombstones are illegible. The older the tombstone, the more faint they are. I think this is a shame, since those names and inscriptions are history, and while they are "engraved in stone", even stone deteriorates. Other old buildings or monuments share the same fate and it should be rectified. Furthermore, even new tombstones will, someday, be etched from memory and recognition.

I propose a device that will maintain the etched inscriptions on stone monuments, such as tombstones. It is composed of three components, namely, scanner, recognizer and etcher. The scanner uses modern 3d laser scanning techniques to detect the current inscription on the tombstone. It generates a complete digital 3d scan. This is then passed to the recognizer, in an attempt to use state-of-the-art deciphering tools to reconstruct the inscription. One can use more sophisticated sources of information, such as cross-referencing GPS of the tombstone, with historical records of people from that area, to have a better chance of finding the correct information. Finally, the etcher, which can be either a powerful laser, or any stone-etching device, emphasize the faint inscriptions so that it is more readable and more apparent. To sum up, the device is a portable one, which is passed over the tombstone and re-etches the inscription.

This device, once available, creates a new job, namely, Tombstone Preserver, which is a person going around the country, from one graveyard to the next, and using the device to highlight inscriptions. Furthermore, the deciphered scripts can then be uploaded to a cloud-based database for research and historical records.

I believe this project can revive a lot of lost history and has great personal value to many people around the world. The technology is available, all is needed is a person to make it. Interested?

Gestures in Chat rooms

Non-verbal communications, such as facial expressions and hand gestures, have a drastic effect on the understanding and social effect a conversation has. There are numerous research projects showing that from psychological and sociological perspectives. However, the current common form of communication is text, e.g. sms, chat rooms, WhatsApp, etc. These lack almost all forms of face-to-face communications, such as tone of voice, gestures and facial expressions. While people use txting, to some extent, to reduce the complexity of the communication, the medium can be enhanced if those are present. While video chats solve most of the problems, texting will not go away once video streaming becomes more accessible. There is the allure of not actually be seen on the other side.

I suggest a research project to investigate the incorporation of non-verbal communications into text-based media. The emoticons were obviously the first step, as they nicely replace facial expressions, where J substitute and smile on the face, while other more complex emoticons can replace others. Two axis of extensions are suggested, namely, including gestures and automatization of inclusions.

How to include gestures? I believe a new form of emoticons can be incorporated. It has been shown that gestures actually relate to the physical reality, where gesturing the word "all" encompasses a large space and gesturing "never mind" performs a discarding motion. I propose creating a hand-based animations for text-based media, very similar to complex emoticons. But now, instead of a face substituting facial expression, there will be hands substituting gestures. One can create many such gesturecons, to include all kinds of meaning. See http://en.wikipedia.org/wiki/List_of_gestures for more examples. The research is the applicability and usage of these gesturecons by chatters: will they use it? How much? In which situations? What are the favorite gesturcons?

The next extension to emoticons is the automatic inclusion of them. Nowadays, facial recognition hardware and software are readily available, e.g. Kinect. There are known algorithms to track the face and also recognize facial expressions such as a smile, a laugh and other expression. I suggest to integrate these automatic recognition into text-related media, such as Facebook, WhatsApp, etc. In other words, when someone sends you a funny picture, and you actually laugh, it will automatically detect it and send an LOL. Research questions: will people like it, or do they like to control their emoticons? Do people send more "fake" emoticons than real ones?

Gestures are also readily detected. Using Kinect or similar devices, there are already algorithms out there to detect gestures. However, there is a crux. When you're in a text-based medium, your hands are occupied typing and you can't really gesture anything. There are two approaches to this problem: the first is that when dictation will become prevalent, such that you speak to text, you can at the same time gesture to gesturecons. The second is creating a whole new field of "typing gestures", e.g. when you lift your hand in exasperation, a gesturecon will be apparent; when you knuckle your fingers, the appropriate gesturecon will be inserted, etc.

Obviously, there is much more to be done in this project, but that's the fun of it, isn't it?