UNIVERSITY PARK, Pa. — Anyone who has listened to jazz artists weave in and out of improvised solos can sense that the musicians share a secret language.
Even jazz greats sense there is a secret behind this musical conversation. Jazz trumpeter Wynton Marsalis said, “In jazz, improvisation isn't a matter of just making any ol' thing up. Jazz, like any language, has its own grammar and vocabulary.”
Now, a team of Penn State-led researchers are using the Institute for Computational and Data Sciences’ (ICDS) Roar supercomputer to try to decipher the grammar and vocabulary the brain uses to allow musicians to spin up new musical ideas on the fly.
The key might be in how the brain eases into the flow of a creative activity, according to Hannah Merseal, a graduate student in cognitive science and a member of the Cognitive Neuroscience of Creativity Lab. Merseal, a lifelong musician who plays clarinet and once led a jazz-funk group, first became intrigued with the link between the brain and creativity after her own experience on stage.
While musicians make it look easy, their brains are busy directing a combo of mental activities, from activating memory to engaging in motor control and attention, all under the watchful eyes — and critical ears — of an audience, said Merseal.
To manage cognitive load, the researchers suggest that jazz musicians may start an improvised solo by resorting first to simple, trusty musical phrases. Later in the solo, they build up into more complex passages. The scientists refer to this as “easy-first” and say it serves as a good roadmap to how the brain approaches other creative endeavors, such as speaking and writing.
“We’ve largely been thinking about improvisation as something called a hierarchical behavior,” she said. “This is something that occurs in real time and is organized so that smaller subunits fall under larger units, which is also kind of the way that language works. We have a sentence and then smaller chunks of the sentence underneath that.”
Musicians may also rely on “chunks” of music, which may be another way the brain handles that considerable cognitive load during the solo.
“When we think about improvising, we're largely thinking about players taking smaller chunks of music -- such as musical notes that they have stored in memory,” Merseal said. “They’re chunking up that information in ways that are easy to store and rearrange.”
Research into creative endeavors, not just music, has offered a glimpse into the role memory plays in creativity and, in particular, how the brain uses networks of associations to fuel creativity, according to Roger Beaty, assistant professor of psychology and principal investigator of Cognitive Neuroscience of Creativity Lab. He said that scientific understanding has evolved over time.
“Our memories might organize categories of things by clustering them together and we can model this as a network,” said Beaty. “Earlier, there have been investigations into modeling of networks and semantic memory. More recently, we, and some of our colleagues, have been trying to test this out using the computational approaches of network science.”
Jazz Greats Analyzed
To explore these musical networks, the researchers analyzed a database of solos from the Weimar Jazz Database that features some of the jazz world’s greatest improvisers, such as saxophonists, Charlie “Bird” Parker and John Coltrane, and trumpet legend, Miles Davis. In an initial analysis, the researchers investigated the complexity of sequences over the course of each musical phrase and found that the level of complexity increased over the course of most phrases, similar to easy-first structures found in language.
The team then broke these solos down into five note sequences to build a large network with all of the notes used by the artists in the database. In an experiment, the researchers asked participants to rate the relatedness of musical ideas, finding that these ratings corresponded to where sequences were located in the network. These findings provide an initial glimpse into how improvised melodies may be stored in memory.
Analyzing this large volume of data would have taken weeks to run on a standard computer: the research team modeled a massive network containing thousands of nodes, requiring extensive computer memory and processing power. Therefore, the team relied on Penn State’s Roar supercomputer.
According to Beaty, network science and supercomputers also give the researchers the ability to explore other areas of creativity. For example, the lab takes a similar approach in probing for signs of creativity among writers through a concept called “semantic distance,” which is a lot like a word association game.
“One of the common ways of measuring creativity that we use is giving people word association contests, and for those who can come up with distant associations, we might consider them to be more creative,” said Beaty.
Supercomputers play a role on facilitating this research, too.
“To do this, we need some huge collections of texts and compute how often words co-occur together, for example,” said Beaty. “And because it's such a vast collection of texts, we need supercomputers run through that data really quickly and identify whether two concepts are close to each other and ones that are farther away. The farther away ones are more novel and therefore more creative.”
The team built a website, SemDis, based on this work that is powered by ROAR, allowing other scientists and educators to explore research into creativity on their own.
Next Steps
According to Merseal, the work into musical improvisation is at the proof-of-concept stage. In 2022, she plans to study the actual brains of musicians as they improvise.
The team will use neuroimaging -- or brain scanning -- technology, such as magnetic resonance imaging -- MRI -- machines, to watch how the brains of musicians react as they play. The technology may offer a glimpse into the role that differing styles and levels of musicianship play in improvising, said Merseal.
“I'm really excited to do some targeted sampling of beginning improvisers and expert improvisers, as well as classical musicians so we can really kind of get into it and see how these networks are different and what strategies they're using,” said Merseal.
You can learn more about the Roar supercomputer and the research it supports here.