This reminds me of my musically gifted friend Ryan.
He has synesthesia (where hearing sounds causes him to see colors). Most of his associations were really unexpected. Like- I would think that shrill sounds like Joanna Newsome's voice would be pink or yellow, but he said it was a deep purple and red. I know the notes that are sung make a difference too though.
There are modern composers that make music compilations based entirely on colors, but I haven't found any samples to listen to. I have got to learn more about this..
(These designs are from Hoagy Houghton)