Change side pane color in Rhythmbox?

Yes, and it's not much different from changing the color of the Nautilus-Elementary sidebar.

Add the following to your ~/.gtkrc-2.0 file (create one if it doesn't exist):

style "rb-sidebar"
{
    GtkTreeView::even_row_color   = "#F2F1F0"

}
widget_class  "*RBSourceList*"  style "rb-sidebar"

Then switch to a different theme and then back to your old one.

As you can see in the screenshot below, there is a border around the sidebar in Rhythmbox unlike in Nautilus-Elementary, but the color matching does improve how Rhythmbox looks, in my opinion.