Corrected Crossing Wires

So it sounded good, right?

"My solution is rather backwards.  I ran audio cables from each of the sources on the TV (Tivo and cable box) to a 4 way AV splitter and then ran the OUT to the Tivo box with the cable's audio running into the Tivo.  In this case, the Tivo basically functions as a passthrough for the cable box's audio and the splitter lets me listen to either source while viewing it."

Yah, not so much.  It turns out that the Tivo is not a good pass through and it also delays the signal slightly as it caches it.

So here's the real solution and this makes even less sense than what I came up with before, but again it works :

from out cable box to out 4 way switch
from tivo out to tv s-video 1 in (the only one that is somewhat straightforward)
from tivo in to switch #2 in
from switch #3 in to tv component #1 in

did ya follow all that?  good, cuz I have no frigging idea why that works, so maybe you can explain it to me.

I also figured out how to run both at the same time by pushing in both of the buttons at the same time, effectively creating a powered split instead of a switch.