This is probably because after switching on the Xbox, the sound is also taken from the Xbox. Overwrites the PC, so to speak. Since I can't test it myself, I recommend removing the optical from the Base Station when using the PC as a workaround.
Greetings