Encoding audio with MP3

In Virtual Dub go to the audio menu. Make sure it's not set to direct stream copy since all compression options are disabled then. If it's not set correctly, change it to full processing.

Now open Audio->Compression :

Select MP3, set it to the rate you want. I am using the maximum rate offered since I often use music inside my videos. For just ingame sounds you might also use a lower quality setting. Press OK to finish the audo configuration.

Note: Virtual Dubs uses CBR (Constant Bit Rate). As some said, Variable Bit Rate (VBR) would result in better quality but needs the use of several tools to encode the mp3s to VBR and then mux (multiplex/embed) the video and the audio streams into an a stream container like AVI. Various reasons make the AVI a bad choice for this though which could be solved with another container format but that's a topic on it's own and clearly a too big one for this guide.