dtgreene: If you're talking DOS versions (the ones GOG sells), there isn't supposed to be any music until Ultima 6.

Ultima 3, 4, and 5 had music on other platforms (some of which is quite good, actually), but the soundtrack is unfortuately completely absent in the DOS versions; this is due to the hardware limitations of the time.

Ultima 1 and 2 never had any music in any version released in English; apparently some Japanese releases did have music, though not from the same composer. (Console versions of Ultima 3, 4 (NES), and 5 also had different soundtracks; I personally prefer the computer version soundtracks, but the NES soundtracks for Ultima 3 and 4 are at least serviceable.)

There exist unofficial upgrade patches for DOS Ultima 3-5 that, among other things, add music from the other computer ports to the DOS version, if you want to play these versions with music.

BananaJane: Thank you very much :D
I got the music patches working!

amccour: Some extra advice on this, but if you want the music to sound better, I'd suggest installing something like VirtualMidiSynth and either the SGM-V2.01 or Timbres of Heaven soundfonts (I don't know why but the U5 midi files had a lot of missing instruments with some of the other soundfonts I tried >:T)

BananaJane: Does that work with other games?
3-4-5 sound amazing but Ultima Underworld's midi music is very irritating

amccour: Ultima Underworld doesn't support general midi afaik. I think by default it's using Adlib music. If you set it to use Roland MT-32, that's a different midi format from general midi so it will not be playing the right instruments unless you install an MT-32 emulator and configure DOSBox to use that.