BLaRgXrvbX: In that case, How would I go about completely removing it? As I said before, there's a .exe that refuses to leave.
First of all, I have never used this Overhaul mod and don't know how to remove it if there is no uninstall procedure. I just thought about what I would do if I were in your situation.
Method 1: Simply leave the Overhaul mod in place, but de-activate it. To do this, launch Morrowind and click on "Data Files". On the left hand side, you will see a column labelled "TES3 Files". Somewhere in that column you will find an entry "Morrowind Overhaul" or similar with a check box ticked. Untick that checkbox, click OK and you're done.
This is a simple, but not nice method, because you still have the Overhaul files on your hard disk (as garbage), but at least, the files won't do any harm.
Method 2
If I wanted to get rid of the Overhaul mod and get my clean system back, I guess I would look for an Overhaul entry in the program list. If there is none, you can delete the mod files safely (a mod folder and an .ESP file, I guess). Otherwise I would double-click that entry and run the uninstall procedure.
If you don't know exactly what to delete or if Windows doesn't let you delete everything (I guess because you installed the game in a standard program files folder), then uninstall the whole game, either from the program list or using the GOG uninstall procedure in the game folder. If you choose to delete everything (the uninstall procedure offers you that option), then the Morrowind folder should disappear when the procedure is complete.
If there is still something left, which you can't delete, then I would simply leave it. It's garbage, but what choice do you have?
Then re-install the game, but do yourself the favour not to install it to a program files folder in C:, but to a different drive, or to a folder like C:\Games or similar.
In both cases you will have a Morrowind installation without any mods. As a person who has played Morrowind from the beginning, when it was released, and has spent 100's of hours with this game and knows probably every game issue you may run into during your first steps, I'm strongly recommending not to use ANY mods in the beginning. The game is perfectly playable and fun without. After you've played several hours and if you feel your game experience should be improved, go through the mod threads on this forum, and choose one or several mods deliberately and only those which you think could solve the shortcoming you perceive. And before you do this, familiarize yourself with the methods to install AND uninstall those mods - otherwise you're likely to end up in the siuation you're in today. Don't install a mod just because others tell you you MUST have it and can't play Morrowind without. It's not true: You don't need any mods to play Morrowind.
But, yes, some mods may help you to enjoy the game even better. Only, to find the ones which do that for YOU, is probably quite difficult, due to the high number (and varying quality!) of those mods. And how do you know what is likely to improve your gaming experience as long as you are not at least somewhat familiar with the game?
Finally: Morrowind has a number of bugs and shortcomings indeed, and the Community Patch does solve many of them, but not one of them is a real game or fun killer - you're unlikely to notice the difference between Morrowind with or without that patch before you have some experience with the game. I advise strongly to keep things as simple as possible.
Have fun!