Any way to set the buttons on the stick to the ship commands in game or should I just do it manually through linux?

EDIT: should note I am playing the 1995 collector's edition version.

You mean the DOS version?
Most certainly not, I use the mapping software on my stick to map most of the buttons for that game.

Which stick are you using?
If it's Logitech, Thrustmaster or CH Products it should have its own mapping software (saitek too, but they tell me it's kinda rubbish) bar a few exceptions for thrustmaster.

logitech extreme 3D pro.

the trigger works like normal also the button next to my thumb but the others don't register at all

It's normal, at the time gameport joysticks didn't support more than three axes and four buttons and most games only supported two and two, Tie Fighter is no exception, premium manifacturers like Thrusmatser or CH Products added a keyboard passthrough to their more expensive products let people use the extra buttons.
Only with windows, USB and directX was possible to have a more thorough joystick support in-game without using ps/2 port passthroughs and even then many of the first low-tier USB sticks still maintained limited layouts (I had a Saitek 4-buttons 3-axis usb myself until last year).

In short, yes, you will have to map the other buttons through linux, unfortunately the Logitech Gaming Software only supports windows.

You can also map joystick buttons and axes to keyboard keys with dosbox's internal remapper (CTRL+F1).