I assume you're playing on the Duke level, which is the lowest level that allows you to design your own castle... If you're playing higher, that could explain your combat woes.
The 6 archers and 6 infantry you're placing aren't individuals, they're units. If you have, say, 60 archers in your employ, and in your military tab you have it set to 10 units of archers, and you place 6 at the start of combat, then you're only actually sending 36 archers to battle! Which sucks, obviously.
Another important note, make sure you're actually employing archers and infantry, and not merely putting up a "Help Wanted" sign in the window. On your hiring/firing menu for military, you should see something like 0/0, or 24/60 archers and a similar display for infantry. The number before the slash is how many are actually working for you, and the number after is how many you're willing to hire. How quickly the number before the slash approaches the number after the slash depends on how much you're paying. Obviously, if you have 6/100 archers by the time your first combat rolls around, you're screwed.
Getting combat-ready:
* Hire as many archers/infantry as you can support and afford. The more castle-pieces you've placed (uncompleted pieces DO count!), the more you can hire. So, set up at least the entire exterior of your castle immediately, don't do it piecemeal.
* Hire your military as soon as your castle-pieces are placed. That way, they have time to actually start working for you before the rest combat.
* Again, hire as much military as you can afford. I always hire at least 200 of each. If you have to skimp, DON'T skimp evenly. Skimp on infantry. You DO need infantry, but if you can only hire 300 military, make it 200 archers and 100 infantry.
* Dig yourself a moat. Make sure you are able to do this by placing one corner of your castle as near to water as possible. If you're further away, it will not work. Do this as soon as you're hired military.
Preparing for battle:
* Place your archers on towers, INSIDE the moat. Place them on the towers NEAREST the enemy's approach. Your archers have a limited attack range, so if you keep them in the interior (or worse, rear) towers, they're worthless.
* How many units you split your archers into is important. If you have a moat and adequate military, you should split into fewer units. Maybe 4? That way, they can all be packed into the best towers. If you've been caught with your pants down, use 10 units, so that they have to destroy the most towers kill all your archers. Hopefully, this will give your archers time to kill them before you lose.
* Infantry are stupid as hell. Don't put them behind castle pieces, because they will get stuck and never engage the enemy.
* Infantry are stupid as hell, and will walk miles afield to engage the enemy. They will walk right out of your archers' range, and therefore fight without their support. This is bad. You want your archers shooting the enemy the entire time your infantry is holding them up. Therefore, place your infantry farther away from the enemy's direction of approach than you've placed your archers. This way, the enemy will reach archer-range by the time your infantry engages.
* It's better to split your infantry into more units. If you have 4 units of infantry, you can only tie up a maximum of 4 units of enemies. The rest beeline for your castle and start knocking down your archers' towers. Not good. So, use 10 infantry. There's no advantage to using fewer units, since 2 or 3 units of infantry can team up on a single unit of enemies, if need be.
Cleaning up after battle:
* Okay, so you've lost some people, I bet. Probably more infantry than archers. Your infantry should be somewhat higher-paid in order to get them replaced faster when they are inevitably killed.
* Did the enemy reach your moat and start filling it in? Re-dig it. Just issue the dig-moat command again from the military menu.
* Did the enemy reach your castle, and destroy some pieces? Those will need to be replaced.
Hope that helps. Good luck.