Topit is right, and another thing to remember is to include interesting camera angles/movements. In-camera pans, shakes, turns and dollies really help sell the action, and turn an otherwise mediocre fight into something amazing.
I've done a number of fight scenes, and find that even tiny pans can make a clip look many times better.
To quote Fancypants:
Fancypants once wrote:"...Camera movement is great because it makes your film more dynamic and cinematic..."
And may I add "that goes double with fight scenes."
I recommend watching a lot of good sword fights, both Lego and real, with the sound off, that way you can concentrate on the exact movements involved. Examining them frame-by-frame is also great way to learn more.
Don't forget to exaggerate the reaction. For example, if Ninja A swings his sword down and hits Ninja B's sword, then Ninja B's sword should drop with the impact, and then come back up a tiny bit as Nina B recovers from the impact and attempts to fight back. And don't forget easing, that's a major part of the fight, and can easily make or break a scene.
Use the whole body, not just the arms/hands. A blow will appear to have more force if the recipients whole body reacts. A big problem with most fight scenes is that people leave the body fairly stiff, and only use the arms.
The Force Unleashed is probably the most referred-to brickfilm sword fight, but there's a reason for that. It's a great example of what doing everything right looks like.
One last thing, everybody does fight scenes differently, so find your style and stick with that. It'll taek a lot of trail and error, but the final product can be spectacular.