Alright, it's a cool shot, and one that is fairly complicated, but as you said it does have a few issues.
First, the trench is about half as deep as it should be, but you may be running out of bricks.
Second, the speed of the camera pan causes it seem jerky, but that's a combination of the frame-rate, the details of the wall attracting too much attention and the speed. I would go a bit faster, and maybe think about adding a bit of motion blur. The walls are much more blurry in the film, while these are sharp and in focus bringing attention away from the X-wing. The X-wings movements also wiggle back and forth a bit.
BUT, those jerky motions and wiggling back resemble the slightly imperfect look of the original effects, and are still good for stop-motion. Overall I like it, and adding motion blur to those walls should fix most of the problems with this.