You should hold the buckle with one hand and open the zipper with the other hand, which will make it easier and smoother.
eBay