Sadly, these things shrink more in length than in girth. So it makes shrinking something to fit you difficult. The bigger problem, if your sweater is wool, is that the process you would use to shrink fabric, water and heat, have a specific effect on wool. It turns it onto felt. This is why wool is so much better known for shrinkage than other fabrics, because not only is it shrinking, the fibers are matting together.
Now the option of taking it in at the sides is still open to you. I realize the sweater may be fully knitted and not have them, but it would be safer than trying to shrink it.