Can't drag multiple elements with mouse

I have multiple elements selected. I try to drag everything with the mouse to reposition, but it only drags one of the elements (the one the mouse is over). I can only move everything using keyboard arrows. Is this by design?