The Instagram app accepts only vertical/portrait video format at aspect ratio of 9:16.
The native supported resolutions are either 504x896 or 396x704 - both are 9:16 aspect ratio.
We can stream other 9:16 resolutions like 608x1080 or 720x1280.
Regardless of the streaming resolution, the Instagram app zoom in the Live video for viewers. The zoom-in ratio is around 7.5%.
When working with streaming software like OBS or Wirecast, it's recommended to set your scene to consume less than the full frame. See samples below.
Here's how the video looks like in the Loola Studio:
And here's how it looks like in Instagram: