"Proszę powtórzyć" is rather a formal phrase, followed by more polite-sounding "Mógłby Pan powtórzyć? / Mogłaby Pani powtórzyć?".
When talking to friends or close family (of similar age or younger), it's perfectly normal to ask "Jeszcze raz?" or just to say "Powtórz. / Powtórz proszę."
The most common I use and hear are variations of "Możesz powtórzyć?"
Actually, I'd say quite the opposite. It does not necessarily communicate disbelief at all, though in some situations, it can, depending on context and tone of voice. And it's acceptable even in the most formal situations. I don't find it relaxed or informal at all. In the informal situation you depict, I would be more likely to say "Wątpię.","To chyba jakiś żart?", "Żartujesz!", "Daj już spokój!", "Co ty bredzisz?", or even "Ale pierdoły.", depending on how strong I want to express it.