Peel and cut the potatoes, place in a bowl with the garlic, 24 Tbs. olive oil, rosemary and salt and pepper. Toss to coat them uniformly.
Place the potatoes in a baking sheet and cook in the preheated ovens (around 350 degrees F) for around 30 minutes, turning them once or twice.
Remove the baking sheet from the oven, move the potatoes around to make space for the fish. Add the fish, olives, capers and 2 Tbs. of olive oil.
Cook for 12/18 more minutes, until the fish is opaque and flakes easily (internal temperature of 145 degrees F.)
Serve with lemon wedges or raw olive oil to taste.