This room is located in a building just inside one of the dodgier sectors of the city.
You can see through the window from the street, but the door to the dentist’s surgery below cannot be opened. You can use the jet boots to get up there and take a look inside. However, there’s no way to smash the window and get through.
There may be other random rooms like this elsewhere. In a previous post for this blog, I described an unreachable power-up located in the grounds of one of the Gotham City mansions, so this kind of thing is not unknown in the game.