A major Buddhist temple has withdrawn from plans to host Japan’s opening stage of the Olympic torch relay.