Rabbi Yoel Teitelbaum is known to have said visiting Kivrei Tzaddikim is only for people on a high level (like Tefillin, Eretz Yisrael… starting to see a pattern?), in “Al Hage’ulah” p. 166 (#108). Only those who can “connect” with the deceased should go.
N.B. This is not exactly the same reason given by the Maimonidean opposition to visiting gravesites…
[I don’t know if this was all a trick to stop people from going to Israel where most Kivrei Tzaddikim are located (and where they are kept up by the state).]
A recent column in Kedushas Tzion already pointed out how so much of Rabbi Teitelbaum’s statements run directly counter to all Jewish Mesorah, current and ancient, and this admonition against prayer is no exception. Nor did he bother making crucial distinctions in gravesite visitation.
But I wish to stress the aspect of cruelty here.
It’s like outspoken atheists who deliberately try to remove all living hope, especially from the terminally ill, the depressed, and the downtrodden, and replace it with “courage” these people simply cannot find within themselves (not on command, anyway).
P.S. What about the Arizal? See this.