But Peter Herbert, chairman of the Society of Black Lawyers, believes Tottenham supporters must take more responsibility in tackling the issue.
"We're not blaming Spurs fans," he told CNN World Sport.
"But they have to take responsibility. Those same fans would not dare use that word on a Saturday afternoon outside the ground. You cannot have an area of London which is ring fenced for racism or anti-Semitism."
In recent years, the FA has shown its sensitivity to the concerns of the English football's Jewish community.
The England national team paid a visit to the World War II concentration camp Auschwitz I in Berkanau ahead of the Euro 2012 tournament in Poland and Ukraine last year.
In October, England manager Roy Hodgson will open the "Four Four Jew" exhibition at the Jewish Museum London.
The exhibition celebrates the Jewish community's contribution to British football.