The fact that the prior owner did not have action taken against him does not legalize these activities or change the law in any way. Absolutely uniform enforcement does not exists for any law anywhere in the land. Question- what effort did you make for the permits, and how were you obstructed if that’s your argument.