I'm not clear how a “day of rest” leads to prohibiting leisure activities. You could coherently prohibit professional golfers from playing on Sundays on that basis, but not people doing it for fun.