I generally wouldn't wear a logo of a course where I haven't at least been on the grounds or hadn't played it.
On the other hand, lots of sports fans wear their team's jersey or hats without ever having actually attended a game in the stadium. So, maybe wearing logos of courses you've not seen or played is kind of like that. Maybe you're just a fan of the club.
To each their own. Do what makes you happy.