In the U.S., according to Dr. Robert Smith of Buffalo, as printed in the June 29, 1949 issue of Golf World:
1. Golfing (monthly, 1894-1898). (A different Golfing, run by Herb Graffis, started in 1933.)
1A. The Golfer (monthly, 1894-October 1903). Which came first? Smith didn't indicate. I'd think the USGA would know.
3. Golf (monthly, September 1897-May 1917). Two like-named magazines have followed, with Golf III still in existence.
4. Western Golfer (monthly, May 1899-May 1902, when it was renamed The Golfer's Magazine, and continued to September 1931). Published in Chicago.
5. American Golf (monthly, March 1900-April 1902). (The similarly named American Golfer started in November 1908.)
As for across the pond, others will have to respond.