H.B. Martin's 1936 book Fifty Years of American Golf contains a listing of courses in existence in the summer of 1895 and notes the year the club was born or first adopted golf. Although not a contemporaneous source, the works proximity to the time period in question certainly gives it some veracity. That being said, Martin's claim that there were around 50 courses in existence by 1895 seems to undershoot the actual number.
1888 - St. Andrews
1889 - Tuxedo Club
1890 - Middlesboro GC, Newport GC, Hotel Champlain Course (Bluff Point)
1891 - Shinnecock Hills GC, Philadelphia CC
1892 - Chicago GC (Belmont), The Warren's Farm GC, Baltimore GC, Powelton GC
1893 - The Country Club, Essex County CC (MA), Swannanoa GC, Montclair GC, Chevy Chase Club
1894 - Weston GC, Santa Barbara GC, Town and Golf Club (Colorado Springs), CC of Lakewood, Albany CC, Otsego GC, New Brunswick GC, Tacoma GC, Fairfield GC, Riverside GC (CA), Jekyll Island GC, Morris County GC, Oyster Bay GC, Staatsburgh GC, apawamis GC, Baltusrol GC, Portland GC (OR), Erlington GC (Seattle), Knollwood GC (NY), Meadowbrook GC, Devon GC, Westbrook CC, Paterson GC, Richmond County CC, Norwich GC
1895 - Palmetto GC, hampden GC, Hoosic-Whisick GC, Wollaston GC, Seabright GC, Morristown Field Club, Thomasville GC (GA), Bloomington GC (IL), St. Louis CC, Concord GC, Stockbridge GC, Sadaquada GC, Westchester CC, Presidio Reservation, Burlingame GC, Brooklawn GC, New haven CC, Ridgefield GC, Waterbury GC, Dyker Meadow GC, Cincinnati CC, Saegkill GC, Haddon Field GC, Glen Ridge GC, Salem GC, Storm King GC, Watervliet Arsenal GC, Allegheny CC, Mt. Pleasant Field Club, Staten Island GC, Van Cortlandt Park Municipal Links, Poland Spring GC, Queens County GC, Point Judith GC
Some additional information taken from Fifty Years of American Golf:
-There were 1,040 courses by 1900,
-By 1900, there was a golf course in every state of the union
Martin also listed state by state numbers, as follows:
NY - 165
MA - 157
PA - 75
NJ - 63
CT - 57
IL - 57
NH - 44
CA - 43
OH - 36
ME - 33
MI - 29
WI - 28
RI - 26
VT - 20
IA - 20
FL - 17
MN - 16
IN - 16
MO - 14
MD - 13
VA - 11
NC - 9
CO - 9
GA - 8
KY - 7
TX - 5
SC - 5
KS - 4
ND - 3
AL - 3
AR - 2
MS - 2
DE - 2
DC - 2
SD - 2
LA - 1
AZ - 1
NM - 1
WY - 1
OR - 1
NV - 1
UT - 1
OK - 1