John, you're just replacing one handicapping system with another more complicated one. There maybe no numbers involved, but it's still a form of handicapping.
The guys I play with are very competitive and love a bet, the handicaps range from 0-36, you would not believe the methods we've come up with to ensure security in fairness and confidence in laying out some bets. For a period we replaced the number deduction system with "free throws" and "free kicks". From memory, the 36 marker was allowed 27 throws and 9 kicks to be taken whenever they choose, the betting games we play determined a strategy of when it was most advantageous to take the throws and kicks. The system was surprisingly accurate. Try it, if you play off a 13 hcp, you will shoot around par if you are allowed to throw the ball 13 times (that are not included as strokes).