I see that dilemma clearly. I only ever found workplace happiness once I started working alone in a small business not involving other people. I had around 90 jobs over 15 professions e.g. barman, bouncer, railway, Telstra etc. My mania was out of control.
However, there isn't many professions that don't have people contact. If you can change to a more appropriate field then do so IMO.