Neo4j cluster with HA Proxy

We have a Neo4j cluster of 6 machines and we are using HA Proxy.
If 5 slaves get stopped and the master remains alone, he stops acting (if you check if it is a master or a slave it returns 404 since he is not both) as a clustered machine and HA Proxy cannot forward the connections to it (since we are using authentication in haproxy settings that connect it as a master using basic authentication).

Also if we restart the remaining machine it cannot start since its not in a cluster anymore and there is a need to switch it to SINGLE so it would work.
Are there any practices that can help us handle this situations with less effort ?

Neo4j version 3.4.1
P.S. It's on private cloud.