Neo4j Browser HTTPS connection in Kubernetes: login problems with Chrome

Hi all. I am Andreas from germany and I am new here.

I'm trying to get my login via Neo4j browser inside a Kubernetes cluster working and I experience some problems with Google Chrome browser. I think it might have to do with the SSL/TLS certificates which Neo4j is unable to load correctly.

The certificates I am using are valid and signed.

What does work:

  • Neo4j Web-UI login via Firefox and Safari

What does not work:

  • Google Chrome login

It seems like Google Chrome prevents the login on the client side. Neo4j error:

ServiceUnavailable: WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use.

Google dev console error:

WebSocket connection to 'wss://neo4j-bolt.domain.com:7687/' failed: WebSocket opening handshake was canceled

Output of the Neo4j Server in the Kubernetes cluster:

chown: changing ownership of '/var/lib/neo4j/certificates/https/..data': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https/private.key': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https/public.crt': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337/public.crt': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337/private.key': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337': Read-only file system
chown: changing ownership of '/var/lib/neo4j/certificates/https': Read-only file system
chmod: changing permissions of '/var/lib/neo4j/certificates/https': Read-only file system
chmod: changing permissions of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337': Read-only file system
chmod: changing permissions of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337/public.crt': Read-only file system
chmod: changing permissions of '/var/lib/neo4j/certificates/https/..2020_12_04_08_46_24.920196337/private.key': Read-only file system
Warning: Some files inside "/data" are not writable from inside container. Changing folder owner to neo4j.
Changed password for user 'neo4j'.
Directories in use:
  home:         /var/lib/neo4j
  config:       /var/lib/neo4j/conf
  logs:         /logs
  plugins:      /var/lib/neo4j/plugins
  import:       /var/lib/neo4j/import
  data:         /var/lib/neo4j/data
  certificates: /var/lib/neo4j/certificates
  run:          /var/lib/neo4j/run
Starting Neo4j.
2020-12-04 08:47:04.447+0000 INFO  Starting...
2020-12-04 08:47:09.767+0000 INFO  ======== Neo4j 4.1.1 ========
2020-12-04 08:47:12.756+0000 INFO  Performing postInitialization step for component 'security-users' with version 2 and status CURRENT
2020-12-04 08:47:12.757+0000 INFO  Updating the initial password in component 'security-users'
2020-12-04 08:47:12.760+0000 INFO  Updating initial user password from `auth.ini` file: neo4j
2020-12-04 08:47:15.343+0000 INFO  Bolt enabled on 0.0.0.0:7687.
2020-12-04 08:47:17.964+0000 INFO  Remote interface available at http://localhost:7474/
2020-12-04 08:47:17.964+0000 INFO  Started.

So here's what I tried to do:
I am mounting the certificates by referencing a Kubernetes secret. As far as I know, volume mounts from secrets and configmaps are "read-only" and this cannot be changed. This might cause the problem here. What do you think?

My Kubernetes config (mostly stripped to the important parts):

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: neo4j
  name: neo4j
  namespace: my-namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: neo4j
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: neo4j
    spec:
      containers:
      - env:
        - name: NEO4J_dbms_connector_bolt_advertised__address
          value: neo4j-bolt.domain.com:7687
        - name: NEO4J_dbms_connector_https_advertised__address
          value: neo4j.domain.com:7473
        - name: NEO4J_AUTH
          valueFrom:
            secretKeyRef:
              key: neo4j_datasource_credentials
              name: neo4j-secret
        - name: NEO4J_dbms_connector_bolt_tls__level
          value: OPTIONAL
        - name: NEO4J_dbms_connector_bolt_listen__address
          value: 0.0.0.0:7687
        - name: NEO4J_dbms_connector_https_enabled
          value: "true"
        - name: NEO4J_dbms_connector_https_listen__address
          value: 0.0.0.0:7473
        - name: NEO4J_dbms_ssl_policy_https_enabled
          value: "true"
        - name: NEO4J_dbms_ssl_policy_https_base__directory
          value: /var/lib/neo4j/certificates/https
        - name: NEO4J_dbms_ssl_policy_https_private__key
          value: private.key
        - name: NEO4J_dbms_ssl_policy_https_public__certificate
          value: public.crt
        - name: NEO4J_dbms_ssl_policy_bolt_enabled
          value: "true"
        - name: NEO4J_dbms_ssl_policy_bolt_base__directory
          value: /var/lib/neo4j/certificates/https
        - name: NEO4J_dbms_ssl_policy_bolt_private__key
          value: private.key
        - name: NEO4J_dbms_ssl_policy_bolt_public__certificate
          value: public.crt
        image: IMAGE_URL
        name: neo4j
        ports:
        - containerPort: 7474
          name: http
        - containerPort: 7687
          name: bolt
        - containerPort: 7473
          name: https
        volumeMounts:
        - mountPath: /data
          name: neo4j-persistent-storage
        - mountPath: /var/lib/neo4j/certificates/https
          name: ssl-certificate
          readOnly: false
      volumes:
      - name: neo4j-persistent-storage
        persistentVolumeClaim:
          claimName: neo4j-pv-claim
      - name: ssl-certificate
        secret:
          defaultMode: 511
          secretName: ssl-certificate-secret

Is there a good way to get my certificates working without doing much "by hand"? So for example, I don't want to copy my certificates "by hand" into the persistent volume claim.

Or is there maybe even still something wrong with my configuration?

Any help would be greatly appreciated! Thank you in advance :slight_smile: