Load balancing Neo4j community edition ami on aws is not providing any response

I have set up the neo4j-community-1-3.5.3-apoc 2019-02-20T06_13_56Z-42e71149-46df-4ee5-ae1b-34d134153fd6-ami-0aeebc5d17c06beff.4 (ami-0462c7494bc1f7e89) on a t2.medium on aws. Since I want to access it via a private (company) network I do not have any public ips on it.

But the way I understand it, I should be able to access the Neo4j browser via the private ip with port 7473/7474, right?

Secondly, I am able to ssh into the machine but where can I find the neo4j-conf file to check if the ports are really open or not?

Third, I have set up a network load balancer (primarily to get a readable url instead of using the private ip/private dns provided by the EC2 instance) but health check fails on the registered target (Neo4j AMI) even though the security group shows ingress for all protocols and all ports is open to traffic from the company's network.

What am I missing? Please help.

No, the private IP will only be visible to other machines on the AWS network. To access the machine you should use the public CNAME, which will point to the public IP. Note that the public IP will change each time the machine is restarted, however the CNAME will always point to what it is.

After you've confirmed access, consider setting firewall rules to limit IP addresses that can access these ports.

Depends how it was set up, but probably you'll need a public key for the public/private key pair. A good place to get help on AWS specific issues is: Forums | AWS re:Post

Third, I have set up a network load balancer (primarily to get a readable url instead of using the private ip/private dns provided by the EC2 instance) but health check fails on the registered target (Neo4j AMI) even though the security group shows ingress for all protocols and all ports is open to traffic from the company's network.

That's one way to obtain an internet route to a resource on the private network, however you could just use the machine cname (shown as 'Public DNS IPv4 in the console).

This will be ok, however, for consistent production loads, I generally recommend M class machines as they can provide consistent non-exhausting CPU and IO resources.

Thanks for your post!

It looks like another will probably fix the issue I am facing. Bug Report - AWS Community AMI fails when deployed to a private subnet

I think I have caused confusion by not stating what exactly the problem was. :( But unfortunately I did not know at the time what might be the cause (not that I am sure now, but kind of feels like I might finally have a handle on it)

I am using a keypair, which is how I am able to ssh into the instance. And the health checks again are probably failing because neo4j is not starting when I have only private ips and dns available (private subnet only) (again, I will check and then reply on the results)

Since it is a private subnet there is no public DNS.

If neo4j is running the load balancer should be able to access it (as you have mentioned further on in your post too). So I am thinking the problem is with Neo4j not starting up when only private ip is allocated to the instance. But you are right, I should (and will) be using the CNAME to access the instance (after setting it up against the load balancer).

Hi @daniel.picardo, can you confirm Neo4j is running by looking at the logfile? Depends on how Neo4j was installed, but usually neo4j.log

I am using the community ami on Ubuntu. And in the /var/log/neo4j directory there is only one file: debug.log . I see a bunch of warnings. What message would I see if Neo4j was successfully loaded?

Following the last few entries:

I tried

neo4j version
neo4j 3.5.3

$ neo4j status
Neo4j is not running

I am trying this with ubuntu user. Do I need to switch to neo4j user? (btw, I am unable to su neo4j since it says the instance-id password is incorrect (failed to authenticate)). Or do I need to be running the reset-password-aws.sh script in /etc/neo4j?

Ok, so I figured out that neo4j is not starting up. I am using ami:

neo4j-community-1-3.5.3-apoc 2019-02-20T06_13_56Z-42e71149-46df-4ee5-ae1b-34d134153fd6-ami-0aeebc5d17c06beff.4 (ami-0462c7494bc1f7e89)

The current pre-neo4j.sh script has the following (in bold) which prevents it from advertising the private ip if no external ip is available.

if [ $? -ne 0 || "$EXTERNAL_IP_ADDR" = "" ] ; then
echo "pre-neo4j.sh: Advertising internal IP since instance lacks external public IP"
export EXTERNAL_IP_ADDR=$INTERNAL_IP_ADDR
fi

The $? refers to setting the EXTERNAL_IP_ADDR in the previous step and the thing is: that command runs successfully but EXTERNAL_IP_ADDR is empty. So EXTERNAL_IP_ADDR needs to be checked first and then the $?, looks like

Then I switched user to root and ran systemctl start neo4j and boom! neo4j up and running and accessible via private port.

I will create another image and verify this just to be on the safe side.

1 Like

confirmed that the issues I was facing was due to internal ip address not being advertised in the case when public ips are not available

1 Like