Bit of backgorund
Hello all, I am new to Neo4J community and in database design in general.
I am a Visual Effects Technical Director based out of London and my day job usually includes creating pictures, tools etc for visual effects in films. I am planning to create an Asset Management system which allows me to iterate much faster and manage the assets in optimal manner.
I have scoured the internet for different database platforms both SQL and NoSQL based but never really felt that these tools aligned in a way I thought about data. I happened run into graph based semantic databases and discovered Neo4J which I think is perfect for the way I want to create my Asset management system.
Since I am fairly new and doing this by myself in my own time, please excuse any noob questions I may have thrown
With that background out the way, I want to actually start throwing questions now.
Primary objectives for AMS
I appreciate that this is quite a daunting task and have set some realistic expectations for the database at the moment. The primary objective of the database right now will be to efficiently resolve asset paths on disks, their dependencies, metadata (who created the assets, when was it created, department etc).
I also want to write few applications which will allow me to analyse this data and put it in production context (how long the iteration of each asset takes, how many assets have been generated etc) which also gives it Task management like properties. This is also an important aspect for me. Which leads me to quite a few design possibilities and I would love to get your opinions and thoughts on this.
-
Should I create one database which handles both the tasks and assets or should I have one database for Assets and another for Tasks.
-
Keeping them together in same database gives me a richer contextual view but is it going to be performant when the tasks and asset count increases dramatically. (VFX is highly iterative process and usually has multiple versions of each assets. There also are huge number of tasks cumulatively)
-
To work around this I thought what if I could separate these databases and have them communicate with each other. So anytime a task gets created in the database handling tasks, the asset management database is made aware of this. Is this over-engineering things? What would be better ways to handling these scenarios?
-
What might be the common pitfalls and noob mistakes when implementing something like this?
-
The plan I have at the moment is to model a database using Neo4j which will be running on a central server (I plan to use a NAS). Ideally there will be multiple users (only two at the moment however, it should scale well with more users) accessing the database using different machines via the official python drivers. How efficient/fast is it going to be to write a python application which is used to interface with the database.
I am sure some of these questions might be super basic however I wanted to get these out here on the forum so I can learn from the community. I look forward to your suggestions and opinions on this.
Cheers,
Bhavesh.