Download / Install
PART 2 - First-Time Deployment of the Brain Annex Full-Stack Technology (including web app)
PART 3 - Routine Deployments on a Server (UPDATES from changes in code base)
PART 1 – Install the Neo4j Database
Requirements
You can install Neo4j on either Windows (10 or later; sorry, won't work on Win7), or on Linux (desktop or virtual machine on any cloud.)
We do all our testing – and only support – Neo4j community version 4.4. (Version 4.3 will also probably work, but we no longer support it. Note that use of the enterprise version is NOT required for Brain Annex – the community version will do just fine!)
We feel that Neo4j version 5 is too recent, and don't support it (yet.) Version 4.4 of the Neo4j database will continue to be supported by the Neo4j company thru June 2025 – and that's the only version that Brain Annex will support in the near future.
Some considerations: version 4.4 is the very mature end-of-line for major version 4 of Neo4j (perhaps "4" is their
lucky number?!?)
If you look at the Neo4j release history, you can see that version 4 has been polished in the course of almost 3 full years (2020,
2021, 2022), from its Jan. 2020 release thru the appearance of version 5.0 at the very end of 2022.
During these 3 years, there have been only 5 minor versions: 4.0 thru 4.4; by contrast, in the span of barely a year from Oct. 2022 to
Sept. 2023, there have been a whopping 12 versions of 5.0 thru 5.11 ! This might speak of major version 5
as still unstable/unsettled... Yet another reason to wait!
You may install Neo4j on your local computer, or on a virtual machine on the Cloud (we use the Google cloud, as well as the inexpensive Oracle cloud); or you may use a Neo4j hosting service such as Neo4j Aura (PROVIDED that you can get Neo4j versions 4.4).
Note: for installations on a local machine, the "Neo4j Desktop" program is a convenient alternative – BUT be aware that it ties you to a trial copy of the Enterprise version of Neo4j, and it tries to push you towards products such as "Bloom", which also require the Enterprise version of Neo4j! Also, it's unclear if it supports versions 4.x of the database. At present, we don't support the "Neo4j Desktop" program.
First install Java, as needed
The Neo4j database is internally written in the Java programming language – so, Java must be available on the system.
If doing your own installation, the Java SE Development Kit 11 is needed. (Some pre-made VM images already includes Java, but make sure that it's the right version.)
You can check if Java 11 (used by versions 4.x of the Neo4j database) is already installed by issuing:
java -version
On Ubuntu, Debian and some other Linux versions, Java 11 may be installed, if needed, with:
sudo apt install openjdk-11-jre-headless
On Windows, use this link from Oracle (then
click on the "Windows" tab, and look for the .exe Windows x64 Installer.)
Installing Neo4j
As stated elsewhere, Brain Annex is for the time being staying with version 4 of the Neo4j database, and more specifically 4.4
The following detailed instructions are ONLY for Debian-based Linux distributions (incl. Ubuntu);
for other versions of Linux or for Windows, scroll down.
On Debian or Debian-based Linux (incl. Ubuntu), go to the server and issue:
wget -O - https://debian.neo4j.com/neotechnology.gpg.key | sudo apt-key add -(if wget isn't installed, it may be added with
sudo apt-get install wget )
echo 'deb https://debian.neo4j.com stable 4.4' | sudo tee /etc/apt/sources.list.d/neo4j.list
deb https://debian.neo4j.com stable 4.4Then issue:
sudo apt-get updatesudo apt-get install neo4j
apt list -a neo4j
sudo add-apt-repository universe
apt-cache search neo4j
cypher-shell - command line shell for neo4jWe now have 3 executable files; to see them listed, issue:
ls -alp /usr/bin/neo*
whereis neo4j
- Config file, neo4j.conf , is in
/etc/neo4j - The neo4j "HOME" is
/usr/share/neo4j - Logs are in
/var/log/neo4j - A "man" page is alse created (man neo4j)
For Other Linux distributions:
Use Neo4j download LINK.
IMPORTANT: make sure to get version 4.4.x (We feel that Neo4j version 5 is too recent, and don't support it yet.)
From that page, pick the most recent version of the 4.4.x branch, for your version of Linux. Download the file, and follow the directions on the download page.
For Windows:
Use Neo4j download LINK.
IMPORTANT: make sure to get version 4.4.x (We feel that Neo4j version 5 is too recent, and don't support it yet.)
From that page, pick the most recent version of the 4.4.x branch, for Windows. Download the file, and follow the directions on the download page.
More info
Database Configuration
Make sure that the database is stopped (see the next section), if you need to edit the configuration file. (Of course, if you just installed it, it will be in a stopped state.)
Locate the Neo4j configuration file.
On Linux installations from the repository (we tested on Debian and Debian-derived versions, such as Ubuntu): /etc/neo4j/neo4j.conf
For local Windows installations from a downloaded: NEO4J_HOME\conf\neo4j.conf , where NEO4J_HOME is the folder you chose for your installation
If you're on Linux and have problems finding the Neo4j configuration file, you can get a list of applicable directories with:
whereis neo4j
IMPORTANT: By default, the neo4j server only listens to the local machine.
If you need to make it accept remote connections (such as from the Neo4j browser interface), as typically done, you'll need to edit the config file,
and UN-comment the line (i.e. remove the # in front of it):
dbms.default_listen_address=0.0.0.0
dbms.security.procedures.unrestricted=apoc.*
sudo vim /etc/neo4j/neo4j.conf
(if you SSH to the server with the awesome client MobaXterm, a handy graphic text editor for remote files is included.)
Then (on Linux) verify by showing the edited line with:
cat /etc/neo4j/neo4j.conf | grep dbms.default_listen_address
Neo4j ports, for remote access to server installations
By default, the Neo4j server runs on port 7474. Only connections from localhost are permitted, unless the config file is changed, as discussed in the previous section!
Other available ports are the following:
HTTPS: 7473
HTTP: 7474
Bolt Protocol: 7687
If you're installing Neo4j on a server, and want to enable remote access to the database, you must open all of those ports (7474, 7473, 7687).
This will be needed if the database and the app are on different machines, or if you want to access the database over the internet from the Neo4j browser or other front ends.
Note: if you use the Brain Annex web app, depending on how you install it, you'll also need access to ports 5000 (for Flask/Werkzeug) or 80 (for HTTP) or 443 (for HTTPS)
Start the Neo4j database
If you ended up using the "Neo4j Desktop" program with a local database, it's very convenient to use that Neo4j Desktop program to stop/start the database. (But beware, as mentioned earlier, about being on the correct version of the Neo4j database!)
If you installed Neo4j on a Linux machine, it can typically be started with:
sudo neo4j start
sudo neo4j stop
sudo neo4j status
more /var/log/neo4j/neo4j.log
Test Neo4j access thru its Browser Interface
Did the installation go well? You can test it by means of the "Neo4j Browser Interface" (a web app that comes with Neo4j),
by opening a browser tab and going to either
localhost:7474 or http://YOUR_IP_ADDDRESS:7474 ,
and then log inas follows:
Install the Neo4j APOC libraries (optional but highly recommended)
The Neo4j "APOC" libraries add a good deal of extra functionality to the database: they provide a large set of functions that can be incorporated into Cypher queries. Those libraries are created and maintained by the Neo4j company itself, but somehow they're not part of the standard installation.
From the official documentation:
"[APOC procedures] are custom implementations of certain functionality, that can’t be (easily) expressed in Cypher itself.
They are implemented in Java and can be easily deployed into your Neo4j instance, and then be called from Cypher directly."
Currently, the APOC library is needed by the Brain Annex web app for making a JSON dump of all the Neo4j data, or of just the schema, using Brain Annex's admin control panel.
To install the APOC libraries on Linux (we tested on Ubuntu and Debian):
cd /etc/neo4j (or use the location of your Neo4j config file; see earlier section)sudo vim neo4j.conf (or use your favorite text editor)Add the following line at the very end of the file:
dbms.security.procedures.unrestricted=apoc.*Then, at the Linux shell command prompt, issue:
tail neo4j.conf (to verify the addition of the last line)sudo neo4j stop (the database needs to be stopped; will be restarted a few lines below)cd /var/lib/neo4j/pluginssudo wget https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/4.4.0.16/apoc-4.4.0.16-all.jar
(note the 4.4, matching the version of the Neo4j database that BrainAnnex uses)If wget isn't available (such as on Debian Linux), one can install it with: sudo apt-get install wget
sudo chown -R neo4j:adm apoc-4.4.0.16-all.jarsudo neo4j startRETURN apoc.version()[No need to disconnect/reconnect the browser, if already running.] It should return the version number (4.4.0.16).
To install the APOC libraries on Windows, please see the official documentation
After installing APOC, don't forget to add the line dbms.security.procedures.unrestricted=apoc.* to the Neo4j config file, as explained in another section.
PART 2 – First-Time Deployment of the Brain Annex Full-Stack Technology (including web app)
This section covers the initial setup. If you're doing a routine upgrade (i.e. to bring up-to-date from UPDATES from changes in code base), skip forward to Part 3.
Note: if you only want to use the lowest-level library, NeoAccess, then the full installation of the entire stack is
NOT necessary; you can simply install the NeoAccess library from the standard
python repository (use your IDE installer, or do a pip install neoaccess )
Verify you have Python 3.8+
Make sure that you have the right version of Python installed.
python3 --version (on Linux)python --version (on the Windows command prompt or Power shell)
Clone the repository
Clone the repository to either your local machine or to your server.
To clone to a Windows machine, we recommend using the awesome program GitHub Desktop.
To clone to a Linux machine, you may use commands such as:
cd /home/ (change the location directory, if desired)sudo mkdir brain_annexsudo git clone --verbose --config http.sslVerify=false https://github.com/BrainAnnex/brain-annex.git brain_annex(Note: if git is not already installed, install it with:
sudo apt-get install git )ls -alF brain_annex
(you should see several files, incl. main.py, and several subdirectories)
Folder setup
Create or designate:
1 - A folder for the media files. In the examples below, we use brain_annex_media, but it could be something else; make sure it matches what you
will specify in the configuration file!
IMPORTANT: for now, the media folder MUST include a subfolder called "resized"
EXAMPLE (on Windows): create a folder named "brain_annex_media" on the "D" drive, and a subfolder named "resized" inside it.
EXAMPLE (on Linux):
cd /home/ (the parents of the desired location; change as needed)sudo mkdir brain_annex_media (or whatever name one wants to use; make sure it matches the one in the config file)sudo mkdir brain_annex_media/resized (a subfolder of the previous one)ls -alF brain_annex_media (you should see the "resized" subfolder)sudo chmod -R o+w brain_annex_media (a more restrictive permission might be used, if desired)
2 - A folder for the uploaded files.
EXAMPLE (on Windows): create a folder named "tmp" on the "D" drive
EXAMPLE (on Linux): do nothing! Will just use
/tmp/Brain Annex Configuration File
Go to the brain_annex folder that you created earlier.
Duplicate the file `config.defaults.ini` and rename the copy as `config.ini`
(on Linux, after navigating to that folder, do: sudo cp config.defaults.ini config.ini )
Edit config.ini (for example, open with "notepad" on Windows, or sudo vim on Linux),
to change, as needed, values for:
| NEO4J_HOST |
the default `neo4j://localhost:7687` is for setups
where the database is on the same machine as the Brain Annex web app;
if it is on another machine, use values such as bolt://123.456.0.29:7687 or bolt://your_domain.com:7687
|
| NEO4J_USER | `neo4j` is a typical default, unless you've changed it |
| NEO4J_PASSWORD | Neo4j makes people change the password the first time they log in thru the Neo4j browser, as discussed in earlier section |
| PORT_NUMBER |
ok to keep the default 5000, unless you have a conflict. Not applicable if you use gunicorn or other web server |
| DEPLOYMENT |
Either LOCAL or REMOTE Use LOCAL if starting the app with Flask (regardless of where the app runs); use REMOTE if starting the app with gunicorn (regardless of where the app runs) |
| MEDIA_FOLDER |
where the user-submitted images, formatted text, documents, etc, are stored. IMPORTANT: names MUST end with a slash EXAMPLE (on Windows): MEDIA_FOLDER = D:/media/ EXAMPLE (on Linux): MEDIA_FOLDER = /home/brain_annex_media/ |
| UPLOAD_FOLDER |
temporary location for uploads EXAMPLE (on Windows): UPLOAD_FOLDER = D:/tmp/ EXAMPLE (on Linux): UPLOAD_FOLDER = /tmp/ |
| INTAKE_FOLDER | optional, if "continuous data ingestion" is used |
| OUTTAKE_FOLDER | optional, if "continuous data ingestion" is used |
| BRANDING | optional, if a different branding is desired. Branding is shown on the Login page |
Install all dependencies in the virtual environment
All dependencies (such as Flask, pandas, etc) are specified in the file `requirements.txt`
If you're running Brain Annex locally, your IDE will probably prompt you as to whether you want to install the requirements
(at least, PyCharm does.)
If you're installing Brain Annex on a Linux server, you need
to manually create a virtual environment, using pip3 or your favorite command.
For example (IMPORTANT: change "home/brain_annex" everywhere, if you used a different location):
$ ls /home/brain_annex (all project files should be there, incl. README.md, but NO venv folder; change the name if you used a different location)
$ sudo chmod go+w /home/brain_annex (because brain_annex is owned by the root; to avoid trouble during pip3 install later!)
$ python3 -m venv /home/brain_annex/venv (IMPORTANT: do NOT use sudo here, or permissions will get messed up! It creates a new virtual environment, but doesn't install any Packages)
$ sudo apt-get install python3-venv (only needed if the previous command failed! Then re-issue the previous command)
$ ls -alF /home/brain_annex/venv/bin (it shows the various "Activate" files, and the links to the python executables)
$ cd /home/brain_annex
$ source venv/bin/activate (to enter the context of the new virtual environment)
[Notice how the shell prompt is now prefixed with the name of your environment (venv, in our case). This is the indicator that venv is currently active, which means the python executable will only use this environment's packages and settings.]
(venv) $ pip3 install -r requirements.txt (no worries if you see "ERROR: Failed building wheel for neo4j"; it automatically recovers from it!)
(venv) $ pip3 list (to see all the installed packages; note that neo4j is there, even if the earlier error message was present)
(venv) $ deactivate (to leave the context of the new virtual environment)
(venv) $ pip install --upgrade pip
(venv) $ pip install wheel
Starting the Brain Annex web app
The first order of business is to test the connection to the Neo4j Database from Python.
IMPORTANT: make sure that the Neo4j server is running! (see the earlier section on "Install the Neo4j Database")
For LOCAL installations:
If it doesn't find the database (e.g., if you forgot to start Neo4j),
it'll give you an error message such as:
"Exception: CHECK WHETHER NEO4J IS RUNNING!"
In case of database-connection problems, you may also try to run the following diagnostic files:
tests_manual/db_connection_1_MANUAL.py
tests_manual/db_connection_3_FROM_CONFIG.py
For LINUX SERVER installations:
On a Linux server, you can do the following (OPTIONAL, but recommended) testing; but, first, make sure that the Neo4j server is running, as discussed in the section on Neo4j:
$ cd /home/brain_annex [CHANGE the directory name, if used a different one]
$ source venv/bin/activate
(venv) $ ls venv/lib (make a note of your version of python; make sure it's 3.8+)
(venv) $ export PYTHONPATH=$PYTHONPATH:`pwd`/venv/lib/python3.8/site-packages [OR whichever version was shown in the previous step]
(venv) $ python3 diagnostics.py (just for testing, if desired; make sure that the 'home/brain_annex' folder, or whichever home folder you used, appears in the sys.path)
(venv) $ cat config.ini (just for testing, if desired; verify the dbase credentials)
(venv) $ python3 tests_manual/db_connection_1_MANUAL.py (just for testing, if desired; enter
neo4j://localhost when prompted about the host;
then it will prompt for the dbase password)(venv) $ python3 tests_manual/db_connection_3_FROM_CONFIG.py (just for testing, if desired; it uses the dbase credentials in the Brain Annex config file)
(Continue to one of the options below: either Flask's built-in server or to gunicorn...)
(OPTION 1 - using Flask's built-in server Werkzeug)
DEPLOYMENT = REMOTE in your Brain Annex config.ini fileMake sure that port 5000, or whichever port is used below, is open; check the firewall rules on your VM
(venv) $ export FLASK_APP=main.py
(venv) $ flask run --host=0.0.0.0 --port 5000 &> flask_log.txt & (to redirect both stdout and stderr; change port if desired - but be aware that 80 won't work)
(venv) $ ps -e | grep flask (to double-check there's a 'flask' process running in the background)
(venv) $ deactivate
$ cat flask_log.txt (optional, to check the log)
TESTING IT
If running remotely, set your browser to
http://YOUR_SERVER_IP_ADDRESS:5000If running locally, set your browser to
http://localhost:5000Problems? Try the following diagnostics (change the port # if needed; the port should be shown as being listened to):
$ sudo lsof -i:5000 (to investigate the network)$ netstat -an | grep 5000 (to further investigate the network)(note: replace 5000 with whatever port number you used in your configuration.)
(OPTION 2 - using Gunicorn; common initial steps for all Gunicorn variations below)
DEPLOYMENT = REMOTE in your Brain Annex config.ini filedeactivate (if needed, from previously being in a venv)
$ sudo su - (To elevate permissions to the root account)
root # cd /home/brain_annex (Or whichever home location is being used)
root # source venv/bin/activate
(venv) root # ls venv/lib (make a note of the version of python; make sure it's 3.8+)
(venv) root # export PYTHONPATH=$PYTHONPATH:`pwd`/venv/lib/python3.8/site-packages (OR whichever version was shown in the previous step)
If you're following Option 2 (i.e. Gunicorn), finish up with either 2a or 2b:
(OPTION 2a - using Gunicorn with HTTP, on port 80)
Make sure that port 80 is open; check the firewall rules on your VM
(venv) root # gunicorn -w 1 -b 0.0.0.0:80 main:app &
OR
(venv) root # gunicorn --worker-class gthread --threads 3 -w 1 -b 0.0.0.0:80 main:app &
(to use multiple threads; it's recommended to replace the 3 with: (2 x $num_cores) + 1)
TESTING IT
Set your web browser to
http://YOUR_SERVER_IP_ADDRESS
(OPTION 2b - using Gunicorn with HTTPS, on port 443)
Make sure that port 443 is open; check the firewall rules on your VM
Get a digital certificate (we recommend the free and convenient Let's Encrypt)
(venv) root #
gunicorn --certfile=/etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem --keyfile=/etc/letsencrypt/live/YOUR_DOMAIN/privkey.pem
--error-logfile gunicorn_log.txt -b 0.0.0.0:443 main:app > brainannex_log.txt & (Replace YOUR_DOMAIN with whatever you're actually using, such as example.com)
TESTING IT
* If you run
ps -e | grep gunicorn , you should see 2 gunicorn processes (running in the background)* Set your web browser to
https://YOUR_DOMAIN* App log can be found in:
brainannex_log.txt (contents get buffered; so, they may not be immediately visible)* For Gunicorn log (incl. errors):
gunicorn_log.txt
