Download / Install

PART 1 - Install the Neo4j Database

PART 2 - First-Time Deployment of the Brain Annex web app

PART 3 - Routine Deployments on a Server (UPDATES from changes in code base)

PART 1 – Install the Neo4j Database

See separate page

PART 2 – First-Time Deployment of the Brain Annex web app

IMPORTANT: This section covers the initial setup. If you're doing a routine upgrade (i.e. to bring up-to-date from UPDATES from changes in code base), skip forward to Part 3.

Note: if you only want to use the BrainAnnex python LIBRARY, then the full installation of the entire software stack is NOT necessary; you can simply pip install either brainannex-neo4jv4 or brainannex-neo4jv5, depending on your version of the database.

THE INSTRUCTIONS IN THIS SECTION ARE PURELY FOR THE BrainAnnex WEB APP INSTALL, if the web app is what you need.

Verify you have Python 3.10+

Make sure that you have the right version of Python installed. Earlier versions will NOT work! We develop and test on 3.10 and 3.11

python3 --version (on Linux)

python --version (on the Windows command prompt or Power shell)

Needed ports, for the web servers

If you're installing Neo4j on a server, you'll need access to ports 5000 (for Flask/Werkzeug) or 80 (for HTTP) or 443 (for HTTPS) or 8000 (if you use Caddy and you want to bypass it for testing.)

This is in addition to any ports that had to be opened for Neo4j (see separate page.)

Clone the repository

Clone the repository to either your local machine or to your server.

To clone to a Windows machine, we recommend using the awesome program GitHub Desktop.
Alternatively, you can download the latest zipped release from GitHub, and unzip it to some folder on your machine.

To clone to a Linux machine, you may use commands such as:

cd /home/ (change the location directory, if desired)

sudo mkdir brain_annex

sudo git clone --verbose --config http.sslVerify=false https://github.com/BrainAnnex/brain-annex.git brain_annex

(Note: if git is not already installed, install it with:    sudo apt-get install git  )

ls -alF brain_annex (you should see several files, incl. main.py, and several subdirectories)

If you're using version 5 of Neo4j, make a few changes to the cloned repository

IMPORTANT! What version of Neo4j are you using?

The files in the repository are configured for version 4.4 of the Neo4j database.

If you want to use version 5 of Neo4j (which we recently started supporting) with the BrainAnnex web app, make sure to apply the following changes to the files from the repo:

1. In the file brainannex/__init.py__ , comment out the line that mentions intergraph_neo4j_4, and uncomment the line that mentions intergraph_neo4j_5
2. In the files requirements.txt, comment out the line that mentions version 4 of Neo4j, and uncomment the line that mentions version 5

Folder setup

Create or designate:

1 - A folder for the media files. In the examples below, we use brain_annex_media, but it could be something else; make sure it matches what you will specify in the configuration file!


EXAMPLE: create a folder named "brain_annex_media" on the "D" drive.



EXAMPLE:

cd /home/ (the parents of the desired location; change as needed)

sudo mkdir brain_annex_media (or whatever name one wants to use; make sure it matches the one in the config file)

sudo chmod -R o+w brain_annex_media (a more restrictive permission might be used, if desired)

2 - A folder for the uploaded files.

EXAMPLE: create a folder named "tmp" on the "D" drive

EXAMPLE: do nothing! Will just use the standard /tmp/ directory

Brain Annex Configuration File

Go to the brain_annex folder that you created earlier.

Duplicate the file `config.defaults.ini` and rename the copy as `config.ini`
(on Linux, after navigating to that folder, do:   sudo cp config.defaults.ini config.ini )

The file config.ini will be your PRIVATE file that will NOT be affected by any future pull from the repository (it is on the list of excluded files in the Git configuration file .gitignore)

Edit config.ini (for example, open with "notepad" on Windows, or sudo vim config.ini on Linux), to change, as needed, values for:

NEO4J_HOST	The default `neo4j://localhost:7687` is for setups where the database is on the same machine as the Brain Annex web app; if it is on another machine, use values such as bolt://123.456.0.29:7687 or bolt://your_domain.com:7687
NEO4J_USER	`neo4j` is a typical default, unless you've changed it
NEO4J_PASSWORD	Neo4j makes people change the password the first time they log in thru the Neo4j browser, as discussed in earlier section
DEPLOYMENT	Use either FLASK (typically for local dev runs) or EXTERNAL (typically for production on a remote server) : - use FLASK if employing the built-in WSGI HTTP server that comes with Flask ("Werkzeug"). NOT meant for production! In that case the Flask web app will get started directly. - use EXTERNAL if the WSGI HTTP server is external to the Flask application (e.g. gunicorn, waitress, hypercorn, etc) In that case the Flask web app will get started by gunicorn or other external software. Note: this is totally UN-related to running the Brain Annex web app locally or on a remote server; it's often the case that one uses Flask built-in capabilities in a local development environment.
PORT_NUMBER	ONLY applicable if you chose the DEPLOYMENT = FLASK option, above (i.e. not relevant if you use "gunicorn" or other WSGI HTTP server) Ok to keep the default 5000, unless you have a conflict.
MEDIA_FOLDER	Where the user-submitted images, formatted text, documents, etc, are stored. IMPORTANT: names MUST end with a slash EXAMPLE (on Windows): MEDIA_FOLDER = D:/media/ EXAMPLE (on Linux):   MEDIA_FOLDER = /home/brain_annex_media/
UPLOAD_FOLDER	Temporary location for uploads EXAMPLE (on Windows): UPLOAD_FOLDER = D:/tmp/ EXAMPLE (on Linux):   UPLOAD_FOLDER = /tmp/
INTAKE_FOLDER	Optional, if "continuous data ingestion" is used
OUTTAKE_FOLDER	Optional, if "continuous data ingestion" is used
BRANDING	Optional, if a different branding is desired. Branding is shown on the Login page, and in the fix top bar of every page

(Note: an alternate way to set the database credentials is thru environment variables; for example, if you're running locally, using PyCharm, you can set the environment variables from Run > Edit Configurations...)

Install all dependencies in the virtual environment

All dependencies (such as Flask, pandas, etc) are specified in the file requirements.txt

STOP: if you're using Neo4j version 5, did you make the needed changes to requirements.txt and another file, as spelled out in a previous section??

LOCAL INSTALL with an IDE: Your IDE will probably prompt you as to whether you want to install the requirements (at least, PyCharm does.)

No IDE: If you're installing Brain Annex on a server, or doing a local install without an IDE, you need to manually create a virtual environment, using pip3 or your favorite command.

For example, on Linux (IMPORTANT: make sure to change as needed /home/brain_annex in all instructions, if you used a different location):

$   ls /home/brain_annex (all project files should be there, incl. README.md, but NO venv folder; change the name if you used a different location)

$   sudo chmod go+w /home/brain_annex (because brain_annex is owned by the root; to avoid trouble during pip3 install later!)

$   python3 -m venv /home/brain_annex/venv (IMPORTANT: do NOT use sudo here, or permissions will get messed up! It creates a new virtual environment, but doesn't install any Packages)

$ sudo apt-get install python3-venv (ONLY needed if the previous command failed! Then re-issue the previous command)

$   ls -alF /home/brain_annex/venv/bin (it shows the various "Activate" files, and the links to the python executables)

$   cd /home/brain_annex

$   source venv/bin/activate (to enter the context of the new virtual environment)

[Notice how the shell prompt is now prefixed with the name of your environment (venv, in our case). This is the indicator that venv is currently active, which means the python executable will only use this environment's packages and settings.]

(venv) $   pip3 install -r requirements.txt

ERRORS? no worries if you see "ERROR: Failed building wheel for neo4j"; it automatically recovers from it!
HOWEVER, in some cases, it says "Failed to build psutil" ; in that case, run:
sudo apt-get install gcc python3-dev
and then repeat the previous command.

(venv) $   pip3 list (to see all the installed packages; note that neo4j is there, even if it had previously said "Failed building wheel for neo4j")

(venv) $   deactivate (to leave the context of the new virtual environment)


In case of failures during the installation, try:

(venv) $   pip install --upgrade pip
(venv) $   pip install wheel

Starting the Brain Annex web app

IMPORTANT: make sure that the Neo4j server is running! (see the separate page on Install the Neo4j Database)




For Windows, here we'll only cover installations employing the built-in WSGI HTTP server that comes with Flask ("Werkzeug"). Typically, LOCAL installations NOT meant for production :

Make sure that the Neo4j server is running, as discussed in the section on Neo4j.

Make sure that your `config.ini` file contains the line: DEPLOYMENT = FLASK

Run `main.py`, in the context of your virtual environment, which starts Flask and the included Werkzeug web server.
If you're using an IDE such as PyCharm, the virtual environment is automatically taken care of, if you simpy run `main.py`

Set your browser to   http://localhost:5000

If it doesn't find the database (e.g., if you forgot to start Neo4j), it'll give you an error message such as:
"Exception: CHECK WHETHER NEO4J IS RUNNING!"

In case of database-connection problems, you may also try to run the following diagnostic files (in the top folder of the installation):

• diagnostics.py
  
• diagnostics_db_connect_1_MANUAL.py
• diagnostics_db_connect_3_FROM_CONFIG.py

Now you can skip to the next section, Using the Brain Annex web app!





For Linux, here we'll cover multiple cases: installations employing the built-in WSGI HTTP server that comes with Flask ("Werkzeug"), OR using an external WSGI HTTP server such as gunicorn. Typically, SERVER installations


First make sure that the Neo4j server is running, as discussed in the section on Neo4j.

Next, you can do the following testing (OPTIONAL, but highly recommended for first installations):

[OPTIONAL SECTION]

( you may skip the following 2 first steps, if you never deactivated the virtual environment from earlier)

$   cd /home/brain_annex	CHANGE the directory name, if you used a different one
$   source venv/bin/activate	To start the virtual environment

(venv) $   ls venv/lib (make a note of your version of python; make sure it's 3.10+)
(venv) $   export PYTHONPATH=$PYTHONPATH:`pwd`/venv/lib/python3.10/site-packages [OR whichever version was shown in the previous step]

(venv) $   python3 diagnostics.py (just for testing, if desired; make sure that the /home/brain_annex folder, or whichever home folder you used, appears in the sys.path)

(venv) $   cat config.ini (just for testing, if desired; verify the dbase credentials)



(venv) $   python3 diagnostics_db_connect_1_MANUAL.py
(just for testing, if desired; enter   neo4j://localhost   when prompted about the host; then it will prompt for the dbase password. If all goes well, it will show some diagnostic info)

(venv) $   python3 diagnostics_db_connect_3_FROM_CONFIG.py
(just for testing, if desired; it uses the dbase credentials in the Brain Annex config file. If all goes well, it will show some diagnostic info)

(venv) $   deactivate

END of TESTING part for the Linux server installation. Continue to one of the 2 options below: either Flask's built-in server or to Gunicorn...

---

(OPTION 1 of 2 – server installation using Flask's built-in server Werkzeug)
Make sure that port 5000, or whichever port is used below, is open; check the firewall rules on your VM

$   cd /home/brain_annex	CHANGE the directory name, if you used a different one
(venv) $   source venv/bin/activate	To start the virtual environment
(venv) $   cat config.ini | grep DEPLOYMENT	to make sure that you have DEPLOYMENT = FLASK in your Brain Annex config.ini file
(venv) $   export FLASK_APP=main.py	to inform Flask on how to start
(venv) $   flask run --host=0.0.0.0 --port 5000 &> flask_log.txt &	to redirect both stdout and stderr; change port if desired - but be aware that 80 won't work
(venv) $   ps -e | grep flask	to double-check there's a 'flask' process running in the background
(venv) $   deactivate	leave the venv
$   cat flask_log.txt	optional, to check the log

TESTING IT
Set your browser to   http://YOUR_SERVER_IP_ADDRESS:5000

Problems? Try the following diagnostics (change the port # if needed; the port should be shown as being listened to):

$   sudo lsof -i:5000 (to investigate the network)
$   netstat -an | grep 5000 (to further investigate the network)

They both should show "LISTEN"

(note: replace 5000 with whatever port number you used in your configuration.)

---

(OPTION 2 of 2 – server installation using Gunicorn, with or without Caddy; COMMON INITIAL STEPS for all Gunicorn variations below)

$   sudo su -	to elevate permissions to the root account; notice the changed prompt in the next line
root #   cd /home/brain_annex	or whichever home location is being used
root #   cat config.ini | grep DEPLOYMENT	make sure it says:   DEPLOYMENT = EXTERNAL
root #   source venv/bin/activate	this will enter the virtual environment; notice the changed prompt in the next line
(venv) root #   ls venv/lib	make a note of the version of python; MAKE SURE it's 3.10+
(venv) root #   export PYTHONPATH=$PYTHONPATH:`pwd`/venv/lib/python3.10/site-packages	OR whichever version was shown in the previous step; it MUST be 3.10+

If you're following Option 2 (i.e. server installation using Gunicorn, with or without Caddy), finish up with EITHER 2a OR 2b OR 2c:


(OPTION 2a - IF using Gunicorn with HTTP, on port 80, NO Caddy)

Make sure to follow the common initial steps of Option 2, above.
Make sure that port 80 is open; check the firewall rules on your VM

(venv) root #   gunicorn -w 1 -b 0.0.0.0:80 main:app &

OR variations such as:
(venv) root #   gunicorn --worker-class gthread --threads 3 -w 1 -b 0.0.0.0:80 main:app > brainannex_log.txt &
(Parameter recommendations in the gunicorn docs)

TESTING IT
Set your web browser to http://YOUR_SERVER_IP_ADDRESS

(OPTION 2b - IF using Gunicorn with HTTPS, on port 443, NO Caddy)

Make sure to follow the common initial steps of Option 2, above.
Make sure that port 443 is open; check the firewall rules on your VM
Make sure that you have a domain name, pointing to your VM
Get a digital certificate (we recommend the free and convenient Let's Encrypt)

EITHER DO A "MINIMALIST" INVOCATION of gunicorn:
(venv) root #   gunicorn --certfile=/etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem --keyfile=/etc/letsencrypt/live/YOUR_DOMAIN/privkey.pem --error-logfile gunicorn_log.txt -b 0.0.0.0:443 main:app   > brainannex_log.txt &
(Replace YOUR_DOMAIN with whatever you're actually using, such as example.com)

OR, ALTERNATIVELY, USE MULTIPLE WORKERS (for example, 5):
(venv) root #   gunicorn -w 5 --certfile=/etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem --keyfile=/etc/letsencrypt/live/YOUR_DOMAIN/privkey.pem --error-logfile gunicorn_log.txt -b 0.0.0.0:443 main:app   > brainannex_log.txt &
(The gunicorn docs recommend  (2 x $num_cores + 1)  as the number of workers to start off with.   To find the number of cores, you can use lscpu)

TESTING IT
* If you run   ps -e | grep gunicorn   , you should see (1 + number of workers) gunicorn processes (running in the background.) For example, in the case of the "minimalist" gunicorn invocation, there should be 2 processes.
* Set your web browser to https://YOUR_DOMAIN
* App log can be found in: brainannex_log.txt (contents get buffered; so, they may not be immediately visible)
* For Gunicorn log (incl. errors): gunicorn_log.txt

(OPTION 2c - IF using Caddy alongiside Gunicorn. In this case, Gunicorn will use HTTP, on port 8000)

Make sure to follow the common initial steps of Option 2, above.
Make sure that port 8000 is open; check the firewall rules on your VM
Make sure that you have a domain name, pointing to your VM
(Note: Caddy takes care of automatically getting a digital certificate for you, if you have a domain name)

If you don't already have a file named "Caddyfile" in your project top folder (for example /home/brain_annex), make sure to create one. A generic one, fine-tuned for using the BrainAnnex app with your own domain name, is https://github.com/BrainAnnex/brain-annex/blob/main/Caddyfile (make sure to change the domain name to your own.)

(venv) root #   gunicorn -w 7 --error-logfile gunicorn_log.txt -b 0.0.0.0:8000 main:app   > brainannex_log.txt &
(The gunicorn docs recommend  (2 x $num_cores + 1)  as the number of workers to start off with.   To find the number of cores, you can use lscpu)

Start Caddy (if not already running from a previous installation):   sudo caddy start


TESTING IT
* Is Gunicorn running? If you enter into the Linux shell:   ps -e | grep gunicorn   , and you should see (1 + number of workers) gunicorn processes (running in the background.)
* To bypass Caddy and only test the gunicorn server, set your browser to http://YOUR_IP_NUMBER:8000
* Is Caddy running? Try the Linux command:   curl localhost:2019/config/ | jq
* To test the complete system, set your web browser to https://YOUR_DOMAIN
* App log can be found in: brainannex_log.txt (contents get buffered; so, they may not be immediately visible)
* For Gunicorn log (incl. errors): gunicorn_log.txt

1. is the port number that you're using open on your server's firewall?
   
2. if you're on Ubuntu, you may need extra steps to open ports; for example, to open port 80 issue: sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT (More info).
   Special instructions for Ubuntu on the Oracle cloud
3. did you start the Neo4j server?
   
4. (if this is an upgrade) did you check the Change Log for any special instructions?

Using the Brain Annex web app

If you followed the directions in the previous sections, when you set your browser to the URL specified in the instructions in that section for your specific installation scenario, you should see a simple web pages with a message and a few links.

Depending on the deployment you used, the URL could be:

• https://YOUR_DOMAIN
• http://YOUR_SERVER_IP_ADDRESS
• http://localhost:5000

Create Database Schema (optional but highly recommended)

If this is your initial install of the web app, there are a few dozen nodes and relationships that will have to be added to the database if you want to use the various core modules and plugins for the "multimedia content management" part of the BrainAnnex web app; if your only intended use are the control panels of the web app, then it's NOT needed.
More info on Schema Layer

(you may skip these first 2 steps, if you never deactivated the virtual environment from earlier)

$   cd /home/brain_annex   [CHANGE the directory name, if used a different one]
$   source venv/bin/activate


(venv) $   python3 initialize_schema.py

---

Use .\venv\Scripts\Activate.ps1 (assuming you are already in the right folder, as stated above; there should be a "venv" subfolder.)
If you get a policy error, first run:
Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned
Then run: python initialize_schema.py

Create User Login Account (first user or additional users)

If this is your initial install of the web app, you must create a first user login account (the first account creation will also add the "Schema" data necessary for User management.)

You may also re-run the same script, below, at any later time to create additional login accounts:

(you may skip these first 2 steps, if you never deactivated the virtual environment from earlier)

$   cd /home/brain_annex   [CHANGE the directory name, if used a different one]
$   source venv/bin/activate


(venv) $   python3 initialize_installation.py
(to create a new app user, it will prompt you for a new username, a desired password, and an optional email)

---

Use .\venv\Scripts\Activate.ps1 (assuming you are already in the right folder, as stated above; there should be a "venv" subfolder.)
If you get a policy error, first run:
Set-ExecutionPolicy -Scope Process -ExecutionPolicy RemoteSigned
Then run: python initialize_installation.py

Test the Login Functionality

Go the the login page from the earlier section. (The specific URL will depend on your installation specifics.)

Log in using any of the accounts that you created in the previous section

ALL DONE!

Congratulations, now you can start using the various control panels offered by the Brain Annex web app.
If you also ran the optional earlier step "Create Database Schema", you will in addition have access to the multimedia content-management capability of Brain Annex.

If, at a later date, you upgrade the software, refer to the instructions in "Part 3", below.

PART 3 – Routine Deployments (UPDATES from changes in code base)

NOTE – this routine deployment updates ALL of : the python libraries, the web API, and the web app from the main branch of the BrainAnnex repository, which is often slightly ahead of the PyPI distribution for the various versions of the brainannex python libraries.

Stop the Web App

On LOCAL machine: simply stop Flask, if running.


On SERVER: Use PuTTY or other client (we recommend the free MobaXterm) to SSH to the server.

* Stop Gunicorn (or Flask's built-in server, whichever you're using)

if using Gunicorn (with or without Caddy):

* Note: no need to stop the Neo4j database

Update the code base from the latest release

You may use GitHub Desktop to pull the latest version from the main branch of the repository.

IF NEEDED (not always): if this upgrade involves changes in the required libraries in requirements.txt (see changelog), then one needs to perform library installs/uninstalls in venv as needed (SAFE TO DO EVEN IF NOT NEEDED!)

If you're on local machine and using an IDE such as PyCharm, you should get prompted if you change your requirements.txt file.

On Linux:

IF NEEDED (usually not): if you have existing data and/or media, follow the MIGRATION PROTOCOL, if any, for the current upgrade

Check any special instructions in the Change Log, for the current release, as well as any release between the latest one and the one you're upgrading from.

Infrequently needed, but there have been cases of needed file changes and/or Cypher queries to manually run (using the Neo4j browser interface) as part of the upgrade, to migrate your existing database to the new version of the BrainAnnex web app.

Final Steps

Finally, proceed as shown in the earlier section Starting Brain Annex