"DEV" branch

in-progress

Beta 48.2

NEW RELEASE IN PROGRESS.  NOT RECOMMENDED TO USE, until this sign gets taken down.

MIGRATION PROTOCOL from past beta versions:

• Images now utilize an extra property, "comments".  You can add it by means of the Schema Manager (under Admin, in the UI).
  Under "ADD A PROPERTY TO AN EXISTING CLASS", in the "Edit Schema" section, add a new property named  comments  to the  Images  class.
   
• For consistency, the Class names  Images, Notes , Headers  and  Categories  are being changed to their respective singulars.
  Issue the following commands (e.g. from a python script or Jupyter notebook, assuming you have already done a  pip install brainannex) :
• from neoaccess import NeoAccess
  from brainannex import NeoSchema
  
  db = NeoAccess(host=host,
                 credentials=("neo4j", password))   # CHANGE host and password
  # Examples of host:  bolt://123.456.789.012   OR   neo4j://localhost 
  # (don't include the port number)
  
  NeoSchema.set_database(db)
  NeoSchema.rename_class(old_name="Images", new_name="Image",                                               rename_data_fields=True)
  NeoSchema.rename_class(old_name="Notes", new_name="Note",                                               rename_data_fields=True)
  NeoSchema.rename_class(old_name="Headers", new_name="Header",                                               rename_data_fields=True)
  NeoSchema.rename_class(old_name="Categories", new_name="Category",                                               rename_data_fields=True)

Sep. 14, 2024

Beta 47.1

NEW FEATURES

• Enhancement to editing of document metadata in UI

INFRASTRUCTURE

• The default subfolder for documents got renamed from  document  to  documents 

PYTHON LIBRARIES
 

• Collections:
  Renamed argument "membership_rel_name" to "membership_link_name"
   
• Categories:
  New method  reassign_positional_values()
  Enhancements to  check_all_categories_for_duplicates()
  Changes to Categories.get_parent_categories(),  count_parent_categories()  and  count_subcategories() to do without the now-obsolete "schema_code" fields.
   
• NeoSchema:
  New method  namespace_exists()
  Changes in return value of  advance_autoincrement()
  Changes to  data_node_exists(), including new argument "id_key"
  Enchancements to  count_data_nodes_of_class()

BUGFIX

• Bugfix, and enhancements, to  Categories.check_for_duplicates()
• Bug fix to  Collections.link_to_collection_at_end()
• Bugfix to  "modules\CKeditor\config.js"  to manage change in MathJax CDN

• It seems necessary to explicitly clear the browser cache, if Firefox is being used.  In case of problems with MathJax (the formula formatter), inspect the file  YOUR_DOMAIN/BA/pages/static/modules/CKeditor/config.js  and verify that the top line mention a 9/14/2024 modification.  If not, clear the browser cache.
   
• The special "schema_code" field is no longer needed on Data nodes.  It can be dropped from the entire database:

• The default subfolder for documents (a subfolder of the  MEDIA_FOLDER  specified in the  config.ini  file for the app) need to be renamed from  document  to  documents 
  If the media is on a server, an FTP client such as FileZilla may be used.  Alternatively, one could SSH into the server, navigate to the media folder, and then issue the shell command :  mv document documents

Sep. 5, 2024

Beta 46.4

NEW FEATURES

• The new plugin for tabular visualization of "recordsets", first introduced in version Beta44 (Vue component 'vue-plugin-rs'), has reached full basic functionality :
  - addition of new recordsets
  - editing of existing ones
  - addition of new records 
  - (but not yet editing of individual records)
   
• Recordsets now automatically show up in the Table-of-Contents navigation column on the left.
  Table headers get modified for ease of getting automatically broken up into multiple lines.
  The display of wide tabular Recordsets offers horizontal scrolling.
   
• Improvements to record-search feature (available thru the UI)

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• New media items can now be inserted at arbitrary positions on Category pages
   
• Added "favicon" to site (icon displayed on browser tabs)
   
• Re-introduced the "dragscroll" JS library, to facilitate horizontal scroll of wide tabular data

INFRASTRUCTURE

• Phaseout of "schema_code" field being stored on Data nodes;
  accordingly, necessary changes in classes  Categories, DataManager, NeoSchema, ApiRouting, PagesRouting
   
• Replaced the identical temporary URI's assigned to all new Content Items, with individual auto-incremented ones; 
  updated all Content-Item Vue components to conform
   
• New file "PLUGINS/plugin_support.py"
  
  PLUGINS:
• "Notes" : dropped one argument in  Notes.before_update_content()
• "Headers" : when in editing mode in the UI, hide the tagging option
• "Documents" : document deletion can now deal with non-default locations of the media files

WEB-APP FRONT END

• Updated "vue-content-items" , "page_viewer.htm", and and most plugin-related Vue components to pass as a parameter the class name
   
• The Vue component "vue-controls" accepts an extra prop (argument): "controls_to_hide"

JavaScript LIBRARIES

• server_communication.js:
  Several changes to server_communication.js , incl. new methods  ServerCommunication.handle_fetch_errors_JSON()  and  sanitize_data_object()
  Tightened handling of edge cases in  contact_server_NEW()  and  parse_data_object()

• Added handling of an edge case to the functions  render_cell()

WEB API

• New web API endpoints  /add_item_to_category_JSON  ,  /update_content_item_JSON  and  /create_data_node_JSON

• Updated web API's  /delete  and  /update_content_item 

• Updated function  delete_content_item()  in "page_viewer.htm"

• Dropped web API endpoint  /update ; changed accordingly several Vue components and "category_manager.htm"

PYTHON LIBRARIES

• UploadHelper:
  Tweaks to  secure_filename_BA()
   
• MediaManager:
  Overhaul of this class
   
• DataManager:
  New methods  add_new_content_item_to_category()  and  create_data_node()
  Expanded methods  add_new_content_item_to_category() 
  Expanded method  update_content_item_NEW()  and renamed it  update_content_item()
  Dropped methods  lookup_media_record()  and  update_content_item()
  Dropped "schema_code" argument from  get_text_media_content()
   
• NeoSchema:
  New methods  get_data_node()  ,  assign_namespace_to_class()  ,  data_link_exists() ,  lookup_class_namespace()  and  generate_uri()
  Method  is_link_allowed()  now takes into account possible intermediate hops thru  "INSTANCE_OF" links
  Added new argument to  data_node_exists()
  Added 2 arguments ("class_from" and "class_to") to  import_pandas_links()
  Renamed 2 of the arguments in  follow_links(), and expanded functionality
  Renamed  get_data_node()  to  search_data_node() 
  Renamed argument "id_type" to "id_key" in  class_of_data_node()

BUGFIX

• Bug fixes to  Categories.swap_content_items() , Categories.get_category_info()  and to  NeoSchema.allowable_props()

DOCUMENTATION

• Updated "tutorial_neoschema_2_import.ipynb"  (in notebooks)

MIGRATION PROTOCOL from past beta versions:

To make use of the new "recordsets" plugin, the database Schema needs to be expanded.  The simplest method is to issue the following commands (e.g. from a python script or Jupyter notebook, assuming you have already done a  pip install brainannex) :

Aug. 13, 2024

Beta 45

NEW FEATURES

• Documents may now be uploaded to, and stored in, user-specified locations, beside the standard default folders
• Expanded document metadata.  That metadata, including file names, is now editable thru the UI
• Multi-letter name filtering for Category selection in the Navigation bar
• The "recordsets" UI visualization now also show the total number of nodes

INFRASTRUCTURE

• For consistency, the Class name  Documents  got changed to the singular,  Document

JavaScript LIBRARIES

• server_communication.js:
  Enhancements to  parse_data_object()

PYTHON LIBRARIES

• DataManager:
  New method  get_records_by_class()
   
• MediaManager:
  New methods  retrieve_full_path()  ,  lookup_media_file(),  before_update_content() ,  move_file() , check_file_name() ,  is_media_class()
  Deprecated  lookup_file_path()
   
• Categories:
  Added URI namespace option to several methods used for addition of new Content Item at various locations on a Category-viewer page

MIGRATION PROTOCOL from past beta versions:

• For consistency, the Class name  Documents  is being changed to the singular,  Document .
  Issue the commands (e.g. from a python script or Jupyter notebook, assuming you have already done a  pip install brainannex) :
• from neoaccess import NeoAccess
  from brainannex import NeoSchema
  
  db = NeoAccess(host=host,
                 credentials=("neo4j", password))   # CHANGE host and password
  # Examples of host:  bolt://123.456.789.012   OR   neo4j://localhost 
  # (don't include the port number)
  
  NeoSchema.set_database(db)
  NeoSchema.rename_class(old_name="Documents", new_name="Document",                                         rename_data_fields=True)
• The default subfolder for documents (a subfolder of the  MEDIA_FOLDER  specified in the  config.ini  file for the app) need to be renamed from  documents  to  document 
  If the media is on a server, an FTP client such as FileZilla may be used.  Alternatively, one could SSH into the server, navigate to the media folder, and then issue the shell command :  mv documents document

Aug. 9, 2024

Beta 44.1

NEW FEATURES

• Enhancements to the record-search feature (file "filter.htm") located on "Seach" page

WEB API

• Eliminated endpoint  /get_filtered_OLD
  
   

BUGFIX

• The new "recordsets" plugin now handles correctly different data types, including datetime; changes in  DataManager.get_nodes_by_filter()

Aug. 7, 2024

Beta 44

NEW FEATURES

• Class names displayed in the Schema Viewer are now clickable - and the corresponding nodes in the graph get selected
• Early implentation (for now read-only view) of a new plugin for tabular visualization of "recordsets" ("rs")

WEB-APP FRONT END

• New Vue component 'vue-plugin-rs'
• Enhancements to Vue component 'vue_cytoscape_2' 

JavaScript LIBRARIES

• server_communication.js:
  Light cleanup to class  ServerCommunication
  New function  contact_server_NEW()
   

WEB API

• New endpoint   /get_class_properties
• Revised endpoint  /get_filtered

PYTHON LIBRARIES

• DocumentationGenerator:
  Misc. enhancements to  generate_documentation()
  New method  clean_up_args()
   
• Categories:
  Added argument "exclude_system" to  get_items_schema_data()
   
• DataManager:
  Bugfix, and expansion, for   get_nodes_by_filter()
  Bugfix to  to_int_if_possible()
  New method  _process_order_by()
   
• NeoSchema:
  import_pandas_nodes()  can now handle duplicate records; added arguments  "primary_key" and  "duplicate_option"
  New methods  rename_class()  and  rename_property()
  Added arg "link_properties" to  create_class_relationship()
  Expanded  add_properties_to_class()
  Tweaks to  get_class_properties()

July 30, 2024

Beta 43

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• The UI permits editing the caption fields in images and documents

WEB API

• New endpoint   /update_content_item

PYTHON LIBRARIES

• DataManager:
  New method  update_content_item_NEW()
   
• Categories:
  New method  import_ontology()
  Added argument "category_uri"  to  add_subcategory()
   
• NeoSchema:
  New method  create_namespace()
  2 new arguments, "drop" and "rename" added to  import_pandas_nodes()
  Tweaks, and new argument "class_name", to  update_data_node()
  Renamed  fetch_data_node()  to  get_data_node()

BUGFIX

• Fixed issue preventing SEE_ALSO links from showing up in Category-viewer pages

July 18, 2024

Beta 42

(note: 41 was not released)

NEW FEATURES

• All Class names get shown in the legend, on the Schema Viewer page 

DEPLOYMENT

• First PyPI release of  brainannex  (all foundational libraries, below the "Flask" layer).
  Simply do  pip install brainannex   (currently, also need to  pip install requests==2.25.1 , but that won't be necessary in the next release)
   
• Fixed bcrypt version to 4.1.2 and jsonschema (used by jupyterlab) to 4.17.3 (later versions bring incompatibility)

PYTHON LIBRARIES

• Categories:
  New method  get_see_also()
  Dropped old method  follow_see_also()

• DataManager:
  Added 2 arguments to  upload_import_json()  and to  upload_import_json_file()
   
• UploadHelper:
  Changed argument "request_obj" to "files" in  store_uploaded_file() , and dropped argument "verbose".
  Moved method  get_form_data()  to class  ApiRouting
   
• NeoSchema:
  New methods  import_triplestore()  and  follow_links()

WEB API

• New utility method  ApiRouting.explain_flask_request()

WEB-APP FRONT END

• Enhancements to Vue component 'vue_cytoscape_2' 
• Completed implementation of the SEE_ALSO link display and creation, in the Category Manager page
• Early partial implementation of deletion of existing SEE ALSO links
• New JavaScript file  "utilities.js" , with static method  find_item_by_uri()

DOCUMENTATION

• Updated all tutorials (in notebooks)

July 9, 2024

Beta 40

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• "SEE_ALSO" links can be added thru the Category manager, in the web app (partial implementation)

INFRASTRUCTURE

• Completion of major re-organization of project structure, for the imminent PyPI release
• Re-organization of all Flask-related files - now all moved under the folder "flask_modules" at the top level
• Re-organization in standard site parameters being passed to Flask template pages
• Added function  version()  in  "brainannex\__init__.py"

PYTHON LIBRARIES

• Categories:
  New methods   create_see_also()  and  remove_see_also()
   
• NeoSchema:
  General re-organization
  New method  is_strict_class() ; the existing method by that name, got renamed  to  is_strict_class_fast()
  New method  is_link_allowed()
  Expanded method  add_data_relationship(), to add new argument "id_type" and more validation
  Ditched obsolete method  add_data_relationship_OLD()
  In method  remove_data_relationship(), renamed argument "from_uri" to "from_id", and "to_uri" to "to_id"; added argument "id_type"

WEB API

• New web API endpoint  /see_also

WEB-APP FRONT END

• Expanded the functionality of the Flask (Jinja) template   "navigation\templates\horiz_status_bar.htm"
• Site branding name, username and  Brain Annex version number are now displayed in more locations across various page

BUGFIX

• Bugfix to record-related Vue.js components

July 3, 2024

Beta 39.1

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Enhancement to the security of storage of login credentials.  User passwords are now fully encrypted prior to being stored in the database (and also contain a random portion, called a "salt", for extra protection)

INFRASTRUCTURE

• New sample Schema file  "MINIMAL_SCHEMA_TO_IMPORT.json"
• New script  "initialize_installation.py"
• Start of re-organization of project structure, ahead of upcoming PyPI release
• Light tweak to  "main.py"
• Dropped file "VERSION_NUMBER.txt".  Expanded  "brainannex/__init__.py"
• New pip import in  "requirements.txt"  and  "environment.yml" : bcrypt 4.1.3  (for password management)

PYTHON LIBRARIES

• New class "UserManager"; switched over user logins to it
• The old  "home\user_manager.py"  is now renamed  "login_manager.py"
• Renamed class "UserManagerNeo4j" to "FlaskUserManagement"
• Upgraded to newer version of class "PyGraphVisual" from sister project Life123.science
   
• NeoSchema:
  Renamed argument "property_list" of  create_class_with_properties()  to "properties"
  New method  create_schema_from_sample_data()
  Method  _advance_autoincrement()  renamed to  advance_autoincrement()

DOCUMENTATION

• Expansion and re-organization of the Download/Install Guide

BUGFIX

• Bugfix to script  "db_connection_3_FROM_CONFIG.py"

MIGRATION PROTOCOL from past beta versions:

• All user login accounts must be re-issued.   See the updated section "Create user login account" on the Download/Install Guide, for manual single-user account creation.  Repeat for every user in the old system.  Old credentials may be retrieved with the following Cypher query (watch out for hidden characters when copying-and-pasting from a web page to the Neo4j browser!):

If bulk creation of users is needed, consider running a script along the lines of:

When you no longer need the old credentials in the database (created by the old storage mechanism), it's recommended that you delete them with the Cypher query:

Note that with this new beta release, the storage of credentials in the database is now in encrypted form, using a different node label  (User)

May 17, 2024

Beta 38.3

BUGFIX

• Fixes to file "SAMPLE_SCHEMA_TO_IMPORT.json"

DOCUMENTATION

• Video on  Full-text-indexing and searching

Apr. 30, 2024

Beta 38

NEW FEATURES

• Allow restriction of search results to Content Items that are linked to the descendants of a specific Category (in libraries and in the UI)
• In UI:  added a TOC (Table of Contents) to the left sidebar of the search-results page.  Note: in the UI, the search page's URL is at   /BA/pages/filter

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Search for multiple words inside Documents or Notes
• The TOC (Table of Contents) on the left sidebar of the Category viewer now shows Documents, too

PYTHON LIBRARIES

• DataManager:
  New methods  search_for_terms()  and  search_for_all_words()
   
• FullTextIndexing:
  Words that contain a digit (but not in start position) are now indexed; previously, they were ignored.
  Added argument "restrict_search" to  search_word()
   
• Collections:
  Correction to return value in  initialize_collections()
  Additional argument validations.
   
• NeoSchema:
  New method  assign_uri()
  Strengthened the use of namespaces for auto-increment URI generation; several methods now have an argument named "uri_namespace"
  Turned  advance_autoincrement()  into a private method,  _advance_autoincrement()

   

WEB-APP FRONT END

• Increased max filesize that Dropzone (the interactive file uploader) allows to upload, now up to 2 GB

VUE.js COMPONENTS

• Added new argument ("show_hide_details") to the Vue component  vue-toc-sidebar

Mar. 24, 2024

Beta 37

NEW WEB-APP FEATURES

• Schema Viewer  (Under  Admin » Schema Manager), which offer multiple styles of visualization of the Schema-related part of the graph database

RE-IMPLEMENTED WEB-APP FEATURES PORTED FROM v. 4

• Restored bulk-edit capability on Category-viewer pages: bulk-relocate (re-tag) Content Items to a different Category

PYTHON LIBRARIES

• New class PyGraphScape, for general graph-database visualization  (utilized by Schema Viewer)
   

• DataManager:
  New method  DataManager.switch_category()

• Collections:
  New methods  relocate_to_other_collection_at_end()  and  bulk_relocate_to_other_collection_at_end()
  The arguments of  link_to_collection_at_end()  now use URI's instead of internal database ID's
  Renamed  create_collections_class()  to  initialize_collections()
   
• Categories:
  New methods  initialize_categories()  and  relocate_across_categories()
  Namespace change in  create_categories_root()
  Changes to  get_content_items_by_category()
  Dropped "label" argument from  link_content_at_end()
  Partial phase out of obsolete convention about root Categories nodes
   
• NeoSchema:
  New method  is_property_allowed()
  Correction to return type of  create_class_with_properties()
  Updated file  SAMPLE_SCHEMA_TO_IMPORT.json

WEB API

• New web API endpoints  /switch_category  and  /schema-viewer
• Added argument "json_decode" to  ApiRouting.extract_post_pars()

WEB APP PAGES

• New page template "schema_viewer.htm"

VUE.js COMPONENTS

• Renamed prop (argument) "allow_editing" to "edit_mode" across all Vue components

• New Vue components "vue_cytoscape_1" and "vue_cytoscape_2", witch CSS files

DOCUMENTATION

• New article  Visualization of Graph Databases Using Cytoscape.js
• New article  Graph Database Data Modeling - ORDERED Sequences
• Expanded README file for NeoSchema library

MIGRATION PROTOCOL from past beta versions:

The above database modification is necessary to phase out the obsolete old convention about root  Categories  nodes.

Feb. 12, 2024

Beta 36

NEW WEB-APP FEATURES

• Double-click a record in any tabular listing, to enter a limited ("in place") editor

RE-IMPLEMENTED WEB-APP FEATURES PORTED FROM v. 4

• Completed the front-end implementation of Category tagging for Content Items, from the Category-viewer pages  [that was started in version Beta 35.1]
• Overhauled the layout of Content Items on Category-viewer pages, to allow display of multiple Content Items on a line
• Implemented "add new subcategory" feature in Category Manager

PYTHON LIBRARIES

• Categories:
  Bugfix to  get_sibling_categories()
  New method  get_categories_linked_to_content_item()
   
• NeoSchema:
  Bugfix to  unlink_classes()
  Dropped argument "assign_uri" from  create_data_node()
  Renamed method  next_available_schema_uri()  to  _next_available_schema_uri()  (i.e., made private)

WEB API

• Change in the payload returned by  /get_categories_linked_to/
• New web api  /get_categories_linked_to/<item_uri>

WEB APP PAGES

• Bugfix affecting Subcategories newly added in the Category-viewer pages

VUE.js COMPONENTS

• Bugfix in general record editor (Vue.component  vue-plugin-r)

• Adjustment to page layout of general records during editing

• Simplifications to  records.js , content_items.js  and table.js

DOCUMENTATION

• Vastly expanded the User Guide for the NeoSchema layer
• Additions to the Download/Install page

Jan. 27, 2024

Beta 35.1

RE-IMPLEMENTED WEB-APP FEATURES PORTED FROM v. 4

• Partial front-end implementation of Category tagging for Content Items, from the Category-viewer pages

PYTHON LIBRARIES

• FullTextIndexing:
  Added more stop-words

• DataManager:
  Changes to  new_content_item_in_category()

• Collections:
  New methods:  set_database()  and  create_collections_class()
  Tweaks to  is_collection()

• Categories:
  New method:  set_database()
  Bugfix in  add_subcategory()
   

• NeoSchema:
  Change from an integer "schema_id" to a string "uri" as identifier for Schema nodes
  New method  reserve_next_uri() ,  to replace  generate_uri()
next_autoincrement()  renamed to  advance_autoincrement()
  Added argument "label" to  data_nodes_lacking_schema()
  Added validation to  set_database()
get_class_relationships()  now accepts as argument a Class name rather than a URI

WEB API

• Changed web API endpoint  /get_links/  to use Class name rather than URI's
• Bugfix in  ApiRouting.add_subcategory()

WEB APP PAGES

• In the  page_viewer.htm  template, renamed "category_id" to "category_uri"

VUE.js COMPONENTS

• Added close button to the "insert new content below" box (on Category-viewer pages)

• Changes to Vue components  vue-content-items  and  vue-controls

PLUGINS

• Partial implementation of "COPY TO CLIPBOARD" control for Notes

DOCUMENTATION

• Update and expansion of various Reference Guides (listed in the Guide page)
• Overhauled the home page of BrainAnnex.org , and added a new page Tech Stack
• New User Guide for the NeoSchema layer

MIGRATION PROTOCOL from recent beta versions:

1. MATCH (n:CLASS) SET n.uri = "schema-" + n.schema_id 
2. MATCH (n:PROPERTY) SET n.uri = "schema-" + n.schema_id 
3. MATCH (n:CLASS) REMOVE n.schema_id
4. MATCH (n:PROPERTY) REMOVE n.schema_id
5. MATCH (n:`Schema Autoincrement` {namespace: "schema_node"}) SET n.prefix = "schema-"

The above database modifications are necessary to conform existing Schema data to the changes made in the  NeoSchema  class in this release.

Jan. 13, 2024

Beta 34

RE-IMPLEMENTED WEB-APP FEATURES PORTED FROM v. 4

• Partial implementation (read-only) of the "SEE ALSO" links for Categories
   

PYTHON LIBRARIES

• FullTextIndexing:
  More fine-grained error catching and reporting in  update_indexing()

• DataManager:
  Moved  get_binary_content()  to class MediaManager
  Phased out the obsolete method  get_records_schema_data()

• Collections:
  Split off the "Collections" class into its own file, collections.py, from its former location in
  categories.py ; changed initialize.py accordingly
  New method:  link_to_collection_at_end()
  Added validation to  link_to_collection_at_end()

• Categories:
  New method:  link_content_at_end() ; used it to simplify  add_content_at_end()
  New methods:  detach_from_category() , follow_see_also()
  Added argument "label" to  link_content_at_end()
  Changes in  get_items_schema_data().  
  Change in return value of  create_categories_root()
  Phased out and dropped the obsolete  get_subcategories_alt() and get_parent_categories_alt()
  Tweak to  get_subcategories(), incl. switch of keys from "id" to "uri"
  Substantial expansion of pytests for Categories

• ImageProcessing:
  Bugfix in upload of SVG images; changes to  process_uploaded_image()

• NeoSchema:
  New method:  set_property_attribute()
  Obsoleted and eliminated method  get_class_properties()
  Method  get_class_properties_fast()  got renamed  get_class_properties() , and added a new argument "exclude_system".  More pytests.
  Enhancements to  import_pandas_links(), to deal with NaN values in relationship properties

WEB API

• New web API endpoints:  /link_content_at_end/  and   /detach_from_category/
• Login is now required for all the web api endpoints, except test ones
• Ditched unused web endpoint  /get_properties/<schema_id>
• Reorganization of class  ApiRouting

WEB APP PAGES

• Bugfix to  viewer_static.htm (SiteLinks are now included)

VUE.js COMPONENTS

• Bugfix in Vue component  vue-category-navbox , to correctly handle multiple Categories with the same name

PLUGINS

• Tweak to CSS of  "Notes" (PLUGINS\notes\notes.css)

BUGFIX

• "system" properties are now hidden from user during edits,
  involving changes in web endpoint  /get_class_schema

DOCUMENTATION

• Enhancements to the Installation/Setup page
• Update and expansion of various Reference Guides (listed in the Guide page)

Jan. 3, 2024

Beta 33

PYTHON LIBRARIES

• FullTextIndexing:
  Renamed  populate_index()  to  add_words_to_index(), and made it far more efficient - incl. switching it to use the improved new version of  NeoSchema.add_data_column_merge()
  Leading/trailing underscores are now dropped in  extract_unique_good_words()
  Expanded list of common words to skip.
  Tweaks to  initialize_schema()
  New argument "uri" to  count_indexed_words(), now renamed  number_of_indexed_words()
  Misc. argument renaming in various methods of  FullTextIndexing

• Categories:
  Bugfix to  get_content_items_by_category(), to deal with Neo4j datetime types.

• DataManager:
  Bugfix in search_for_word()  to deal with Neo4j datetime values.
  get_binary_content()  now also handles cases when the thumbnail is missing.

• NeoSchema:
  New methods:  import_pandas_nodes(),  import_pandas_links(), 
  data_node_exists(), is_valid_class_name(), assert_valid_class_identifier(), add_data_relationship_hub() .
  Complete redesign of   add_data_column_merge() , with major speed improvement.  
  Overhauled  add_data_node_merge()   and  class_relationship_exists() .
  Re-designed  create_data_node()  to make use of new helper function _create_data_node_helper() .
  Renamed   get_class_name()  to  get_class_name_by_schema_id() ; renamed get_class_name_by_neo_id()  to  get_class_name()
  Added argument "use_link_node" to  create_class_relationship() ;
  renamed argument "schema_id" to "class_node" in  get_class_properties() .

PLUGINS

• Streamlining of Notes editing, in front- and back-end,
  incl. several safeguards for integrity; started to use the "basename" field.
   
• Renamed Notes.update_content()  to  before_update_content()

INFRASTRUCTURE

• Moved  ImageProcessing  class from  upload_helper.py  to  media_manager.py

DOCUMENTATION

• New "NeoSchema library" tutorial 2 - Setting up a Schema and importing Pandas data frames into Neo4j
   
• Updated notebook "tutorial_indexing_1"

RECOMMENDED DATABASE CHANGES:

Version 4.4 of Neo4j (the version currently supported by BrainAnnex) seems to have major problems with the recently-introduced database Index of type "TEXT".  Use the old "BTREE" type of indexes only!

1. Run the following Cypher query to see what database indexes you have:

2. If you have any indexes of type "TEXT", delete them with commands such as:

3. Create new indexes of the default type (which is "BTREE").  Of particular importance if indexing documents and notes:

Dec. 18, 2023

Beta 32

RE-IMPLEMENTED WEB APP FEATURES PORTED FROM v. 4

• "Pinning" of Categories (in the "Category viewer" page)
   

PYTHON LIBRARIES

• NeoSchema:
  New method  is_valid_uri()
  Added argument  "prefix" to  next_available_datanode_uri()
   
• Categories:
  New methods:  pin_category()  and  is_pinned()
get_all_categories()  now also returns the pinned status of the Categories.
  Ditched deprecated method  get_all_categories_alt()
  Standardized error handling in swap_content_items()
  Change in the argument of  delete_category()
   
• DataManager:
  New methods:  add_data_relationship_handler() , remove_data_relationship_handler() , extract_website_title()
  Better validation in  update_content_item()
  Improved error checking/reporting in  delete_content_item()
   
• DocumentationGenerator:
  Separated the class "DocumentationGenerator" into its own file.  Formerly, it was inside "data_manager.py" ; now it's in  "documentation_generator.py"
  Enhancements to method  generate_documentation()

JavaScript LIBRARIES

• server_communication.js:
  Massive simplification of this library.
  Dropped the option of payload type "TEXT" , in favor of the remainig type "JSON".
  Enhancements to  extract_server_data_JSON()

   

INFRASTRUCTURE

• Complete phaseout of the deprecated "simple" protocol from the class  ApiRouting.
  Dropped  /simple/  from the URI of many web endpoints.
  Major re-organization and streamlining of  the class  ApiRouting.
  New method  extract_get_pars()

• Global switch of data URI's from int to strings : all "uri" fields in datanode got converted from int to string.

• The "category_manager.htm" page now receives the "pinned" status from Flask (PagesRouting); renamed Flask variable "category_id" to "category_uri"
   
• On "page_viewer.htm", the Flask variable  "all_categories"  now uses the key "uri" in lieu of "id": changed some Vue components and  the class PagesRouting accordingly
   
• Simplification to Vue component  vue-toc-sidebar  for the Table of Contents of the Category-viewer pages.
   
• Vue component  vue-category-navbox  now shows pinned categories.
   
• New web API :  /pin_category/<uri>/<op>
   
• Dropped unused argument "safe_html" from  exception_helper() , in "exceptions.py"

MIGRATION PROTOCOL from recent beta versions:

1. Run the following Cypher query (if you have any Content Items; no harm in running it in any case):

2. To verify that all the "uri" properties on data nodes now contain strings, run he following Cypher query:

You should see "STRING"

Oct. 27, 2023

Beta 31

NEW WEB-APP FEATURES

• Full-text indexing for PDF and EPUB files (as well as some other formats); now they are fully searchable
   
• (UI)  The option to double-click Content Item for editing them, was extended to also include Notes and Headers
   

LIBRARIES

• FullTextIndexing:
  Added the  pymupdf  library (for parsing of PDF, EPUB, etc.)
  PDF documents (and some other formats) are now included in parsing and indexing
  Misc. tweaks to full-text indexing
  Expanded the list of common words to exclude from indexing
  Added "drop_html" argument to two indexing methods
  More pytests

• NeoSchema : renamed "item_id" to "uri" throughout the library

• Upgraded the  pillow  library

INFRASTRUCTURE

• Replaced all occurrences of the name "item_id" with "uri" throughout all files and notebooks
   

DOCUMENTATION

• New article  What are Graph Databases - and Why Should I Care?? : "Graph Databases for Poets"
   
• Cleanup of the Installation Guide
   
• Update, and major expansion of the table of contents, of the NeoSchema Reference Guide
   
• New Reference Guide for the Data Manager Layer

MIGRATION PROTOCOL from recent beta versions:

1. Run the following Cypher query:

It should return a count of zero.

2. Run the following Cypher update query:

 It will replace all occurrences of the property (i.e. field name) "item_id" to "uri"

Sep. 27, 2023

Beta 30.1

NEW FEATURES

• Substantial enhancements to the automated generation of HTML documentation pages from imported python files.  ("DocumentationGenerator" class)

INFRASTRUCTURE

• Moved the "DataManager" class to its own "data_manager" folder
• Updated and corrected the sample file  "SAMPLE_SCHEMA_TO_IMPORT.json"

LIBRARIES

• Switched to  NeoAccess 4.4.1  (see separate section, below)
   
• NeoSchema:
  In the data modeling for Schema Classes, replaced the text attribute "type" with bool attribute "strict".

DOCUMENTATION

• New video  Neo4j web app - BrainAnnex Status Report
   
• Updated and corrected the Programmer's Guide
   
• Updated and corrected the Installation Guide

MIGRATION PROTOCOL from recent beta versions:

1. If you were using Schema Classes, run the following 2 Cypher update queries:
   MATCH (n :CLASS)
   WHERE n.type = "S"
   SET n.strict = true
   REMOVE n.type
2. MATCH (n :CLASS)
   WHERE n.type = "L"
   SET n.strict = false
   REMOVE n.type

NeoAccess library (independently distributed):  Sep. 21, 2023  v. 4.4.1

LIBRARY

• Substantially expanded the capabilities of the method  load_pandas()   [Special thanks to Alexey Kuznetsov]
  New arguments to deal with integration of new records with existing data, and several other scenarios.  Extensive new pytests.
   
• New helper method  pd_datetime_to_neo4j_datetime()   
  [Special thanks to Alexey Kuznetsov]
   
• Redefined the role of the "debug" mode; added a "block_query_execution" mode that shows the automatically-constructed Cypher queries, without actually executing them 

• Change in the value returned by  query()  in case of no records found when the option single_row=True  is used.  Expanded pytests.  [See compatibility notes, below]

• Added argument "order_by" to  get_siblings()

• Tweak to  get_node_internal_id()

• New methods  count_nodes()  and  CypherMatch.extract_where_clause()

• Added validation on  "host"  name when instantiating the  NeoAccess  class

• Early implementation of  import_csv_nodes()   [to be released in future versions]

• Added argument "caller_method", and more validation, to CypherUtils.process_match_structure().   That new new argument is now employed by all its calling functions, to provide better error messages.

DOCUMENTATION

• Substantial expansion and cleanup of the  NeoAccess Reference Guide
   
• New tutorial on importing Pandas data frames into Neo4j

KNOWN BUGS

When started in debug mode, NeoAccess prints out a message that "no Cypher query will actually be executed".  That message is incorrect: the only effects of being in debug mode are extra messages being printed out.  (If wanting to suppress the execution of Cypher queries, use the new "block_query_execution" mode.)

COMPATIBILITY CONSIDERATIONS

The method  NeoAccess.query() , when used with the option single_row=True  , now returns None when no records are located, for consistency with other methods.  Previously, it returned  { }  , i.e. an empty dictionary.

Sep. 14, 2023

Beta 29

NEW FEATURES

• Site Links (saved website bookmarks) now have their own plugin providing full, customized UI support, including porting of features from version 4 (such as auto-fill of website name from its url), and new ones (new layout, and option to double-click for editing)

INFRASTRUCTURE

• The list of registered plugins got moved to the more central location  page_viewer.htm

WEB API

• New web api endpoint   /simple/fetch-remote-title

LIBRARIES

• NeoSchema:
  New method  update_data_node()
  Dropped 2nd return value from method  create_data_node()
   
• DataManager:
  Character-encoding change in  get_text_media_content()

  Improvement to  update_content_item(), to allow blank fields to get completely dropped from the database

  Deprecated  get_records_schema_data()  in favor of the new function Categories.get_items_schema_data()

DOCUMENTATION

• New guide : Server Communication
   
• Updated all tutorials affected by the changes

BUGFIX

• Bugfix to  absorb_item_updates() , in  page_viewer.htm
• Bugfix to web api endpoint  /simple/update  , to allow to completely drop blank fields

Sep. 2, 2023

Beta 28

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Support for management of  documents  (general files) as Content Items.
  Changes to backend, web API and UI, as needed.
   
• Sibling categories shown in page views are now alphabetized

NEW FEATURES

• Documents that are plain-text files get automatically indexed, and their contents are available to the Full-Text Word Search

INFRASTRUCTURE

• Split the media folder into subfolders by content type (currently "documents", "images", "notes")

• In the UI, Content Items with no registered plugin are now dispatched to the Vue.js plugin for "r" (records); i.e., anything without a special UI handler gets treated as a generic record

LIBRARIES

• Major revamp of class  NeoSchema.
  New method  generate_uri()
  Revised   create_data_node(),  which now returns a pair of values; its argument "labels" got renamed to "extra_labels"
  Also revised  next_autoincrement()
  Obsoleted  add_data_point()  in favor of  create_data_node()
  Bugfix to  allowable_props()
  Argument in  allowable_props()  renamed from "class_neo_id" to "class_internal_id"
   
• Bugfixes and tweaks to class  MediaManager.
  Renamed  get_from_file()  to  get_from_text_file(), added new argument, and fixed problem with text file encoding during uploads.
  Adding new media to a page is now done as an atomic operation (to fix the concurrency problem affecting its positioning within "Category" pages)
  Added new argument, "class_name" , to  lookup_file_path()
  Consolidated the responsibility for choosing the directory of the media file : now handled by the new method  lookup_file_path()

• Bugfix and changes to class  Categories
  Tweak to  get_sibling_categories()
  Dropped add_content_media()  and  paths_from_root()
  Bugfix to create_parent_map()

• Tweaks to  DataManager.get_binary_content()

• Tweaks to  utilities/exceptions.py

WEB API

• Major simplification/streamlining of the Web API.
  Eliminated the class  PagesRequestHandler, and absorbed all its methods into the classes   APIRequestHandler  and  Categories .
  Renamed class APIRequestHandler  to  DataManager , and renamed its file from BA_api_request_handler.py  to  data_manager.py
   
• Bugfix and simplication to  ApiRouting.upload_media()
   
• Expanded  UploadHelper.store_uploaded_file(), to also return the MIME type

• Tweaks to the endpoint  /add_label/

DOCUMENTATION

• Updated all tutorials

MIGRATION PROTOCOL from recent beta versions:

1. If you're using the web app, you need to create  notes , images , documents  subfolders inside whichever folder you've designated as your media folder in the Brain Annex configuration file
    
2. Move all your existing media, if any, into one of the above folders, based on their type : all notes are files whose name starts with "notes-" ; all images are the remaining files.  Make sure to also move the  resized  folder (used for images) inside the new  images  folder
    
3. Using the Neo4j browser interface, run the following update query:
   MATCH (n:`Schema Autoincrement` {kind: "schema_node"})
   SET n.namespace= "schema_node", n.next_count = n.last_id + 1
   REMOVE n.kind, n.last_id

4. Using the Neo4j browser interface, also run the following update query:

   MATCH (n:`Schema Autoincrement` {kind: "data_node"})
   SET n.namespace= "data_node", n.next_count = n.last_id + 1
   REMOVE n.kind, n.last_id

5. Log into the Brain Annex web app, and use the Schema Manager (under the "Admin" tab) to create a new class named "Documents", with a single property named "caption"; make sure to request that the new class be made an INSTANCE_OF the "Media" class 
   [Note: future releases will include the expanded schema in the default schema file that can simply be imported into a new installation]

Aug. 14, 2023

Beta 27.2

NEW FEATURES

• Support for Full-Text Word Search
• API endpoint for Full-Text Word Search
• New UI page to show search results, and additional UI support for searches (including search boxes added to various pages)

INFRASTRUCTURE

• Refinements and additional new methods in class  FullTextIndexing , for indexing and searching
• Expansion to  DocumentationGenerator.generate_documentation()

LIBRARIES

• Dropped argument "schema_type" from  NeoSchema.create_class()

• Renamed argument "class_name" to "name" in  create_class_with_properties()

• Experimental new class  NeoSchema.DataNode

DOCUMENTATION

• Updated tutorial about Full-Text Indexing (notebook)
   
• New article  Full-Text Search with the Neo4j Graph Database

Aug. 4, 2023

Beta 26.1

NEW FEATURES

• Support for Full-Text Indexing
• Started Full-Text Indexing for Content Items of type "notes"

INFRASTRUCTURE

• Moved function  exception_helper()  to new file  utilities/exceptions.py
• New method  MediaManager.set_media_folder()
• New methods  Notes.delete_content() and  delete_content_before()
• Added special handling for deleted Content Items of type "notes" to the class  APIRequestHandler
• Simplified names of methods in class  Notes
• Change in return value of  APIRequestHandler.delete_content_item()
   

LIBRARIES

• Major expansion of the class  FullTextIndexing , including pytests
• Major revamp, expansion and cleanup of  NeoSchema , including overhaul of the supporting library  SchemaCache.  Additional pytests
• Restored  jupyterlab  to version 3.4.8
• Restored  requests  to version 2.25.1

DOCUMENTATION

• New tutorials about the Schema layer (notebook) and about Full-Text Indexing (notebook)
• Cleanup of the Brain Annex Download / Install page
• Update and improvements to the guide for the NeoSchema library

BUGFIX

• Bugfix and streamlining to  APIRequestHandler.lookup_media_record()

June 26, 2023

Beta 25.2

NEW FEATURES

• Added "Generate Documentation Page"  (experimental, for python documentation) on the "Data Import" page, reachable under "Admin"

INFRASTRUCTURE

• Separated from general import the special use case of generating python documentation from imported data
• New class "DocumentationGenerator"
• New API endpoint:   /document_python
• Partial implementation of a Time-To-Live approach for logins, to provide a way to let logins expire, if the user is removed from the database
• Renamed class "Home" to "HomeRouting"

LIBRARIES

• Switched to NeoAccess 4.3.1
• Upgraded  jupyterlab  from 3.4.8 to 3.6.4
• Upgraded  requests  from 2.25.1 to 2.31.0
• Removed the  neo4j  library from  requirements.txt  (importing NeoAccess takes care of that dependency)

DOCUMENTATION

• Major cleanup of the README file

BUGFIX

• Bugfix to all notebooks about the import of the  NeoAccess  library
• Corrections to  environment.yml

NeoAccess library (independently distributed):  June 23, 2023  v. 4.3.1

LIBRARY

• Split off a  NeoAccessCore  class from the main class,  NeoAccess
• Introduced new helper classes  NodeSpecs  and  CypherMatch
• Moved some CypherUtils methods to  CypherMatch
• Renamed method CypherUtils.validate_internal_id()  to  valid_internal_id()
• Renamed argument of  NeoAccess.update_query()  from "cypher" to "q"
• Simplified several  NeoAccess  methods  by moving the "compatibility test" (of the "match" objects) into the method  CypherUtils.combined_where()
• Tweaks to constructor of  CypherMatch  class
• CypherUtils.combined_where()  and  combined_data_binding()  now take just 2 arguments, rather than a list of them
• Cleanup in  cypher_utils.py
• Additional pytests
• Bugfix in  NeoAccess.export_dbase_json()

DOCUMENTATION

• Article introducing and explaining the NeoAccess library
• Revised and expanded the existing tutorials (Jupyter notebooks: tutorial_1 and tutorial_2)
• Expanded README

May 12, 2023

Beta 24

INFRASTRUCTURE

• Implemented login-session robustness against webapp restarts (note: if you use gunicorn for deployment, you'll see lots of those!), as well as login-data caching
• For login management , added new method  UserManagerNeo4j.obtain_user_obj() , and  obsoleted  set_user_obj()
• Added tests of setting/clearing site cookies

BUGFIX

• Bugfix in test scripts for database connection,  db_connection_1_MANUAL.py  and  db_connection_3_FROM_CONFIG.py

May 1, 2023

Beta 23.1

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• "Sibling" categories are now shown in the Category-page viewers

INFRASTRUCTURE

• New methods  Categories.create_categories_root() ,  get_sibling_categories() , and viewer_handler()
• Tweak to  PagesRouting.category_page_viewer()
• Changes to  Categories.get_all_categories() and to add_subcategory()
• Changes to  category_navbox.js , category_navbox.css , and  page_viewer.htm
• New pytest file   test_categories.py , with several new pytests for the  Categories  class

LIBRARIES

• Switched to NeoAccess 4.2.0
• Upgraded the  neo4j  library (Neo4j python connectivity library) to 4.4.11
• New method  NeoSchema.get_data_point_internal_id()
• Changes to  NeoSchema.add_data_point(), incl. new argument "class_name"
• Tweaks to  NeoSchema.get_class_attributes()  and to SchemaCacheExperimental.get_cached_class_attrs()
• Renamed NeoSchema.add_data_point() to add_data_point_OLD(), and add_data_point_new() to add_data_point()
• Renamed  NeoSchema.next_available_id()  to  next_available_schema_id()

DOCUMENTATION

• Much-expanded section on Download / Install for Brain Annex

BUGFIX

• Bugfix in data file  Schema_MINIMAL_plus_root_category.json

NeoAccess library (independently distributed):  Apr. 29, 2023  v. 4.2.0

• Upgraded the neo4j library dependency to 4.4.11
• Added new method NeoAccess.get_siblings(), with pytests
• Renamed argument "params" of  query_extended() to "data_binding" (for consistency with other methods)
• Renamed argument "node_id" to "internal_id" in method  get_parents_and_children()

Apr. 27, 2023

Beta 22

LIBRARIES

• Switched to NeoAccess 4.1.0
• The NeoAccess library is now independently distributed with its own repo , and can simply be pip installed from the standard PyPI distribution. 
  It is now treated as an external library requirement (to pip install) for the overall BrainAnnex web app
   

BUGFIX

• Bugfix in experimental class  NodeExplorer

 NeoAccess library (independently distributed from now on):  Apr. 27, 2023  v. 4.1.0

• Bugfilx, and tweaks, to  NeoAccess.import_json_dump().  New optional argument "extended_validation".  Additional pytests for it
• Replaced "neo4j_id" with "internal_id", for consistency with other libraries, throughout the  NeoAccess  library
• Pytests for  NeoAccess.exists_by_internal_id()

Apr. 19, 2023

Beta 21

NEW FEATURES

• Provided support for app branding thru a new config parameter, "BRANDING"
• Expanded the Schema Manager to allow adding a new Property to an existing schema Class.   Changed the UI and the API accordingly

INFRASTRUCTURE

• Tweak in argument passing from the main file to the initalization  setup()  methods

LIBRARIES

• Renamed  NeoSchema.get_class_neo_id()  to  get_class_internal_id()  for consistency with the  NeoAccess  library

DOCUMENTATION

• Continued the migration of parts of the README file to the  Programmer's Guide

BUGFIX

• Bugfixes in  PagesRouting.category_page_viewer()  and in the script   db_connection_3_FROM_CONFIG.py 

• Newly-created data nodes of schema "Categories" now also get a "Categories" label

• Correction to json files with the recommended schemas to import into new installations

Apr. 12, 2023

Beta 20.2

LIBRARIES

• NeoAccess 4.0.3 , now available directly from the Python Package Index (PyPI) repository.  You can just pip install  neoaccess – which will take care of dependencies – and start using it, as an independent library, if you don't need the full BrainAnnex technology stack
   

DOCUMENTATION

• Took out from the README file the parts now better covered in the  Programmer's Guide

Mar. 22, 2023

Beta 20.1

INFRASTRUCTURE

• Added JupyterLab support, incl. environment.yml
• Provided support for trying it LIVE at Binder, with 1 click!  (akin to Google's Colab: short-term runs on a hosted JupyterLab environment) 
  CAUTION: as of Aug. 2023, Binder no longer seems to work, in terms of connecting to outside databases, sorry.  It can still be used to browse the software library and the Jupyter notebooks.
• Added Jupyterlab notebooks for diagnostics, incl. database connection
• Expansion of REGEX-based file import
• Reorganization of classes  APIRequestHandler  and  PagesRequestHandler
• Notched up several version numbers of dependencies

LIBRARIES

• New major version  NeoAccess 4.0.1 :

Major simplification in the structure passed as argument to indentify ("match") a node.  This change also substantially reduces the potential for accidental "collisions in dummy names" of subclauses.   The  match() method replaced the old  find()  method

Increased consistency in method/argument names.  In particular, phased out "neo_id" in favor of "internal_id", and "edge" in favor of "link"

Fixed a few bugs, incl. in  add_egdes() and in the "merge" option of create_node_with_links()

New method  add_links_fast()

Changed argument defaults in  empty_dbase()

Renamed create_node_with_children() to create_attached_node(), and re-implemented it using create_node_with_links()

Changes to  delete_nodes()

Reorganized the NeoAccess class; moved the CypherUtils class to its own file, cypher_utils.py

New methods  CypherUtils.assert_valid_internal_id(), validate_internal_id(), process_match_structure(), extract_dummy_name()

Additional unit testing
 

• NeoSchema:

Tweak to  add_data_point_merge()

DOCUMENTATION

• Released new video Neo4j & BrainAnnex's Overhauled v. 5 (DRAFT)
• New Programmer's Guide
• Article on Using Schema in Graph Databases such as Neo4j
• Jupyterlab notebooks with tutorials about the NeoAccess library
• Updated the main README

Aug. 4, 2022

Beta 19

INFRASTRUCTURE

• Continued implementation of full-text indexing.  Split off   FullTextIndexing  as a separate class from Notes, and moved it to a separate file.  Added pytests.  Substantially expanded list of stopwords.
   

LIBRARIES

• Switched to NeoAccess 3.9.2 :
  Added methods  bulk_delete_by_label(),  merge_node(),  exists_by_neo_id() and  assert_valid_neo_id()
  Added argument to  create_node()
   
• NeoSchema : 
  Bugfix to  add_data_point_with_links() and merge_node()
  Completed and tested  add_data_point_merge()
  New methods:  get_class_id_by_neo_id() ,  class_neo_id_exists() ,
  allowable_props(), get_class_name_by_neo_id(), get_class_attributes(), is_strict_class(), add_data_point_merge(), add_data_point_new()
  Introduce new class  SchemaCacheExperimental
  Tweaks to several methods - especially, new extra arguments
  Many new pytests
   

BUGFIX *

•  Fixed bug in  APIRequestHandler.do_bulk_import()

(*) see also bugfixes under individual libraries

July 23, 2022

Beta 18.1

LIBRARIES

• Switched to NeoAccess 3.9.1
  Added option to "profile" the run of all the Cypher queries
  Bugfixes to  create_node_with_links()  and  define_match()
   
• NeoSchema:  major performance-oriented improvements, incl. the use of the new SchemaCache  class. 
  Substantially sped-up import of large JSON files.
  Several new methods, and tweaks to some existing methods, to be more performant.
  Several new pytests

July 21, 2022

Beta 18

INFRASTRUCTURE

• Added new Vue component,  vue_cytoscape_1, for visualization of networks
• Changes in class  Collections , to assign the Schema name of new data nodes as their (additional) node label.
  Simplification to method add_to_collection_after_element()
• First steps towards implementation of indexing of Note contents (incl. parsing into words the text in notes)
• Started the migration of plugin-specific python code out of  APIRequestHandler - in particular, migrated media-related methods to new class MediaManager;  also, migrated plugin-related methods to the class Notes
   

LIBRARIES

• Switched to NeoAccess 3.9
  Fixed typo in old method name "edges_exists".
  New methods: number_of_edges() and create_node_with_links(), with pytests
  Cleanup of debug printing.
  Bugfix to method  create_node_with_links(); also, now correctly handling blanks in link names in several methods
   
• NeoSchema:  renamed method class_exists()  to  class_id_exists().  
  Added several new methods:  
  add_data_point_with_links(), add_data_relationship_fast() , 
  assert_valid_class_name(), get_class_neo_id()
  Tweaked several methods, and obsoleted 2, by using using the new ones.
  Streamlined  create_data_nodes_from_python_data()
  Tweaks and simplifications to  delete_class_relationship().  
  Changed return value in  create_class()
  Partially migrated  next_available_id()  to a better way to generate ID's
  Cleanup of debug printing and other parts.
  Added numerous new pytests.

UX

• Polish of UI form for Continuous Data Ingestion
   

BUGFIX *

•  Fixed bug that prevented export of full dbase

(*) see also bugfixes under individual libraries

June 28, 2022

Beta 17

NEW FEATURES

• Continuous Data Ingestion of all JSON import files in an intake folder

• Schema Class deletion thru the UI

API

• Added endpoint  /simple/delete_class

INFRASTRUCTURE

• Early implementation of a site log

DEPLOYMENT

• Streamline deployment, by getting rid of need for environment variable FLASK_APP

LIBRARIES

• Switched to NeoAccess 3.8 : renamed argument in  update_query()  for consistency
• NeoSchema:  added method  delete_class()

June 9, 2022

Beta 16.3

INFRASTRUCTURE

• Now employing (.ini) config files, in lieu of environmental variables and values set in main.py 
   
• Made the API endpoint  /simple/remote_access_note/ (added in Beta 15.1) tolerant of cross-origin requests (i.e., invocations from other web domain names)
   
• Added diagnostics to aid deployment using the gunicorn WSGI server  (gunicorn is now in the requirements.txt file)
   

LIBRARIES

• Switched to NeoAccess 3.7.1
  Added method  test_dbase_connection().  The  connect()  method now requires credentials to be present, and it verifies that a connection to the database is possible
   

DOCUMENTATION

• Adapted to the new configuration method

May 29, 2022

Beta 15.1

NEW FEATURES

• New API endpoint  /simple/remote_access_note/ , to serve publicly-accessible Content Items of type "notes"

INFRASTRUCTURE

• Changes to main.py (and updated README), to streamline server deployment

• Expanded and revised  APIRequestHandler.get_content()  , now renamed  get_text_media_content(), which includes a "public_required" argument

• Phased in login requirements (introduced in Beta 14) to more API endpoints

• Re-organized pytests

LIBRARIES

• Switched to NeoAccess 3.7
  Upgraded the Neo4j Bolt driver for Python to 4.3.9  (suitable for versions 4.3.x of the Neo4j database)
  Renamed  fetch_nodes()  to  get_nodes()
  New methods:  edges_exists()  and  extract_node().  Streamlining of various functions using the latter method.
  Tweak to  reattach_node().  Misc. cleanup, especially in  get_parents_and_children(). 
  Expanded use of Neo4j ID as alternative for "match" arguments. 
  Expanded and reorganized pytests.
   
• NeoSchema:  switched JSON imports to use Neo4j ID's for vastly-improved speed; adjustments, accordingly, to several methods and pytests.  Increased used of Neo4j ID's.
  New methods:  locate_node(), get_data_point_id(), class_of_data_point().
  Revised add_data_relationship().  Renamed  get_related_class_names()  as  get_linked_class_names()

DOCUMENTATION

• Expanded instructions about deployment to servers, and about Neo4j installation

May 19, 2022

Beta 14.1

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Implementation of login (Flask-based; login data stored on Neo4j)
  Most pages and API endpoints are now login-protected.
  Added a new Flask "blueprint", home (primarily login-related)

CHANGES TO EXISTING FEATURES

• Expanded the UI Schema editor.
  Now offering capabilities such as adding and deleting Class relationships

• Completed, with extensive pytests, the Schema-based JSON import.  UI enhancements.

LIBRARIES

• Switched to NeoAccess 3.6
  Phased out get_nodes().  Re-organized pytests
   
• Bugfix, expansion, and additional pytests in NeoSchema

INFRASTRUCTURE

• First deployment of the web app to a server (same or different server from the one hosting the Neo4j database.)
• Added several diagnostics to aid the deployment process

Apr. 27, 2022

Beta 13

NEW FEATURES

• Substantial exansion of the Schema-creation editor in the UI (in the Schema Manager page)
   
• A schema-based import of JSON files (thru the UI)
   
• The UI now allows to specify a node label for import root(s), for the no-schema JSON data imports

CHANGES TO EXISTING FEATURES

• Parentheses in names of uploaded images are now kept

LIBRARIES

• Switched to NeoAccess 3.5
  NeoAccess.fetch_nodes() now can accept a Neo4j ID in lieu of a "match" structure.
  Change in handling of argument single_row in fetch_nodes().
  Bugfix in create_node_with_children()
  Misc. enhancements to various recently-added methods, and general re-organization.
  Streamlined re-write of methods for the no-schema imports, and additional pytests for them.
  New method get_node_labels()
  Merged "verbose" and "debug" flags
   
• Tweaks to NeoSchema.create_class_relationship(), and other misc. enhancements.
  New methods for schema-based JSON imports, and pytests for them.

INFRASTRUCTURE

• Revision and reorganization of  the upload-related methods, now split off into their own separate class  UploadHelper
   
• New method  ServerCommunication.contact_server_UPLOAD()

Mar. 30, 2022

Beta 12

NEW FEATURES

• Data imports from JSON files (thru the UI)

UX

• Re-organization of the "admin" pages, and separation of all the data-import features into a dedicated page
• Re-organization of the navigation bar at the top

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Automatic typesetting of math formulas entered into Notes (as well as chemical equations and anything that can be represented with TeX notation), using MathJax v. 2

IMPROVEMENTS IN FEATURES PORTED FROM v. 4

• MathJax Typesetting is now automatically applied dynamically upon completing editing of Notes (overcoming a limitation of the old Brain Annex, which required a page refresh)

INFRASTRUCTURE

• API endpoint for data imports from JSON files
• Support for JavaScript-initiated file uploads (using the fetch API)
• Reorganization of CSS
• Reorganization of file locations

"Notes" PLUGIN:

• Cleaned up the initialization of Vue variables in notes.js, using a "Vue hook" for media loading rather than employing side-effects of variable initialization

DOCUMENTATION

• Added a guide to using TeX notation for math formulas, chemical equations, etc : TeX for Equation formatting

Mar. 25, 2022

Beta 11

NEW FEATURES

"Notes" PLUGIN:

• Implemented "title" fields for Notes, and included them in TOC (Table of Contents)

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Creation and deletion of Subcategory relationships (in Category Manager)

INFRASTRUCTURE

• Streamlined and made more modular - and consistent - the setup of the various Flask modules ("Blueprints")
• Major parts of Category Manager switched from Flask to Vue.js

PLUGINS:

• Simplification of the DOM structure of "Notes".  Standardization in the Vue component structure and server communication

LIBRARIES

• Switched to NeoAccess 3.4 . 
  Moved all NeoAccess util methods, incl. those related to the "match" structure, to a separate class "CypherUtils".
  Expanded  NeoAccess.remove_edge().  Added  import_json().
  Bugfix in  NeoAccess.find()
  Additional pytests; bugfixes in some existing pytests
• NeoSchema:  added set_database()
• requirements.txt now spells out exact version numbers of required libraries

Mar. 2, 2022

Beta 10

NEW FEATURES

• Image captions now included in table-of-contents sidebar

• Checkbox to turn on or off details in the table of contents

RE-IMPLEMENTED FEATURES PORTED FROM v. 4

• Editing of Category names (in Category Manager)

API

• Improved error handling, and other enhancements, in endpoint  /simple/update

LIBRARIES

• Switched to NeoAccess 3.3 .   Improvements to set_fields()
• Enhancements to NeoSchema; in particular, added method class_exists(), and improved remove_property_from_class().  Reorganization of NeoSchema tests
• Change to handling of blank fields in ServerCommunication

DOCUMENTATION

• Added technology-stack diagram 

BUGFIXES

• In "records" Vue component

Feb. 22, 2022

Beta 9

NEW FEATURES

• New Schema classes, with their Properties, can now be created in the UI

UX

• Re-organization of the Schema Manager in a separate page
• Several stylistic enhancements to the Category navigation in the right-side box

INFRASTRUCTURE

• The Schema data is now passed by Flask to the category-page viewer

LIBRARIES

• Switched to NeoAccess 3.2 .   Removed a limitation from set_fields()
• Tweaks and enhancements to NeoSchema; in particular, to get_class_properties()

SECURITY

• Updated to a new version of the "Pillow" library (for image resizing) 

Feb. 16, 2022

Beta 8

RE-IMPLEMENTED FEATURES PORTED (and expanded) FROM v. 4:

• Grouping together similar (compatible in structure) adjacent records

UX:

• Better exception handling if APOC not installed on Neo4j
• More concise rendering of URL's in table cells

INFRASTRUCTURE:

• New Vue components 'vue-plugin-table' and 'vue-plugin-single-record'
• Cleaner JavaScript function to determine the "canonical field list" of a  Content Item of type "record"

BUGFIXES:

• Fields not in the Schema are now correctly shown by the Vue component 'vue-plugin-r'

Feb. 12, 2022

Beta 7

RE-IMPLEMENTED FEATURES PORTED FROM v. 4:

• The right sidebox with the Category navigation is now collapsible
• Restored navigation at the top of the Category-Manager pages

NEW FEATURES:

• The icon to collapse/expand the right sidebox is now in fixed position
   
• "Records" PLUGIN:

* Added option to hide sub-records when following links

* URL's in record fields now shown smaller, and if very long they get truncated with ellipsis

INFRASTRUCTURE:

• Streamlined structure of Category-viewer pages
• Table-of-contents sidebar turned into a Vue component
• Reorganized layout of the pages: separated the fix status bar at the top from the top navigation bar
• Re-organization and cleanup of navigation and CSS across site
• Consolidation of the various "experimental" pages
• Early partial implementation of grouping of similar records, in category-viewer pages

UX:

• Eliminated the unnecessary scroll bar at the bottom

Feb. 8, 2022

Beta 6

NEW FEATURES:

* A viewer for linked records (i.e., following relationships from any record)

INFRASTRUCTURE:

* Completed the reorganization of Vue components, plus general cleanup

Feb. 6, 2022

Beta 5

NEW FEATURES:

* An in-place category filter in the category-navigation sidebar (an improvement over the whole-page downscrolls of v. 4)

UX:

* If a category-viewer page has no content and no subcategories, the page's global editing mode is turned on by default

* Any new Category added to such a page, will now immediately appear in its category navigation

* The add-content section at the bottom of the category-viewer page is now under the control of the page's global editing mode

* Cleanup of the the Category Manager.  In particular, the deletion of a Category now gives rise to an overlay to offer navigation to the Home category, or to any of the parents of the deleted category

INFRASTRUCTURE:

* The category-navigation sidebar converted from Flask to Vue, and turned into a Vue component

* Start of reorganization of Vue components

RE-IMPLEMENTED FEATURES PORTED FROM v. 4:

* Implementation of a media file server, also handling the thumbnail images

* The media folder can be kept anywhere in the file system

* The category-navigation sidebar can now be collapsed, and also displays the subcategories and the parent categories.

* Any "remarks" attached to a Category are now shown next to its name at the top of the category-viewer pages

BUGFIXES:

* Better handling of non-existent categories

DOCUMENTATION:

* Expansion of README file; in particular, more information on setting up and configuring Brain Annex

Feb. 2, 2022

Beta 4

INFRASTRUCTURE:

* Eliminated unnecessary multiple connections to Neo4j.  Introduced a shared initialization module

* Flask routing modules made more readable and modular by making use of  Python Classes

* Improved modularity of site-pages data (for site navigation)

* Simplified and streamlined integration of Brain Annex into host sites

* Cleanup and centralization of configurable parameters

* Simplification to records editing

LIBRARIES:

* Expansion of NeoSchema and Categories

API:

* Several new API endpoints, primarily schema-related, and for supporting editing of relationships (links) among data nodes (data records)

NEW FEATURES:

* Early partial implementation of link editing in records

BUGFIXES:

* Restored functionality to the "Cancel" operation in record editing

Dec. 23, 2021

Beta 3

RE-IMPLEMENTED FEATURE PORTED FROM v. 4:

* Bread-crumbs navigation.  New in v. 5: hovering over the Category names brings up the "remarks" field, if present.

Dec. 18, 2021

Beta 2

DOCS:

* Major expansion of README file, including detailed installation instructions, and the JSON file needed to initialize the Schema.

LIBRARIES:

* Expansion of NeoSchema and NeoAccess, to support exports of desired database portions as JSON files.

* Switched to NeoAccess 3.1

FEATURES:

* Implementation of  export of Schema as JSON file.

Dec. 12, 2021

Beta 1

Initial public release

(Using the library NeoAccess 3.0)

Nov. 2021

Beta 0

Early Beta phase, with the newly-added rearrangement/insertion within pages, and addition of new categories.

Categories are now generalized (to Collections), and SiteLinks (website bookmarks) are more usable.

Oct. 2021

Alpha (late)

The Neo4j backend deployed to a virtual machine on the Google cloud.

Expansion and better utilization of the new schema layer.

Version 5 enters active daily use.

Aug. 2021

Alpha (mid-stage)

July 2021

Alpha (early)

Version 5 begins to take shape as a fully usable – though limited in functionality and plugins – implementation, with Neo4j, Python/Flask, and extensive use of Vue.js

Porting of basic features from v. 4, as well as introducing all-new ones, such as JSON import/export.

Joint development at GSK Pharmaceuticals of what will become the open-source library NeoInterface (from which the NeoAccess library was forked at the end of the year)