DataManager Reference Guide
This guide is for versions 5.0-rc.8+
Source code
Background Information:
A Technology Stack on Top of a Graph Database
For general, high-level database-interaction operations.
Used by the UI for Page Generation,
as well as by the web API to produce data for the endpoints.
This library is primarily a common entry point for data requests:
many specific operations get delegated to other, more specialized, libraries.
Static class that does NOT get instantiated;
however, it must be initialized with a call to set_database()
| name | arguments | returns |
|---|---|---|
| set_database | db :GraphAccess | None |
IMPORTANT: this method MUST be called before using this class!
:param db: Database-interface object, created with the GraphAccess library
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| to_int_if_possible | s: str | |
Convert the argument to an integer, if at all possible; otherwise, leave it as a string
(or leave it as None, if applicable)
:param s: Value to convert to integer, if possible
:return: Either an int version of the passed value, or that same value
|
||
| name | arguments | returns |
|---|---|---|
| str_to_int | s: str | int |
Helper function to give more friendly error messages in case non-integers are passed
in situations where integers are expected.
Without this function, the user would see cryptic messages such as
"invalid literal for int() with base 10: 'q123'"
EXAMPLE of usage:
try:
i = cls.str_to_int(i_str)
except Exception as ex:
# Do something
:param s: A string that should represent an integer
:return: The integer represented in the passed string, if applicable;
if not, an Exception is raised
|
||
| name | arguments | returns |
|---|---|---|
| all_schema_classes | [str] | |
Return a list of all the existing Schema classes
:return:
|
||
| name | arguments | returns |
|---|---|---|
| get_schema_visualization_data | ||
Create and return an object with all the info
to visualise a graph displaying the database Schema info
:return: An object of class PyGraphScape
|
||
| name | arguments | returns |
|---|---|---|
| new_schema_class | class_specs: dict | None |
Create a new Schema Class, possibly linked to another existing class,
and also - typically but optionally - with the special "INSTANCE_OF" link
to an existing class (often, "Records")
In case of error, an Exception is raised.
:param class_specs: A dictionary with the following
DICTIONARY KEYS:
new_class_name The name of the new Class (tolerant of leading/trailing blanks)
properties_list The name of all desired Properties, in order
(all comma-separated). Tolerant of leading/trailing blanks, and of missing property names
instance_of Typically, "Records"
[ALL THE REMAINING KEYS ARE OPTIONAL]
linked_to The name of an existing Class node, to link to
rel_name The name to give to the above relationship
rel_dir The relationship direction, from the point of view of the newly-added node
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| add_schema_relationship_handler | class_specs: dict | None |
In case of error, an Exception is raised.
:param class_specs: A dictionary with the following
DICTIONARY KEYS:
from_class_name
to_class_name
rel_name
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| schema_add_property_to_class_handler | specs_dict: dict | None |
Add a new Property to an existing Classes
In case of error, an Exception is raised.
:param specs_dict: A dictionary with the following
DICTIONARY KEYS:
prop_name (any leading/trailing blanks are ignored)
class_name (any leading/trailing blanks are ignored)
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| delete_schema_relationship_handler | class_specs: dict | None |
Delete the relationship(s) with the specified name
between the 2 existing Class nodes (identified by their respective names),
going in the from -> to direction direction.
In case of error, an Exception is raised.
:param class_specs: A dictionary with the following
DICTIONARY KEYS:
from_class_name
to_class_name
rel_name
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| remove_data_relationship_handler | data_dict: dict | None |
Remove the specified relationship (edge) between data nodes.
In case of error, an Exception is raised.
:param data_dict: A dictionary with the following
DICTIONARY KEYS:
from The uri of the node from which the relationship originates
to The uri of the node into which the relationship takes
rel_name The name of the relationship to remove
schema_code (optional) If passed, the appropriate plugin gets invoked
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| get_leaf_records | [str] | |
Get all Classes that are, directly or indirectly, INSTANCE_OF the Class "Records",
as long as they are leaf nodes (with no other Class that is an INSTANCE_OF them.)
EXAMPLE: if the "Foreign Vocabulary" Class is an INSTANCE_OF the Class "Records",
and if "French Vocabulary" and "German Vocabulary" are instances of "Foreign Vocabulary",
then "French Vocabulary" and "German Vocabulary" (but NOT "Foreign Vocabulary")
would be returned
:return: A list of strings with the Class names
EXAMPLE:
["Cuisine Type","Entrees","French Vocabulary","German Vocabulary","Restaurants","Site Link"]
|
||
| name | arguments | returns |
|---|---|---|
| get_text_media_content | uri :str, class_name :str, public_required = False | str |
Fetch and return the contents of a media item stored on a local file,
optionally requiring it to be marked as "public".
:param uri: A string identifying the desired Content Item, which ought to be text media
:param class_name:
:param public_required: If True, the Content Item is returned
only if its database node has an the attribute "public: true"
:return: A string with the HTML text of the requested note;
or an Exception in case of failure
(e.g., if public_required is True and the item isn't public)
|
||
| name | arguments | returns |
|---|---|---|
| get_records_by_class | class_name :str, field_name :str, order_by :str | [] |
Return a list of values of one particular field, of all the records of the given Class,
optionally sorted by the given field
:param class_name: String with the name of the desired Schema Class
:param field_name: Name of the single field to retrieve
:param order_by: Name of the field to sort the results by
:return: A list of values
|
||
| name | arguments | returns |
|---|---|---|
| get_records_by_link | request_data :dict | [dict] |
Locate and return the data (properties) of the nodes linked to the one specified
by either its uri or internal database ID, up to a max of 100.
From that node, follow the relationships named by `rel_name`, in the direction specified by `dir`.
If the internal database ID is provided, then the internal database ID's of the matched nodes is also returned.
:param request_data: A dictionary with the keys, "rel_name" and "dir",
plus either "entity_id" or "internal_id" (the latter takes priority)
:return: A list of dictionaries with all the properties of the neighbor nodes,
including an extra field called "_node_labels", with a string of label names
If the internal database ID is provided, then the internal database ID's
of the matched nodes is also returned.
|
||
| name | arguments | returns |
|---|---|---|
| get_link_summary | entity_id :str, omit_names = None | dict |
Return a dictionary structure identifying the names and counts of all
inbound and outbound links to/from the given data node.
:param entity_id: String with the Entity ID of a data node
:param omit_names: Optional list of relationship names to disregard
:return: A dictionary with the names and counts of inbound and outbound links.
Each inner list is a pair [name, count]
EXAMPLE:
{
"in": [
["BA_served_at", 1]
],
"out": [
["BA_located_in", 1],
["BA_cuisine_type", 2]
]
}
|
||
| name | arguments | returns |
|---|---|---|
| update_content_item | entity_id :str, class_name :str, update_data :dict, label=None | None |
Update an existing Content Item.
No harm if new values are identical to the earlier old values.
Notes:
- if a field is blank, it gets completely dropped from the node
- if a field isn't mentioned, no change is applied to it
- leading/trailing blanks in the field values are stripped away
:param entity_id: String with a unique identifier (within the given Class) for the Content Item to update
:param class_name: Name of the Schema Class of the Content Item
:param update_data: A dict of data field names and their desired new values
EXAMPLE: {'basename': 'my new filename', 'caption': 'my new caption'}
:param label: [OPTIONAL] String with a Label of the Content Item
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| delete_content_item | uri :str, class_name :str | None |
Delete the specified individual Content Item.
:param uri: String with the unique entity ID of the Content Item
:param class_name: Name of the Schema Class of the Content Item
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| create_data_node | class_name :str, item_data: dict | dict |
Create a new Data Node.
RESTRICTION: currently, not to be used for any Content Item that
requires plugin-specific actions
:param class_name:
:param item_data:
:return: A dict with the internal database ID and uri
assigned to the newly-created node
EXAMPLE: {"_internal_id": 123, "entity_id": "rs-8"}
|
||
| name | arguments | returns |
|---|---|---|
| add_new_content_item_to_category | category_uri :str, class_name :str, insert_after_uri :str, insert_after_class :str, item_data: dict | str |
Create a new Content Item attached to a particular Category,
at a specified position on the Category page
:param category_uri: String to identify the Category that we're linking to
:param class_name: Name of the Class of the new Content Item
:param insert_after_uri: Either the URI of an existing Content Item attached to this Category,
or one of the special values "TOP" or "BOTTOM"
:param insert_after_class: The name of the Class of the preceding Content Item, if applicable
:param item_data: Dict with all applicable plugin-specific fields (all the key/values for the new Content Item)
:return: The URI of the newly-created Data Node
|
||
| name | arguments | returns |
|---|---|---|
| new_content_item_in_category | post_data: dict | str |
Create a new Content Item attached to a particular Category,
at a specified position on the Category page
:param post_data: A dict containing the following keys
- "category_id" (for the linking to a Category)
- Schema-related keys:
* schema_uri (Optional)
* class_name (Required only for Class Items of type "record")
- insert_after_uri Either the URI of an existing Content Item attached to this Category,
or one of the special values "TOP" or "BOTTOM"
- *PLUS* all applicable plugin-specific fields (all the key/values for the new Content Item)
:return: The URI of the newly-created Data Node.
In case of error, an Exception is raised
|
||
| name | arguments | returns |
|---|---|---|
| new_content_item_in_category_final_step | insert_after_uri :str, insert_after_class :str, category_id :str, new_uri, class_name, post_data, original_post_data | |
| name | arguments | returns |
|---|---|---|
| directories_stored_in | internal_id=None, limit=100 | dict |
Extract the directory location of the given Content Item, if specified,
as well as the list of all registered directories
See also get_records_by_class()
:param internal_id: [OPTIONAL] To identify the Content Item of interest
:param limit: [OPTIONAL] Max number of folder names to return
:return: The dictionary containing:
1. "location": the name of the directory of the specified Content Item,
if applicable (or None if not specified)
2. "all_directories": the sorted list of all directory names
EXAMPLE:
{"location": "documents/Ebooks & Articles/SYSTEMS BIO",
"all_directories":
[
"documents/Ebooks & Articles/SYSTEMS BIO",
"documents/Ebooks & Articles/math"
]
}
|
||
| name | arguments | returns |
|---|---|---|
| switch_category | data_dict :dict | None |
Switch one or more Content Items from being attached to a given Category,
to another one
:param data_dict: Dict with 3 keys:
items list of string URI's of Content Items
to relocate across Categories
from URI of the old Category
to URI of the new Category
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| search_for_terms | words :str, search_category="" | ([dict], str) |
Carry out a full-text search for a word, or a set of words - possibly restricted to some Categories
:param words: String containing one or more words to search for
:param search_category: [OPTIONAL] URI of a Category. If supplied, all searching will
be limited to Content Items in this Category
or in any of its sub-categories
:return: A pair consisting of:
1) list of dictionaries, each with the record data of a search result
2) a string with a caption to describe these search results
|
||
| name | arguments | returns |
|---|---|---|
| search_for_word | word :str, search_category="" | [dict] |
Look up any stored words that contains the requested string
(ignoring case and leading/trailing blanks.)
Then locate the Content nodes that are indexed by any of those words.
Return a (possibly empty) list of the data of all the found nodes.
:param word: A string, typically containing a word or word fragment;
case and leading/trailing blanks are ignored
:param search_category: (OPTIONAL) URI of Category. If supplied, all searching will
be limited to Content Items in this Category
or in any of its sub-categories
:return: A list of dictionaries, each with the record data of a search result
|
||
| name | arguments | returns |
|---|---|---|
| search_for_all_words | word_list :[str], search_category="" | [dict] |
Look up any stored words that contains the requested string
(ignoring case and leading/trailing blanks.)
Then locate the Content nodes that are indexed by any of those words.
Return a (possibly empty) list of the data of all the found nodes.
:param word_list: A list of strings, each typically containing a word or word fragment;
case and leading/trailing blanks are ignored
:param search_category: (OPTIONAL) URI of Category. If supplied, all searching will
be limited to Content Items in this Category
or in any of its sub-categories
:return: A list of dictionaries, each with the record data of a search result
|
||
| name | arguments | returns |
|---|---|---|
| extract_node_neighborhood | node_internal_id :int|str, known_neighbors :[int|str], max_neighbors=10 | dict |
Given a database node, identified by its internal ID,
locate all its neighbors EXCEPT the specified ones,
and return the full data for those neighbors,
their links to the original node, and their links to each other (if applicable).
The data is returned in a format compatible with the Cytoscape visualization.
Note: the original node is NOT returned (but links to it are)
Any date/datetime value will first be "sanitized"
into a string representation of the date;
the time portion, if present, will get dropped
:param node_internal_id: The internal database ID of the node whose neighbors we want to explore
:param known_neighbors: (Possibly empty) list of internal database ID nodes to exclude.
None is also acceptable in lieu of an empty list.
:param max_neighbors: [OPTIONAL] The max number of NEW neighbors to return
:return: A dict with 2 keys: "nodes" and "edges"
EXAMPLE:
"nodes": [
{
"_node_labels": ["Car", "Vehicle"],
"date_created": "2025/06/23",
"id": 23487,
"_internal_id": 23487,
"brand": "Toyota",
"color": white
}
],
"edges": [
{
"id": "edge-29",
"name": "OWNS",
"source": 23487,
"target": 853
}
]
|
||
| name | arguments | returns |
|---|---|---|
| extract_website_title | url :str | str |
Retrieve the Title of a remote webpage, given its URL
:param url: URL of the website whose title we want to fetch.
EXAMPLE: "https://brainannex.org"
:return: The "Title" of the website.
In case unable to locate the web page, or unable to extract its Title,
raise an Exception
|
||
| name | arguments | returns |
|---|---|---|
| get_filtered | filter_dict :dict | ([dict], int) |
Return the list of the database nodes that match all the requirements spelled out in the given filter
:param filter_dict: A dictionary, with keys:
"label" The name of a node label
"class_name" NOT CURRENTLY IMPLEMENTED
"key_name" A string with the name of a node attribute;
if provided, key_value must be passed, too
"key_value" The required value for the above key; if provided, key_name must be passed, too.
Note: no requirement for the key to be primary
"case_sensitive" Boolean
"clause_key" Name of a node property (field)
"clause_value" Value to match for the above property;
if the value is a string, then do a case-sensitive CONTAINS
"order_by" Field name, or comma-separated list;
each name may optionally be followed by "DESC"
NOTE: if ordering by a non-existing field, "skip" may not work as expected;
this seems to be a Cypher/Neo4j bug
"skip" The number of initial entries (in the context of specified order) to skip
"limit" The max number of entries to return
EXAMPLES:
{"label": "BA", "key_name": "entity_id", "key_value": "sl-123"}
{"label": "doctor", "limit": 25, "skip": 50}
{'label': 'YouTube Channel', 'clause': "n.name CONTAINS 'sc'", 'order_by': 'name'}
{'label': 'Quote', 'clause': "n.quote CONTAINS 'kiss'", 'order_by': 'attribution,quote'}
:return: A pair with two elements:
1. A (possibly-empty) list of dictionaries; each dict contains the data for a node,
including a field called "_internal_id" that has the internal database ID,
and a field called "_node_labels" with a list of the node's label names
2. What the number of nodes would be in the absence of limit/skip value
|
||
| name | arguments | returns |
|---|---|---|
| export_full_dbase | dict | |
Export the entire Neo4j database as a JSON string.
IMPORTANT: APOC must be activated in the database, to use this function.
Otherwise it'll raise an Exception
EXAMPLE:
{ 'nodes': 2,
'relationships': 1,
'properties': 6,
'data': '[{"type":"node","id":"3","labels":["User"],"properties":{"name":"Adam","age":32,"male":true}},\n
{"type":"node","id":"4","labels":["User"],"properties":{"name":"Eve","age":18}},\n
{"id":"1","type":"relationship","label":"KNOWS","properties":{"since":2003},"start":{"id":"3","labels":["User"]},"end":{"id":"4","labels":["User"]}}\n
]'
}
:return: A dictionary specifying the number of nodes exported ("nodes"),
the number of relationships ("relationships"),
and the number of properties ("properties"),
as well as a "data" field with the actual export as a JSON string
|
||
| name | arguments | returns |
|---|---|---|
| upload_import_json | files, upload_dir :str, return_url=None, verbose=False | str |
Modify the database, based on the contents of the uploaded file (expected to contain the JSON format
of a Neo4j export)
:param files: An ImmutableMultiDict object.
EXAMPLE: ImmutableMultiDict([('imported_datafile',
|
||
| name | arguments | returns |
|---|---|---|
| upload_import_json_file | files, upload_dir :str, post_pars, verbose=False | str |
Manage the upload and import into the database of a data file in JSON format.
:param files: An ImmutableMultiDict object.
EXAMPLE: ImmutableMultiDict([('imported_datafile',
|
||
| name | arguments | returns |
|---|---|---|
| data_intake_status | ||
| name | arguments | returns |
|---|---|---|
| do_stop_data_intake | None | |
Request that the continuous data import cease upon the completion of the current import
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| do_bulk_import | intake_folder: str, outtake_folder: str, schema_class: str | str |
Bulk-import all the JSON files in the intake_folder directory.
The import will continue until the folder is empty,
or until the cls.ongoing_data_intake property is set to False
Failure of individual imports is logged, but does not terminate the operation.
An Exception is raised if any of the following happens:
* The log file cannot be accessed/created
* Any of the data files cannot be moved to their final destination
:param intake_folder: Name of folder (ending with "/") where the files to import are located
:param outtake_folder:
:param schema_class:
:return:
|
||
| name | arguments | returns |
|---|---|---|
| process_file_to_import | f: str, intake_folder: str, outtake_folder: str, schema_class: str | None |
Import a JSON file, located in a particular folder, and then move it to a designated folder.
Keep a log of the operations.
Exceptions are caught and logged; if the Exceptions involves moving the process file, a new one is raised
:param f: Name of file to import
:param intake_folder: Name of folder (ending with "/") where the files to import are located
:param outtake_folder:
:param schema_class:
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| archive_data_file | src_fullname, dest_fullname, outtake_folder | None |
Move the processed file to its final location.
In case of error, the Exception is caught, logged and re-raised
:param src_fullname:
:param dest_fullname:
:param outtake_folder:
:return: None
|
||
| name | arguments | returns |
|---|---|---|
| import_datafile | basename, full_filename, test_only=True | str |
:param basename: EXAMPLE: "my_file_being_uploaded.txt"
:param full_filename: EXAMPLE: "D:/tmp/my_file_being_uploaded.txt"
:param test_only: If True, the file is parsed, but nothing is actually added to the database
:return: String with status message (whether successful or not)
|
||
| name | arguments | returns |
|---|---|---|
| define_pattern | str | |
Define a REGEX pattern for parsing of data files, for use in import_datafile()
The pattern is expected to be used in a re.findall() that uses re.DOTALL as the last argument
:return: A string with a REGEX pattern
|
||
| name | arguments | returns |
|---|---|---|
| init_logfile | None | |
Prepare a handle for the log file
:return:
|
||
| name | arguments | returns |
|---|---|---|
| append_to_log | msg | None |
:param msg:
:return:
|
||