Introduction

This section outlines the general usage of the BDB Server module.

Connect to a BDB Server

The following will create a simple connection to a running BDB Server (Node Manager):

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)

Retrieve a list of databases

The following will retrieve and display a list of databases registered on the server, and the state of each database (started/stopped):

  • NodeManager.databases
from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)

for db in nm.databases:
    print(db + ' (' + str(nm.get_database_state(db)) + ')')

Query a database

A query is applied to a running Database and returns a list of Feature objects which meet that query criteria.

Simple query

The most basic database query retrieves all features of a chosen type from a running database:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

features = db.query('surfac')

for feature in features:
    print(feature.id)# print feature id

FeatureRanges

Queries return FeatureRange objects. FeatureRanges are single-pass Iterators that return Feature objects. This means that it can only be iterated over once before the FeatureRange object is destroyed and for this reason, there is no len() function defined on a FeatureRange. This iterator is single-pass for performance reasons in case there are too many objects returned from a query. If a caris.bathy.db.Feature or set of Features need to be referenced after iterating through the FeatureRange then you must keep individual references or store references in a container. Examples:

# find a single feature
my_feature = None
for feature in database.query('surfac'):
    if feature['OBJNAM'] == 'MyFeature':
        do_something(feature)

    # or store a reference to it for later
    my_feature = feature

do_something(my_feature)

# store all features in a collection for later
my_features = []
for feature in database.query('surfac'):
    my_features.append(feature)

print('Number of features: {}'.format(len(my_features)))

do_something_to_features(my_features)

Query by attribute

A CQL query can be added to filter for attributes with specific values:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

cql = "OBJNAM = 'my_surface'"

features = db.query('surfac', cql)

for feature in features:
    print(feature.id) # print feature id

Query by polygon

A Database can also be queried by polygon:

from caris.bathy.db import *
import caris

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

polygon = """POLYGON((-004.153110200 50.333355300,
-004.147552500 50.333200600,
-004.147310900 50.326781500,
-004.153714300 50.327400300,
-004.153110200 50.333355300))"""
geometry = caris.Geometry(db.crs, polygon)

cql = "OBJNAM = 'MY_SURFACE'"

features = db.query('surfac', CQL=cql, intersects=geometry)

for feature in features:
    print(feature.id) # print feature id

Note

Any object which intersects the query geometry will be returned, not just objects that fall within the geometry.

Described geometry can be defined in the Well-known text markup language (http://en.wikipedia.org/wiki/Well-known_text), and the object type must be a (closed) Polygon. The Attribute and Polygon queries can be combined into a complex query.

Journal query

The Database contains a journaling system that records information about changes made to the data. This system can tell us when data was created, modified, or deleted. In addition to recording what data was modified, it also records who made the change and when they did it.

from caris.bathy.db import *
from datetime import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

operationTypes = [OperationType.CREATE, OperationType.EDIT]
modifiedBeforeToday = db.query_journal(operationTypes, Operator.LESS, datetime.utcnow())

for entry in modifiedBeforeToday:
    print(entry.identifier, str(entry.operation), str(entry.time), entry.userName)

Get a Feature by Id

Any feature can be retrieved via it’s unqiue Id:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '9245b052-634c-11e7-8dcf-1866da47ee87'
feature = db.get_feature(id)

Transactions

Edits on databases or database objects are done implicitly through transactions. When creating features, editing object attributes, attachments, geometries, etc, tasks are sent and batched up on the database. The changes are not persisted until Dataset.commit is called. If the changes need to be cancelled, the Dataset.rollback can be called.

   db = NodeManager('dba','secretpassword', 'localhost').get_database('bathydb')

   # create a couple of features and set some attributes
   geom = caris.Geometry(db.crs, 'POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))')

   feature1 = db.create_feature('surfac', geom)
   feature1['OBJNAM'] = 'Feature1'

   feature2 =db.create_feature('surfac', geom)
   feature2['OBJNAM'] = 'Feature2'

   # commit the changes
   db.commit()

   # make some other changes
   feature1['OBJNAM'] = 'ToBeReverted'

   new_geom = caris.Geometry(db.crs, 'POLYGON((1 1, 0 2, 2 2, 2 1, 1 1))')
   feature2.geometry = new_geom

   #rollback the changes
   db.rollback()

Note

Uploading coverages is not done through transactions. They are stored immediately so rollbacks are not supported and Dataset.commit does not need to be called.

Metadata

Metadata on objects in the database can be read, added, modified and removed, providing the login credentials used to connect to the server have permission for these actions. Below is an example of the command used to retrieve metadata:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

features = db.query('surfac')

for feature in features:
    # you can iterate over a feature to view it's attributes
    for attribute in feature:
        print('{} - {}'.format(attribute, feature[attribute]))

    # you can also retrieve an attribute dictionary reference
    attributes = feature.attributes
    # do something with it
    foo(attributes)

Metadata from a Feature object is returned as an attribute dictionary. Attribute values can be read, assigned or replaced within this dictionary. Changes are only saved when commit() is called on the Database:

...

boid = '9245b052-634c-11e7-8dcf-1866da47ee87'
feature = db.get_feature(boid)

# display the object name
print(feature['OBJNAM'])

# set an attribute value
feature.atrributes['INFORM'] = 'Test'
feature.attributes['DUNITS'] = 'metres'

# commit the changes
db.commit()

...

Note

For enumerated lists, these attributes can be described by either list number or list text. In the example above, DUNITS has a list value of 1, which corresponds to the description, metres. When retrieving metadata, the attribute will report a value of 1. When assigning a value to DUNITS, either 1 or metres can be used.

...


#these statements are equivalent
feature.attributes['DUNITS'] = 'metres'
feature.attributes['DUNITS'] = '1'


...

Working with data

For each feature in a database, a CSAR file can be be stored and downloaded containing the stored bathymetry, in either grid or point cloud format. This holds true regardless of the original source format when the data was loaded into the Server as the data is converted and stored in the CSAR format.

Download a Coverage

The coverage object (Raster, Cloud, or VRS) can be retrieved via the Feature.coverage attribute. The coverage data can be downloaded to a CSAR file via create_copy

from caris.bathy.db import *
import caris.coverage

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '9245b052-634c-11e7-8dcf-1866da47ee87'
feature = db.get_feature(id)
coverage = feature.coverage
coverage.create_copy('C:/my_surface.csar')

A coverage object can also be downloaded via a BDB URI as seen in the following example.

from caris.bathy.db import *
import caris.coverage
import caris

# URI format:  bdb://username:password@hostname/bathydatabasename/featureid
uri = 'bdb://dba:sql@localhost/apiguard/b69061bc-a43f-11e7-8000-1866da47ee87'

# Use the appropriate constructor for the dataset:  Raster, Cloud, or VRS
coverage = caris.coverage.Raster(uri=uri)
coverage.create_copy('C:/my_surface.csar')

Note

Note that when opening a coverage via a BDB URI, importing both caris.bathy.db and caris.coverage is required.

Remove a Coverage

The coverage stored in a database Feature object can be removed. This will not change the geometry of the Feature.

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '6df9623e-3fca-405a-96da-1cd8373adadc'
surface = db.get_feature(id)

#removes the coverage from the feature
surface.remove_coverage()

Replace a Coverage

The coverage stored in a database Feature object can be replaced at any time. Any change to the stored coverage will cause an automatic update to the boundary that represents the surface object, ensuring it matches with the current bathymetric data.

from caris.bathy.db import *
from caris.coverage import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '9245b052-634c-11e7-8dcf-1866da47ee87'
surface = db.get_feature(id)

# it can take a path to a CSAR file
surface.upload_coverage('C:/my_surface.csar')

# or it can take a caris.coverage.Raster|Cloud|VRS
raster = Raster('C:\raster.csar')
surface.upload_coverage(coverage=raster)

Upload a new Coverage

The upload coverage operation resembles a replace bathymetry command, with an additional step of creating the database feature and setting the default metadata.

from caris.bathy.db import *
import caris

user = 'dba'
passwd = 'sql'
host = 'localhost'

# get the database
nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

# create a feature
crs = db.crs
dummy_polygon = 'POLYGON((0 0,0 1,1 1,1 0,0 0))'
geom = caris.Geometry(crs, dummy_polygon)
surface = db.create_feature('surfac', geom)

# set some metadata
surface['OBJNAM'] = 'my_surface'

# commit the feature to the database
db.commit()

# upload a coverage
surface.upload_coverage('C:/my_surface.csar')

The first step is to create a new database feature to store the bathymetry. To add a new object, geometry must be provided for this object when it is created. If the upload fails or no upload is performed, this geometry will be representative of the object in the database. In this instance, a simple 1-pixel polygon is used temporarily. A better practice is to use a minimum bounding rectangle (MBR), which is the behaviour found in BASE Editor when uploading a new surface object.

Note

A Feature must be commited to the database by calling db.commit before a coverage can be uploaded to it. A coverege is automatically committed when you upload it so you do not need to call db.commit() after.

After creating the new Dataset object, CSAR (or other supported format) data can be uploaded to the object. Once complete, metadata for this object can optionally be set.

Note

Through the Python interface, only the system attributes of an object are defined. All other attributes, including OBJNAM, must be defined by the user when a new object is created. The standard practice for defining OBJNAM in BASE Editor is to use the filename without the file extension, as shown in the above example.

Working with Attachments

A Feature object can have any number of attachments, which are stored in the file system on the BDB Server node location (potentially separate from the back-end database, PostgreSQL or Oracle). Attachments can be accessed directly from a surface object returned from a database.

See the following:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '9245b052-634c-11e7-8dcf-1866da47ee87'
surface = db.get_feature(id)

if surface.attachments:
    for attachment in surface.attachments:
        print(attachment.name + ' ' + attachment.size)

Attachments can be downloaded and uploaded in the same manner as bathymetry.

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

id = '9245b052-634c-11e7-8dcf-1866da47ee87'
surface = db.get_feature(id)

surface.add_attachment('C:/my_document.pdf')
surface.download_attachment('my_document.pdf', 'D:/my_document.pdf')

Examining the Catalog

The Database's Catalog can be retrieved and examined to view to view definitions of object and attributes.

from caris.bathy.db import *
import caris

user = 'dba'
passwd = 'sql'
host = 'localhost'

nm = NodeManager(user, passwd, host)
db = nm.get_database('Global_DB')

catalogue = db.catalogue

for feature_def in catalogue.definitions:
    print(feature_def.code)

    for attribute_code in feature_def.attributes: 
        attribute_def = feature_def.attributes[attribute_code]

        # list type attributes are modelled differently
        if (type(attribute_def)) == caris.AttributeDefinitionDictionary:
            attribute_def = attribute_def['Value']

        print('Code: {}, Type:'.format(attribute_def.code))
        print(attribute_def.type)
        
        # list expected values of enum/list types
        if type(attribute_def) == caris.EnumType:
            print('Expected Values:')
            for value  in attribute_def.type.possible_values:
                print('\t{} - {}'.format(value, attribute_def.type.possible_values[value]))

Error handling

Any errors returned from BDB are passed to Python as RunTimeError exceptions, which can be intercepted using Python syntax. The returned exception will contain the error code and description from BDB. In the following example, this error number and description is replaced with a simplified description of the problem when reported to the user:

from caris.bathy.db import *

user = 'dba'
passwd = 'sql'
host = 'localhost'

a_server = None

try:
    #connect to node
    a_server = NodeManager(user, passwd, host)
    
except RuntimeError as e:
    if "Error code = 30" in str(e):
        sys.exit("Error: Login information incorrect or account disabled.")
    else:
        sys.exit(str(e))
        
db = None

try:
    db = a_server.get_database('Global_DB')
    
except RuntimeError as e:
    if "Error code = 21" in str(e):
        sys.exit("Error: Database does not exist")
    elif "Error code = 29" in str(e):
        sys.exit("Error: Database not started.")
    else:
        sys.exit(str(e))