mckoi

The Mckoi Database; the JDBC driver is in this package.

The core database classes for Mckoi.

Data Representation

A database is represented by a single TableDataConglomerate, which contains a collection of MasterTableDataSource objects, each of which represents a single table, including both committed and uncommitted rows.

Access to a table is usually through the TableDataSource interface, or its subclass MutableTableDataSource The MasterTableDataSource objects wrap themselves in an instance of the private class MasterTableDataSource.MMutableTableDataSource, which implements MutableTableDataSource.

The format of a table is defined by DataTableDef, which is returned by TableDataSource.getDataTableDef, and which does not change during the lifetime of the table; if the table format is changed, a new MasterTableDataSource is created within the Transaction with the new format, and the data and unchanged columns from the old table are copied to the new one. The format of a column is defined by DataTableColumnDef.

Modifications to a row of a table are handled by a RowData, which carries a reference to the TableDataSource which it modified, and from which data is retrieved. Each column within the row is represented by a DataCell, which is either retrieved from the table or created by DataCellFactory.

Transactions

A database can be associated with one or more simultaneous Transaction objects. All changes to the data in the database are done through Transactions. The current set of open Transaction objects is managed by an OpenTransactionList object, which is pointed to by both the TableDataConglomerate and all of the MasterTableDataSource objects.

Changes to a row are handled by creating a new row in the MasterTableDataSource containing the changed data plus any old data which is unchanged. The data for the modified row are kept in a RowData, which references a QueryContext, which references a DatabaseConnection, which references a Transaction, which is the Transaction under which that change was made. Each field of the row is represented by a DataCell.

When an application issues an SQL request to update the database, it eventually makes it down to Statement.evaluate, for example in Insert. That evaluate method uses its DatabaseConnection to get a DataTable for a table name, which is a wrapper around the MutableTableDataSource returned by the DatabaseConnection's Transaction for the table of that name. The MutableTableDataSource (created by Transaction.getTable) is in turn a wrapper around a MasterTableDataSource created using the private class MasterTableDataSource.MMutableTableDataSource. The Statement uses its DataTable to create a RowData, then passes it to its DataTable, which passes it to the MutableTableDataSource, which passes it to the MasterTableDataSource, which actually makes the change.

The Transaction maintains a TransactionJournal, in which are listed all of the tables which have been changed by the Transaction.

Eventually the Transaction is closed (committed or rolled back), which is handled by the TableDataConglomerate ( processCommit or processRollback), which, for each MasterTableDataSource, gets a MasterTableJournal for it from the TransactionJournal specifying what changes have been made in that table from this Transaction, and tells that MasterTableDataSource to commit or roll back the changes in that MasterTableJournal.

Locking

Locking is used to control concurrent access by two requests in the same Transaction. This is handled during query execution in JDBCDatabaseInterface.execQuery

Each DatabaseConnection has associated with it a single LockingMechanism object, which is used to lock and unlock the DatabaseConnection's Transaction as a whole. The active lock is represented by a LockHandle, which is returned by LockingMechanism.lockTables, and which is passed back to LockingMechanism.unlockTables to drop the lock.

A lock on an individual table is represented by a Lock, which is kept in a LockingQueue, which maintains the link to the locked table.

Update Sequence

When a change is made to the database (insert, update, or delete), the following list shows the sequence in which various steps are taken:

• Check to see if user has privileges to make the change (Insert.evaluate, UpdateTable.evaluate, Delete.evaluate)
• Check to see if the table is read-only ( MasterTableDataSource.MMutableTableDataSource.addRow, updateRow, removeRow)
• Mark the old row for removal ( MasterTableDataSource.MMutableTableDataSource.updateRow, removeRow)
• Add the new data and write the changes out to disk ( MasterTableDataSource.addRow)
• Check table constraints for removal of old data (MasterTableDataSource.MMutableTableDataSource calls TableDataConglomerate.checkRemoveConstraintViolations)
• Check field constraints on the new data (MasterTableDataSource.MMutableTableDataSource calls TableDataConglomerate.checkFieldConstraintViolations)
• Check table constraints on the new data (MasterTableDataSource.MMutableTableDataSource calls TableDataConglomerate.checkAddConstraintViolations)

This package includes database constants such as Types.

These classes interpret specific SQL statements, such as Insert and Delete.

The JDBC interface to Mckoi.

Establishing a Connection

An application establishes a JDBC Connection by calling java.sql.DriverManager.getConnection(String url, Properties info). DriverManager.getConnection calls java.sql.Driver.connect, which figures out that the URL is a Mckoi URL, so calls com.mckoi.database.jdbc.MDriver, which was registered by a static initializer in com.mckoi.JDBCDriver.

If the URL is local (embedded mode), MDriver creates an instance of com.mckoi.database.jdbcserver.LocalDatabaseInterface, which in turns creates and wraps up an instance of com.mckoi.database.jdbcserver.JDBCDatabaseInterface, and calls its connectToJVM method to initialize it.

If the URL is remote (client/server mode), MDriver creates an instance of TCPStreamDatabaseInterface and calls its connectToDatabase method in order to establish a TCP connection to the Mckoi database server. For more information on how the server handles connections, see the package com.mckoi.database.jdbcserver.

In either case, the resulting DatabaseInterface is wrapped up in a MConnection, and returned to the application as an instance of java.sql.Connection.

Executing a Query

When an application calls java.sql.Connection.createStatement() on its MConnection, it gets back an instance of MStatement, which carries a pointer to the MConnection.

When the application calls java.sql.Statement.executeQuery(String) on its MStatement, Mckoi creates an SQLQuery from the String, creates an empty MResultSet, then calls MStatement.executeQuery with those two objects. MStatement.executeQuery turns around and calls Mconnection.executeQuery, which calls execQuery on its DatabaseInterface. Depending on whether the connection is local or remote, this call is either to com.mckoi.database.jdbcserver.LocalDatabaseInterface.execQuery for local connections, or to RemoteDatabaseInterface.execQuery for remote connections. These are described more fully in the package description for com.mckoi.database.jdbcserver under Local Queries and Remote Queries.

Implementation of the DatabaseInterface interface, including classes to handle local (embedded) mode and remote (client/server) mode.

See DatabaseInterface.

Local (Embedded) mode

Data is written to the server in the calling thread. There is a separate thread (an instance of the private class RemoteDatabaseInterface.ConnectionThread in RemoteDatabaseInterface) to read the data coming back from the server, which is placed into a buffer where it can be accessed by other threads.

Query Execution

• Set user and connection data into the WorkerThread.
• Output debug log info.
• Record start time.
• Parse sql statement into statement_tree, check cache.
• Bind values for query parameters.
• Prepare all expression in statement tree.
• Convert to instance of some Statement subclass (such as Insert or Delete.) These are all in the package com.mckoi.database.interpret.
• Set up locking.
• Finish preparing statement.
• Get list of tables to read and write.
• Lock tables.
• Evaluate the Statement using Statement.evaluate. This is different for each different type of Statement.
• Place results into result set cache.
• Unlock tables.
• Check autoCommit, do commit or rollback if set.
• Record end time.
• Return QueryResponse info (an instance of the private class JDIQueryResponse in JDBCDatabaseInterface).

The server listener thread sits in a loop blocked waiting for new connections. When a new connection is accepted, the server creates a new JDBCDatabaseInterface for that connection, wraps it in a TCPJDBCServerConnection, and adds that to the ConnectionPoolServer.

Client Requests

In single-thread mode, the SingleThreadedConnectionPoolServer handles all client reads in a single thread. For each connection with a request, it calls TCPJDBCServerConnection.processRequest on that connection, using com.mckoi.database.DatabaseSystem.execute to execute it in a WorkerThread.

In multi-thread mode, the MultiThreadedConnectionPoolServer starts a separate thread to handle reads for each connection. Each thread blocks until it has a request available, then calls TCPJDBCServerConnection.processRequest on that connection, using com.mckoi.database.DatabaseSystem.execute to execute it in a WorkerThread.

In either single-thread or multi-thread mode, client requests end up getting passed to TCPJDBCServerConnection.processRequest. This method reads the command data, then calls JDBCProcessor.processJDBCCommand with those bytes. That method calls JDBCProcessor.processQuery, which looks at the first byte of the requests to see what it is. If the request type is a database query (as opposed to retrieval of part of a ResultSet), it creates a SQLQuery from the data and calls execQuery on its JDBCDatabaseInterface (see Query Execution above). The result is encoded as a QueryResponse, which is converted back to bytes and written back to the client. Note that this is all running in a WorkerThread, so the call to DatabaseInterface.execQuery is in a WorkerThread, as required.

If there is an exception during processing of a client request, the message and traceback for the exception are encoded and returned to the client, which recognizes the packet as an exception, decodes the data on its side, and throws an exception in the client.

Implementations of the RegexLibrary interface, which allows the application to select which regular expression package to use.

See RegexLibrary.

These classes parse SQL statements in Mckoi. The SQL syntax is defined in the file SQL.jj and compiled by JavaCC into the SQL class.

Debugging classes for Mckoi.

JFC classes to access a Mckoi database from a Swing application.

The main (starting) class for a Mckoi database server.

Starting the Server

The META-INF/MANIFEST file in the jar file specifies the starting class as McKoiDBMain in this package, so java starts up the server application at the main method of McKoiDBMain.

The main method prints a banner, parses options, loads the config file, and eventually calls BootMain.boot to complete the startup. BootMain creates an instance of Database, initializes it, and finally starts the service that listens for and handles client connections.

Tools for manipulating a Mckoi database.

Miscellaneous utility classes.