DBAdapters – database adapters for statistics

Warning

the use the of a DB Adapter can reduce the performance of the Genetic Algorithm.

Pyevolve have a feature in which you can save the statistics of every generation in a database, file or call an URL with the statistics as param. You can use the database to plot evolution statistics graphs later. In this module, you’ll find the adapters above cited.

See also

Method GSimpleGA.GSimpleGA.setDBAdapter()
DB Adapters are set in the GSimpleGA Class.
class DBAdapters.DBBaseAdapter(frequency, identify)

DBBaseAdapter Class - The base class for all DB Adapters

If you want to create your own DB Adapter, you must subclass this class.

Parameters:frequency – the the generational dump frequency

New in version 0.6: Added the DBBaseAdapter class.

commitAndClose()

This method is called at the end of the evolution, to closes the DB Adapter and commit the changes

getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Insert the stats

Parameters:ga_engine – the GA Engine
open(ga_engine)

This method is called one time to do the initialization of the DB Adapter

Parameters:ga_engine – the GA Engine
setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBFileCSV(filename='pyevolve.csv', identify=None, frequency=1, reset=True)

DBFileCSV Class - Adapter to dump statistics in CSV format

Inheritance diagram for DBAdapters.DBFileCSV:

Inheritance diagram of DBAdapters.DBFileCSV

Example:
>>> adapter = DBFileCSV(filename="file.csv", identify="run_01",
                        frequency = 1, reset = True)
param filename:the CSV filename
param identify:the identify of the run
param frequency:
 the generational dump frequency
param reset:if is True, the file old data will be overwrite with the new

New in version 0.6: Removed the stub methods and subclassed the DBBaseAdapter class.

close()

Closes the CSV file handle

commitAndClose()

Commits and closes

getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Inserts the stats into the CSV file

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

open(ga_engine)

Open the CSV file or creates a new file

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBMySQLAdapter(user, passwd, host='localhost', port=3306, db='pyevolve', identify=None, resetDB=False, resetIdentify=True, frequency=1, commit_freq=300)

DBMySQLAdapter Class - Adapter to dump data in MySql database server

Inheritance diagram for DBAdapters.DBMySQLAdapter:

Inheritance diagram of DBAdapters.DBMySQLAdapter

Example:
>>> dbadapter = DBMySQLAdapter("pyevolve_username", "password", identify="run1")

or

>>> dbadapter = DBMySQLAdapter(user="username", passwd="password",
...                            host="mysqlserver.com.br", port=3306, db="pyevolve_db")

When you run some GA for the first time, you need to create the database, for this, you must use the resetDB parameter as True.

This parameter will erase all the database tables and will create the new ones. The resetDB parameter is different from the resetIdentify parameter, the resetIdentify only erases the rows with the same “identify” name, and resetDB will drop and recreate the tables.

Parameters:
  • user – mysql username (must have permission to create, drop, insert, etc.. on tables
  • passwd – the user password on MySQL server
  • host – the hostname, default is “localhost”
  • port – the port, default is 3306
  • db – the database name, default is “pyevolve”
  • identify – the identify if the run
  • resetDB – if True, the database structure will be recreated
  • resetIdentify – if True, the identify with the same name will be overwrite with new data
  • frequency – the generational dump frequency
  • commit_freq – the commit frequency
close()

Close the database connection

commit()

Commit changes to database

commitAndClose()

Commit changes on database and closes connection

createStructure(stats)

Create table using the Statistics class structure

Parameters:stats – the statistics object
getCursor()

Return a cursor from the pool

Return type:the cursor
getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Inserts the statistics data to database

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

open(ga_engine)

Open the database connection

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

resetStructure(stats)

Deletes de current structure and calls createStructure

Parameters:stats – the statistics object
resetTableIdentify()

Delete all records on the table with the same Identify

setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBSQLite(dbname='pyevolve.db', identify=None, resetDB=False, resetIdentify=True, frequency=1, commit_freq=300)

DBSQLite Class - Adapter to dump data in SQLite3 database format

Inheritance diagram for DBAdapters.DBSQLite:

Inheritance diagram of DBAdapters.DBSQLite

Example:
>>> dbadapter = DBSQLite(identify="test")

When you run some GA for the first time, you need to create the database, for this, you must use the resetDB parameter:

>>> dbadapter = DBSQLite(identify="test", resetDB=True)

This parameter will erase all the database tables and will create the new ones. The resetDB parameter is different from the resetIdentify parameter, the resetIdentify only erases the rows with the same “identify” name.

Parameters:
  • dbname – the database filename
  • identify – the identify if the run
  • resetDB – if True, the database structure will be recreated
  • resetIdentify – if True, the identify with the same name will be overwrite with new data
  • frequency – the generational dump frequency
  • commit_freq – the commit frequency
close()

Close the database connection

commit()

Commit changes to database

commitAndClose()

Commit changes on database and closes connection

createStructure(stats)

Create table using the Statistics class structure

Parameters:stats – the statistics object
getCursor()

Return a cursor from the pool

Return type:the cursor
getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Inserts the statistics data to database

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

open(ga_engine)

Open the database connection

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

resetStructure(stats)

Deletes de current structure and calls createStructure

Parameters:stats – the statistics object
resetTableIdentify()

Delete all records on the table with the same Identify

setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBURLPost(url, identify=None, frequency=100, post=True)

DBURLPost Class - Adapter to call an URL with statistics

Inheritance diagram for DBAdapters.DBURLPost:

Inheritance diagram of DBAdapters.DBURLPost

Example:
>>> dbadapter = DBURLPost(url="http://localhost/post.py", identify="test")

The parameters that will be sent is all the statistics described in the Statistics.Statistics class, and the parameters:

generation
The generation of the statistics
identify
The id specified by user

Note

see the Statistics.Statistics documentation.

Parameters:
  • url – the URL to be used
  • identify – the identify of the run
  • frequency – the generational dump frequency
  • post – if True, the POST method will be used, otherwise GET will be used.

New in version 0.6: Removed the stub methods and subclassed the DBBaseAdapter class.

commitAndClose()

This method is called at the end of the evolution, to closes the DB Adapter and commit the changes

getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Sends the data to the URL using POST or GET

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

open(ga_engine)

Load the modules needed

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBVPythonGraph(identify=None, frequency=20, genmax=False)

The DBVPythonGraph Class - A DB Adapter for real-time visualization using VPython

Inheritance diagram for DBAdapters.DBVPythonGraph:

Inheritance diagram of DBAdapters.DBVPythonGraph

Note

to use this DB Adapter, you must install VPython first.

Example:
>>> adapter = DBAdapters.DBVPythonGraph(identify="run_01", frequency = 1)
>>> ga_engine.setDBAdapter(adapter)
Parameters:
  • identify – the identify of the run
  • genmax – use the generations as max value for x-axis, default False
  • frequency – the generational dump frequency

New in version 0.6: The DBVPythonGraph class.

commitAndClose()

This method is called at the end of the evolution, to closes the DB Adapter and commit the changes

getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Plot the current statistics to the graphs

Parameters:ga_engine – the GA Engine
makeDisplay(title_sec, x, y, ga_engine)

Used internally to create a new display for VPython.

Parameters:
  • title_sec – the title of the window
  • x – the x position of the window
  • y – the y position of the window
  • ga_engine – the GA Engine
Return type:

the window (the return of gdisplay call)

open(ga_engine)

Imports the VPython module and creates the four graph windows

Parameters:ga_engine – the GA Engine
setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump
class DBAdapters.DBXMLRPC(url, identify=None, frequency=20)

DBXMLRPC Class - Adapter to dump statistics to a XML Remote Procedure Call

Inheritance diagram for DBAdapters.DBXMLRPC:

Inheritance diagram of DBAdapters.DBXMLRPC

Example:
>>> adapter = DBXMLRPC(url="http://localhost:8000/", identify="run_01",
                       frequency = 1)
param url:the URL of the XML RPC
param identify:the identify of the run
param frequency:
 the generational dump frequency

Note

The XML RPC Server must implement the insert method, wich receives a python dictionary as argument.

Example of an server in Python:

import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer

def insert(l):
    print "Received statistics: %s" % l

server = SimpleXMLRPCServer(("localhost", 8000), allow_none=True)
print "Listening on port 8000..."
server.register_function(insert, "insert")
server.serve_forever()

New in version 0.6: The DBXMLRPC class.

commitAndClose()

This method is called at the end of the evolution, to closes the DB Adapter and commit the changes

getIdentify()

Return the statistics identify

Return type:identify string
getStatsGenFreq()

Returns the frequency of statistical dump

Return type:the generation interval of statistical dump
insert(ga_engine)

Calls the XML RPC procedure

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

open(ga_engine)

Open the XML RPC Server proxy

Parameters:ga_engine – the GA Engine

Changed in version 0.6: The method now receives the ga_engine parameter.

setIdentify(identify)

Sets the identify of the statistics

Parameters:identify – the id string
setStatsGenFreq(statsGenFreq)

Set the frequency of statistical dump

Parameters:statsGenFreq – the generation interval of statistical dump