Dark Planet Development Platform

Get Dark Planet Development Platform at SourceForge.net. Fast, secure and Free Open Source software downloads

Data Model API

The Data Model API is designed to make it easy to store and manipulate EVE character and corporation data, for example as retrieved using the EVE Server API. As such, the data model is less of an API and more of a class hierarchy and relationship model with API methods which aid in managing and manipulating the classes which make up the model.

Certain character and corporation data is more naturally stored in a database (think wallet journal entries). For this reason, certain data model classes reference the persistence service as provided by the IEvePersistenceFactory interface. Most of the description on this page can be understood without understanding the persistence API, but eventually you'll want to read up on persistence as well.

The data model provides three key features:

We describe each of these features below.

API Overview

The main gateway interface for the data model is org.dps.core.model.IEveDataModel. This interface provides basic methods for instantiating and traversing the model, as well as some convenience methods:

Return value Method Function
IModelFactory getModelFactory() Get the model factory used to instantiate all model objects.
void setModelFactory(IModelFactory newFactory) Change the model factory. The new model factory will be used to instnatiate all future model objects.
void addObserver(IDataObserver observer) Add an observer to be invoked during a traversal of the data model.
void removeAllObservers() Remove all traversal observers.
boolean removeObserver(IDataObserver observer) Remove the given observer from all future model traversals. Returns true if the observer exists and was removed, and false otherwise.
void apply(IDataVisitor v, CachedData d) Traverse the data model rooted at d using the visitor v.
Date convertEpochToDate(long epochTime) Convert epoch time (time in millise since January 1, 1970) to a Java Date.
String formatDate(Date d) Format a Java Date in human-readable form.
String formatDate(long e) Format an epoch time date in human-readable form.

Model object creation and traversal are described below. The utility methods provided by the data model are self-explanatory.

Data Model Objects and Instantiation

Data model objects are organized into two distinct trees which share some common elements. The "character" tree is rooted by an instance of Capsuleer and contains references to character related data. The "corporation" tree is rooted by an instance of Corporation and contains corporation related data. Common elements are things like Asset, instances of which may exist in either tree. Almost all non-root data model objects have a parent reference to establish their position in the tree. See the appendix at the bottom of this page for the complete data model tree.

All data model objects have the following features:

Data model objects are purposely given non-public constructors so that API users are forced to use an IModelFactory for constructing model objects. This is to allow customization of the object model without breaking any existing code (possibly third party) which makes use of the object model. For example, suppose you have some third party code which creates a character as follows: 

     IEveDataModel dataModel = ... reference to your data model ...;
     IModelFactory factory = dataModel.getModelFactory();
     Capsuleer myCharacter = factory.makeCharacter(keyID, vCode, characterID);
Let's say you need to customize the default data model, perhaps you decide to extend the Capsuleer class with your own class which you'd like to be instantiated any time someone needs a new character. This is easy to do by implementing your own IModelFactory and registering with the data model as follows: 

     IModelFactory myFactory = ... your custom data model object factory ...;
     IEveDataModel dataModel = .. reference to your data model ...;
     dataModel.setModelFactory(myFactory);
Now the same third party code will still work and will instantiate the appropriate objects.

Timeliness and meta-data support are provided by the CachedData class which all data model objects inherit from. This class defines the following methods:

Return value Method Function
long expiresIn() Returns the number of milliseconds from now at which the data stored in this object will be considered expired. Returns 0 if the data has already expired.
boolean isExpired() Returns true if this data is expired (relative to the current time), and false otherwise.
void putMeta(String key, Serializable value) Attach meta data to this data object.
Serializable getMeta(String key) Retrieve meta data in generic form from this object, or null if the named meta data does not exist.
<K extends Serializable> K getMeta(String key, Class<K> clazz) Retrieve meta data from this object and cast the result to the given class, or null if the named meta data does not exist.

The instance field cachedUntil stores the DPS epoch time at which the data model object is considered expired. By default, data model objects are instantiated as expired but an alternative constructor is provided for objects which should never be considered expired. An example is Asset, which is not directly synchronized with the EVE servers, but instead managed by a "container" object which can expire.

Meta data may be attached to a data model object as long as it implements Serializable. This is to allow for generic persistence of data model objects, for example as provide by the IEvePersistenceFactory service. In general, instances of the data model are intended to be persisted using Java serialization.

Some data model objects do not inherit directly from CachedData but instead inherit from HibernateCachedData (which in turn inherits from CachedData). This class is used as the superclass for data model objects which store content using the IEvePersistenceFactory service. The methods of this class are used to interact with Hibernate, for example getSession, begin, commit, rollback and close. There are also convenience methods for directly populating each of the data model types stored via Hibernate. We provide examples below for data model objects which inherit from HibernateCachedData.

Multi-threaded Access to Data Model Elements

As noted above, all data model fields are public with no explicit concurrency controls. However, the CachedData class includes a lock field which is instantiated with a ReadWriteLock. Since every data model object inherits from CachedData, every data model object therefore has a lock. Use of this lock is optional. The only case where the lock is explicitly used is in the implementation of getMeta and setMeta. Nonetheless, it is highly recommended that reference implementations and other users of the data model make use of locks to ensure consistent access.

If you choose to use locks, you should ensure proper acquire/release semantics so that locks are not left hanging. This is normally done as follows: 

     CachedData obj = ... your data model object ...;
     
     // Safely perform a read operation
     obj.lock.readLock().lock();
     try {
          ... do some read operation ...
     } finally {
          // safely release the lock
          obj.lock.readLock().unlock();
     }

     // Same pattern goes for write operations
     obj.lock.writeLock().lock();
     try {
          ... do some write operation ...
     } finally {
          // safely release the lock
          obj.lock.writeLock().unlock();
     }
At time of writing, the available lock implementation is a ReentrantReadWriteLock which allows concurrent readers, single writers, and properly handles the case where a reader or writer attempts to acquire a lock they already own. We instantiate this lock in a fair way so that writers are not arbitrarily starved behind a continous stream of readers.

Data Model Traversal

Although you are free to traverse your own data model object trees any way you like, the data model API provides a few convenience methods for making this easier. The first such method is apply which takes as arguments an IDataVisitor and a CachedData. This method uses the visitor pattern to traverse the data model tree rooted at the CachedData you provide to the method. The visitor pattern separates the code which traverses a data structure from the code which operates on the data structure. The caller of apply supplies "visitors" in the form of an IDataVisitor which contains a callback called "visit" for each data model object. During traversal, the apply method calls the appropriate visit method based on the data model object it is currently processing.

In more practical terms, the apply method does a depth first traversal of the data model tree starting at the CachedData object you provide. The appropriate "visit" method is called on the current data model object before any of its children are processed.

In addition to directly traversing a data model tree, you can also register observers which are called when a tree traversal is in process. An observer implements the IDataObserver interface which contains a single method visiting which in turn takes a single CachedData argument. This method is called by apply the first time it processes a given CachedData object.

Observers must be registered by calling the addObserver method on IEveDataModel. Once registered, an observer will be called for all traversals of all data model objects. You can remove an observer either by calling IEveDataModel.removeAllObservers() or IEveDataModel.removeObserver().

Example: Create a Simple Character Tree and Populate It

The root of most data model trees will either be a Capsuleer or Corporation. Astute readers may wonder why we insist on calling the character class Capsuleer instead of Character. The reason is that Character is already one of the core classes in Java. If we named our character class as Character then we'd have to include the complete package name every time we specify this type. That's annoying, so instead we decided to represent characters with the Capsuleer class.

To create a new Capsuleer, you simply need to call the appropriate method on the model factory. We've already done this in an earlier example above: 

     IEveDataModel dataModel = ... reference to your data model ...;
     IModelFactory factory = dataModel.getModelFactory();
     Capsuleer myCharacter = factory.makeCharacter(keyID, vCode, characterID);
Note that a Capsuleer stores credentials which may be used to update character data using the EVE server APIs. A Corporation data model object likewise stores server credentials. You can, of course, not pass valid credentials if you'd prefer to manage this information elsewhere.

Since all data model fields are public you can populate your character by directly setting the instance fields. However, make sure you also populate any parent references. For example, to add a CharacterAccountBalance to our character tree: 

     myCharacter.balance = factory.makeCharacterAccountBalance();
     myCharacter.balance.character = myCharacter;

Example: Using Data Model Traversal

Now that we have a simple character tree, let's traverse it. Recall from above that you need to supply an IDataVisitor when calling apply. To make it easier to write your own visitors, you can extend the AbstractVisitor abstract class, which provides a stub implementation for every method of IDataVisitor. Here's a simple visitor for the character tree we just created: 

     import org.dps.core.model.visitor.AbstractVisitor;
     import org.dps.core.model.DPSException;
     import org.dps.core.model.character.Capsuleer;
     import org.dps.core.model.character.CharacterAccountBalance;

     public class MySimpleVisitor extends AbstractVisitor
     {
          public void visitCharacter(Capsuleer c)
            throws DPSException
          {
               System.out.println("Character: " + c.name);
          }

          public void visitCharacterAccountBalance(CharacterAccountBalance b)
            throws DPSException
          {
               System.out.println("  balance = " + b.balance);
          }
     }
This example is a little silly since we could have easily displayed our character's balance from the visitCharacter method, but it demonstrates that all data model elements are visited, and that parent elements are visited before their children. Here's a more interesting example that shows when we should synchronize our tree with the EVE servers again: 

     import org.dps.core.model.visitor.AbstractVisitor;
     import org.dps.core.model.DPSException;
     import org.dps.core.model.CachedData;

     public class ExpiryVisitor extends AbstractVisitor
     {
          public long expiryMillis = Long.MAX_VALUE;

          public void visitCachedData(CachedData d)
            throws DPSException
          {
               long expireWhen = d.expiresIn();
               if (expireWhen < expiryMillis)
                    expiryMillis = expireWhen;
          }
     }
When the apply completes with the ExpiryVisitor visitor you can reference the expiryMillis field to determine when to synchronize with the EVE servers.

Example: Using Data Model Elements with Hibernate

Some data model classes store information using Hibernate instead of storing data directly in instance fields. An example is CharacterWalletJournal. Data model classes which store data in Hibernate make use of one or more persistence defined in the org.dps.core.persist package. Wallet journals use the IWalletJournalRecord interface.

A new journal entry is created using one of the helpers in IEvePersistenceFactory interface. In our case, we use getNewJournalRecord. The helper methods in IEvePersistenceFactory take the key fields as arguments and return a new Hibernate record ready to be populated.

There are two ways to store new records. The first is to use one of the helper methods defined in HibernateCachedData. The alternative is to directly store records using the Hibernate methods provided by HibernateCachedData. Here's an example using both procedures: 

     IEvePersistenceFactory hibFactory = ... your persistence factory ...;
     CharacterWalletJournal charJournal = ... your char journal ...;
     IWalletJournalEntry newEntry = ... an entry from the EVE server ...;

     // Create a new journal record
     IWalletJournalRecord newRecord = hibFactory.getNewJournalRecord(keyID, charID, accountKey, newEntry.getRefID());

     ... populate the fields of newRecord ...

     // Store the new record using a helper method.  This method will 
     // throw HibernateException if it fails but will rollback the
     // transaction before doing so.
     charJournal.storeEntry(newRecord);

     // Now store the same record again using the Hibernate methods.
     // Note that since we've already stored the record with the same
     // key, the code below won't actually change the database.  If
     // we had created a new record then the database would be changed
     // to store the new record.
     try {
          charJournal.begin();
          charJournal.getSession().merge(newRecord);
          charJournal.commit();
     } catch (HibernateException h) {
          charJournal.rollback();
     }
The helper methods in HibernateCachedData are convenient for one-off data storage. You should normally use the Hibernate API directly for bulk updates as this will be more efficient.

To access journal records, you must use the Hibernate API to perform a select: 

     // Example: retrieve all journal entries after a given date
     CharacterWalletJournal charJournal = ... your char journal ...;
     List results;
     long afterDate = ... date after which we want journal entries ...;

     try {
          charJournal.begin();
          Query select = charJournal.getSession().createQuery(
               "from org.dps.core.persist.IWalletJournalRecord where date >= :date");
          select.setLong("date", afterDate);

          // Java will complain about the cast here.  You can safely
          // ignore, it's an annoying Hibernate "feature".
          results = select.list();

          charJournal.commit();
     } catch (HibernateException e) {
          charJournal.rollback();
     }
In the future, we plan to provide convenience methods for queries like the one above.

Using Meta-Data to Extend the Data Model

Recall that the CachedData base class allows for the storage of arbitary meta-data which implements the Serializable interface. Meta-data is stored in a simple key value map where the key is a String and the value is the meta-data.

The data model is structured to mimic the data which can be retrieved from the EVE API servers. However, the EVE API doesn't return all data which is available using the game client. For example, blueprint assets do not include material and production levels. Using meta-data, however, we can store this information ourselves in the data model. For example: 

     Asset blueprint = ... your blueprint asset ...;
     int materialEfficiency = ... your blueprint ME level ...;
     int productionEfficiency = ... you blueprint PE level ...;

     blueprint.putMeta("ME", materialEfficiency);
     blueprint.putMeta("PE", productionEfficiency);
Later, when we need to retrieve material and production levels, we can do so as follows: 

     Asset blueprint = ... your blueprint asset ...;

     // Two ways to do this:
     // 1) explict cast
     int materialEfficiency = (Integer) blueprint.getMeta("ME");

     // 2) implicit cast
     int productionEfficiency = blueprint.getMeta("PE", Integer.class);
As shown in the example above, you can choose to use an explicit or implicit cast to recover the type of your meta-data. There is no material difference in the two approaches, choose whichever you prefer according to your coding style.

Appendix: Complete Data Object Model Graph

Each data model object is documented below. Indentation is used to indicate child data objects.