New personal weblog

My wife and I have started a new personal weblog, about our emigration to New Zealand: http://naarnz.blogspot.com/. This new weblog will be in Dutch, as its primary purpose is to keep our families and friends back in Holland and Belgium up to date. So, if you understand a bit of Dutch and want to know more about our decision to move to the other side of the world and follow how we fare, have a look!

SWC 0.10.1 - bugfix release

I've just put version 0.10.1 of the SWC online. This is a bugfix release that addresses a number of annoying problems:

• several buttons on the construct query tab were not visible due to layout problems;
• the namespaces tab and context comboboxes were not properly updated after uploading new data;
• when adding a file was aborted for any reason, the tool forgot to close the file stream.

As always, we welcome your feedback. By the way, for anyone interested in discussing the tool with us, there is a mailinglist for this purpose at Sourceforge.

Cooking with Sesame: the RepositoryManager

The Sesame Cookbook has moved to my new site: http://rivuli-development.com/

(This is part of series of weblog postings on using the Sesame framework more effectively.)

As is also explained in the official Sesame user manual, the Repository API (see also the Sesame Javadoc) is the starting point for anyone wanting to program against Sesame. While the user manual gives some good examples on getting you started with using that API, however, there are some tips and tricks for more effective use that I thought I might share, in a series of weblog postings I'm planning on "Cooking with Sesame". This first episode, I'll go a bit deeper into creation and management of Sesame repositories, specifically using the RepositoryManager and related classes.

Creating repositories: the basics

Sesame supports a number of different types of Repository. The two that are most commonly used are the SailRepository, for local repositories, and the HTTPRepository, a proxy for repositories on a remote Sesame server.

A SailRepository is an object that represents a local Sesame database. Note that, although we call it a database, this does not automatically mean persistence, or anything "heavy" - it can be a simple in-memory representation of an RDF graph. In fact, the easiest way to load any RDF file into memory using Sesame is by creating a SailRepository using an in-memory database.

The type of database (main memory, native, relational, ...) is determined by the SAIL stack provided to the SailRepository (the SAIL API is a Sesame system-internal API that is used to wrap the details of the underlying storage and reasoning mechanism). For example, for creating a simple in-memory repository, we do the following:

import org.openrdf.repository.Repository;
import org.openrdf.repository.sail.SailRepository;
import org.openrdf.sail.memory.MemoryStore;

Repository repository = new SailRepository(new MemoryStore());

repository.initialize();

The MemoryStore object referred to here is a SAIL backend that simply stores all RDF in-memory. Sesame provides various different SAIL backends, as well as various options for configuring each SAIL backend. Some further examples of the creation of various types of repositories (including also the use of the HTTPRepository) are shown in chapter 8 of the Sesame user manual.

Once you have created a Repository object like this, you can open a RepositoryConnection on it to make further use of the repository: adding or removing data, executing queries, and so on.

The RepositoryManager

So far so good: we can easily create and use various different types of Sesame repositories. However, when developing an application in which you have to keep track of several repositories, sharing references to these repositories between different parts of your code can quickly become complex. Of course, you could declare some static references for use throughout your application, but if several repositories have to be juggled this way, this too can become cumbersome. Ideal would be one central location where all information on the repositories in use (including id, type, directory for persistent data storage, etc.) is kept. This is the role of the Sesame RepositoryManager.

Using the RepositoryManager for handling repository creation and administration offers a number of advantages, including:

• a single RepositoryManager object can be more easily shared throughout your application than a host of static references to individual repositories;
• you can more easily create and manage repositories 'on-the-fly', for example if your application requires creation of new repositories on user input;
• the RepositoryManager stores your configuration, including all repository data, in one central spot on the file system.

The RepositoryManager comes in two flavours: the LocalRepositoryManager and the RemoteRepositoryManager.

A LocalRepositoryManager manages repository handling for you locally, and is always created using a (local) directory. This directory is where all Sesame repositories handled by the manager store their data, and also where the LocalRepositoryManager itself stores its configuration data.

You create a new LocalRepositoryManager as follows:

import java.io.File;
import org.openrdf.repository.manager.LocalRepositoryManager;

File baseDir = new File("/path/to/storage/dir/");
LocalRepositoryManager manager =
               new LocalRepositoryManager(baseDir);
manager.initialize();

To use a LocalRepositoryManager to create and manager repositories works slightly differently from what we've seen above about creating Sesame repositories. The LocalRepositoryManager works by providing it with RepositoryConfig objects, which are declarative specifications of the repository you want. You add a RepositoryConfig object for your new repository, and then request the actual Repository back from the LocalRepositoryManager:

import org.openrdf.repository.config.RepositoryConfig;

String repositoryId = "test-db";
RepositoryConfig repConfig = 
      new RepositoryConfig(repositoryId, repositoryTypeSpec);

manager.addRepositoryConfig(repConfig);
Repository repository = manager.getRepository(repositoryId);

In the above bit of code, you may have noticed that I provide an innocuous-looking variable called repositoryTypeSpec to the constructor of our RepositoryConfig. This variable is an instance of a class called RepositoryImplConfig, and this specifies the actual configuration of our new repository: what backends to use, whether or not to use inferencing, and so on.

Creating a RepositoryImplConfig object can be done in two ways: programmatically, or by reading a (RDF) config file. Here, I will show the programmatic way (in a future article we may look at the other method, using RDF config files).

import org.openrdf.sail.config.SailImplConfig;
import org.openrdf.sail.memory.config.MemoryStoreConfig;
import org.openrdf.repository.config.RepositoryImplConfig;
import org.openrdf.repository.sail.config.SailRepositoryConfig;

// create a configuration for the SAIL stack
SailImplConfig backendConfig = new MemoryStoreConfig();

// create a configuration for the repository implementation
RepositoryImplConfig repositoryTypeSpec = 
                new SailRepositoryConfig(backendConfig);

As you can see, we use a class called MemoryStoreConfig for specifying the type of storage backend we want. This class resides in a config sub-package of the memory store package (org.openrdf.sail.memory). Each particular type of SAIL in Sesame has such a config class.

As a second example, we create a slightly more complex type of store: still in-memory, but this time we want it to use the memory store's persistence option, and we also want to add RDFS inferencing. In Sesame, RDFS inferencing is provided by a separate SAIL implementation, which can be 'stacked' on top of another SAIL. We follow that pattern in the creation of our config object:

import org.openrdf.sail.inferencer.fc.config.ForwardChainingRDFSInferencerConfig;

// create a configuration for the SAIL stack
boolean persist = true;
SailImplConfig backendConfig = new MemoryStoreConfig(persist);

// stack an inferencer config on top of our backend-config
backendConfig = 
     new ForwardChainingRDFSInferencerConfig(backendConfig);

// create a configuration for the repository implementation
SailRepositoryConfig repositoryTypeSpec = 
                new SailRepositoryConfig(backendConfig);

The RemoteRepositoryManager

A useful feature of Sesame is that most its APIs are transparent with respect to whether you are working locally or remote. This is the case for the Sesame repositories, but also for the RepositoryManager. In the above examples, we have used a LocalRepositoryManager, creating Sesame repositories for local use. However, it is also possible to use a RemoteRepositoryManager, using it to create and manage Sesame repositories residing on a remotely running Sesame server.

A RemoteRepositoryManager is initialized as follows:

import org.openrdf.repository.manager.RemoteRepositoryManager;

// URL of the remote Sesame server we want to access
String serverUrl = "http://localhost:8080/openrdf-sesame";
RemoteRepositoryManager manager = 
              new RemoteRepositoryManager(serverUrl);
manager.initialize();

Once initialized, the RemoteRepositoryManager can be used in the same fashion as the LocalRepositoryManager: creating new repositories, requesting references to existing repositories, and so on.

Disclaimer: although I have done my best to check the correctness, I can give no guarantees that any of the provided code examples work as expected. Also I make no claims on whether any of this is the 'official' way of working with Sesame.

SWC 0.10 - OWL Reasoning support

Anton and I have just released version 0.10 of SWC, the Sesame Windows Client. The main new feature of this release is support for OWL reasoning.

The Sesame Windows Client now includes the latest version of Ontotext's SwiftOWLIM store, a high-performance in-memory store and OWL reasoner. SWC allows you to create a local SwiftOWLIM store like any other Sesame repository, with the added advantage that it supports OWL semantics. OWL reasoning made easy!

Also, we have made a number of improvements in the performance of data upload and data export, as well as several minor changes in the UI.

Next steps in the development of SWC are: adding support for other triplestores that have a Sesame API bridge (including Virtuoso and AllegroGraph), as well as improving the SPARQL endpoint support (by allowing additional parameters to be set). And oh yeah, we're still looking for a good new name for the SWC...

Semantic Mediawiki Conference (SMWCon) Fall 2010

I just came back from an exhausting but highly inspirational 2 days of talks and demos about Semantic Mediawiki and various extensions and use cases: SMWCon Fall 2010 (hosted at the Open University in Amsterdam).

Some of my personal highlights (in no particular order):

• Semantic Maps as presented by Jeroen de Dauw. This was a bit of an eye-opener, as I had previously briefly thought about using something like this, but decided it wouldn't be worth the bother if it involved having to type in a lot of coordinates.
  Well guess what, you don't have to: it provides full geocoding functionality, effectively allowing you just to type in a street address. It ties in with Google maps as well as Open StreetMaps, Yahoo maps and OpenLayers. Overall it looks just massively useful.
• WikiTags by Jesse Wang: a very impressive demo on how to link the world of (semantic) wikis to the world of MS Office. Automated annotation of terms in MS Word, live links between Outlook and the wiki, quick and easy adding of e-mail messages to the wiki, etc. etc. The thought behind this effort appeals to me: don't rage against the machine but accept that MS Office tools are the daily working environment for many people and organisations, and cater to that.
• The Suite of Halo extensions presented by Daniel Hansch: although I was already aware of the Halo extensions I had not yet completely grokked the full range of features and improvements it encompasses. In particular its support for fine-grained access control is something that captured my attention. On the ToDo list to take for a test drive.
• SparqlExtension by Alfredas Chmieliauskas and Chris Davis: particularly impressive about this demo was the way in which this extension enables integration of (RDF) data from many different sources and creating all sorts of reportings on that data. Can't wait to tweak this one to talk to a Sesame server and start testing it on our internal group wiki.
• Rudi van Bavel and Michael Cariaso both showed very interesting stuff involving the creation, maintenance and use of SNPedia, a wiki containing huge amounts of human genetic data.

To be honest, though, I found all presentations and demos, not just the ones mentioned above, interesting and engaging. The atmosphere throughout the two days was positive and pragmatic, with lots of interesting group discussion going on. It was great to see people from so many different background coming together to share experiences and show new and innovative ways in which the SMW platform can be used and extended.

SWC 0.9 released: out-of-the box triplestore and SPARQL query library

Hot on the heels of the previous SWC release comes another major update of the Sesame Windows Client. Version 0.9 has a whole range of useful new features and improvements: 

• You can now create and use local repositories (many thanks to the IKVM developers and Enrico Minack for making this possible);
• "My Query Book" allows you to save and reuse your favorite SPARQL queries;
• You can now open the result of a SPARQL SELECT query in a new (paged) window;
• You can now directly save the result of a SPARQL SELECT query as comma-separated values, XML, or JSON;
• Your most recently used server URLs are now remembered;
• Autocompletion when typing in a server URL.

The use of local repositories means that you can use the SWC as a quick and easy RDF triplestore. Create a new store, add your RDF data, and start querying. No need to install a separate triplestore server.

Query Book screenshot (click to enlarge)

The Query Book allows you to not only save your queries, but also to document them. This way, you can quickly create a library of useful SPARQL queries and document not only what they do, but also on which repository or SPARQL endpoint the query is most likely to give a useful result. The Query Book's search feature makes sure that even in a large library you can still easily find back any query.

Another new feature is the opening of a SPARQL SELECT query result in a new (paged) window. What is especially useful about this is that you can now execute one query, keep the result in a separate window, and then execute another query and compare its result with the previous one.

If you want to save the result of a SPARQL query, you can now do so, in a choice of three formats: comma-separated values (CSV), which is useful for import in e.g. Excel, is the default choice, but the official SPARQL Query Results XML format and the SPARQL JSON format are also suppported.

Both Anton and I are quite happy with this new release, and hope you find it useful as well. As always, the new release can be found on the  project homepage on Sourceforge.

Sesame 2 Windows Client - or is that SPARQL Windows/Linux Client?

I've just released a new version of the Sesame 2 Windows Client. Thanks to a new co-developer, Anton Andreev (of OntoText), the SWC tool is now a full-fledged SPARQL client: it can connect to any SPARQL endpoint, not just Sesame servers.

Apart from this new feature, the tool also has a couple of other improvements:

• context information fetching can now be disabled when connecting to large repositories;
• namespace clauses can now be automatically generated, based on the prefixes used in the query.

The only problem with this new release is that the name of the tool is now even more of a mismatch. Not only is the Sesame 2 Windows Client not just for Windows (it also runs on Linux under Mono), but now it is also not just for Sesame. A free drink to whoever suggests a good new name!

As always, the new release can be found on the  project homepage on Sourceforge. The source code can from now on be found in the Sourceforge SVN repository.

Moving to New Zealand

Karen and I are taking the plunge together. Well, another plunge: we are emigrating to New Zealand at the end of this year.

This is not, of course, something you decide on a whim. We've both long felt that we might like to make a "big change" for the better, and a long holiday spent getting to know New Zealand and its people convinced us that it is everything we were hoping it would be. New Zealand is an absolutely breathtakingly beautiful place, its people are friendly and easy-going, and for what is basically a predominantly Anglosaxon culture they actually make very good coffee as well.

The plan is quite simple. We have both obtained our permanent residence visa, and will move to New Zealand, to the Wellington area, in December. Although neither of us has secured a job yet, I am actively on the lookout, and confident that something suitable will turn up. NZ and especially Wellington is full of smaller and larger IT firms, and every job vacancy site I check literally posts 2 to 6 positions for (Java) developers a day, at least. However, what I've not been able to find much of yet, is companies or institutes who work in Semantic Web/Ontologies, or who have a need for expertise in that area. If any of you have tips for me (or a job offer ;-)), it would be much appreciated!

I will post updates on developments now and then on this weblog. Oh and yes: we will of course be giving a farewell party later this year ;-)

wurvoc.org - linked open data for Quality of Life

I am proud to announce the official launch of wurvoc.org. Built by the Intelligent Systems Group at Wageningen UR Food & Biobased Research, the goal of wurvoc.org is to serve as a hub for various vocabularies and semantic web services we have developed and are still developing. Many of these vocabularies (modeled in RDF and OWL) have been built using public funding and we feel it important that this information is also publicly available in a way that is open and reusable.

Vocabularies

Currently, wurvoc.org publishes 4 vocabularies:

• The Ontology of Units of Measure models concepts and relations important to quantitative scientific research. It has a strong focus on units and quantities, measurements, and dimensions.
• The Food Additives vocabulary describes substances added to food to improve the flavour, taste, shelf-life, stability et cetera of the food product. Food additives that are approved by EFSA, the European Food Safety Authority are labelled with an E-number.
• The Dairy ontology contains a hierarchy of types of dairy products and provides general information about these products.
• The Drinks vocabulary contains a classification hierarchy of various types of beverages.

Our aim is both to publish more vocabularies and data in the near future, and to extend and improve the current vocabularies: although the data is Open, it is not yet truly Linked in the sense that it has very few relations with external datasets.

Software

We've developed the wurvoc publication platform, a set of REST services on top of the Sesame framework. In line with linked data principles, the platform publishes each vocabulary and each vocabulary term on its own URI. For example, an ontology on food additives is available at http://www.wurvoc.org/vocabularies/food-additives/, and the food additive Pectin is reprsented by http://www.wurvoc.org/vocabularies/food-additives/Pectin.

The platform uses HTTP content negotation to determine the representation format, currently supporting XHTML, RDF/XML, Turtle, N3, NTriples, and TriG (with JSON on the ToDo list).

OUM Web Services

Apart from a vocabulary publication platform, wurvoc.org also offers a number of SOAP-based web services on top of the Ontology of Units of Measure (OUM). These web services provide a number of useful functions, including lookup and matching functions as well as more advanced stuff, such as unit conversion or formulaic consistency checking.

Future plans

We are aiming to publish the publication platform software as open source as soon as possible. Various improvements are also planned, including a SPARQL endpoint and support for JSON.

I'd very much like to hear your comments on what we've brewed sofar, what you think is good and what you think could be better. I encourage you to reuse our linked data (and would appreciate it if you could let us know if you do).
I also plan to keep you up to date on our findings regarding use, performance, and general lessons that we learn after the launch of this project. Watch this space :)

Accessing DBpedia's SPARQL endpoint with Sesame

I'd like to share a tip on using Sesame as a client for SPARQL endpoints. This may be rather trivial to some but perhaps new to others.

We are developing a tool called ROC (Rapid Ontology Construction) (see our paper@ASWC'08), which is a tool that allows domain experts to quickly build a basic vocabulary for their domain, re-using existing terminology whenever possible. How this works is that the ROC tool asks the domain expert for a set of keywords that are 'core' terms of the domain, and then queries remote sources for concepts matching those terms. These are then presented to the user, who can select terms from the list, find relations to other terms, and expand the set of terms and relations, iteratively. The resulting vocabulary (or 'proto-ontology', basically a SKOS-like thesaurus) can be used as is, or can be used as input for a knowledge engineer to base a more comprehensive domain ontology on.

ROC is developed on top of Sesame, and up until now we simply supplied the tool with 'remote sources' by adding data to a locally running Sesame repository. In order to act with the LOD cloud, we obviously needed something a bit less awkward, so I started looking into ways to extend the functionality to be able to query linked open data.

Fortunately, I didn't have to look far, because Sesame actually already supports this: Sesame's client server protocol is a superset the SPARQL protocol. This I already knew, but what I hadn't yet tried was to see if that meant you could use Sesame's client libraries to query any SPARQL endpoint (instead of just connect to a remote Sesame server, which is what it is primarily designed for, after all). And guess what, it turns out that you can!

Here's a bit of code that connects to DBPedia's SPARQL endpoint and fires a query. The idea is simply to reuse Sesame HTTPRepository class, supply the endpoint URL as the server, and specify no repository:

String endpointURL = "http://dbpedia.org/sparql";
HTTPRepository dbpediaEndpoint = 
         new HTTPRepository(endpointURL, "");
dbpediaEndpoint.initialize();

RepositoryConnection conn = 
         dbpediaEndpoint.getConnection();
try {
  String sparqlQuery = 
         " SELECT * WHERE {?X ?P ?Y} LIMIT 10 ";
  TupleQuery query = conn.prepareTupleQuery(SPARQL, query);
  TupleQueryResult result = query.evaluate();

  while (result.hasNext()) {
      ... // do something linked and open
  }
}
finally {
  conn.close();
}

Now, I'm perhaps easily impressed, but to me this was beautifully easy. This makes integrating arbitrary linked open data in most of our Sesame-based tooling (including ROC) completely painless.