A data source backed by a SPARQL endpoint accessed through the SPARQL protocol. More...

Inheritance diagram for RemoteSPARQLDataSource:

Collaboration diagram for RemoteSPARQLDataSource:

[legend]

Public Member Functions
	RemoteSPARQLDataSource (String endpointURL, String defaultGraphURI)

	RemoteSPARQLDataSource (String endpointURL, String defaultGraphURI, boolean supportsSPARQL11, Set< String > resourceQueries, Set< String > propertyQueries, Set< String > inversePropertyQueries, Set< String > anonPropertyQueries, Set< String > anonInversePropertyQueries, CachedPropertyCollection highIndegreeProperties, CachedPropertyCollection highOutdegreeProperties)

void	setGraphContentType (String mediaType)
	Sets the content type to ask for in graph requests (CONSTRUCT and DESCRIBE) to the remote SPARQL endpoint. More...

void	addGraphQueryParam (String param)

void	addSelectQueryParam (String param)

boolean	canDescribe (String absoluteIRI)
	Indicates whether this data source may have some information about a given IRI. More...

Model	describeResource (String resourceURI)
	Returns a subgraph of the data source describing one resource. More...

Map< Property, Integer >	getHighIndegreeProperties (String resourceURI)
	If describeResource(String) omits properties of high indegree, then those properties must be returned here with the count of arcs. More...

Map< Property, Integer >	getHighOutdegreeProperties (String resourceURI)
	If describeResource(String) omits properties of high outdegree, then those properties must be returned here with the count of arcs. More...

Model	listPropertyValues (String resourceURI, Property property, boolean isInverse)
	Returns a subgraph of the data source. More...

List< Resource >	getIndex ()
	A list of IRI resources described in this data source. More...

AutocompleteEngine< SearchRecord >	getLabelIndex ()

String	getPreviousDescribeQuery ()

Model	describeResource (String absoluteIRI, String language)
	Returns a subgraph of the data source describing one resource. More...

Public Attributes
final String	endpointURL

Static Public Attributes
static final int	MAX_INDEX_SIZE = 1000000

Package Attributes
QueryEngineHTTP	endpoint

Private Member Functions
String[]	parseQueryParam (String param)

Map< Property, Integer >	getHighDegreeProperties (String query, String resourceURI)

Model	execQueryGraph (String query)

ResultSet	execQuerySelect (String query)

String	preProcessQuery (String query, String resourceURI)

String	preProcessQuery (String query, String resourceURI, Property property)

String	replaceString (String text, String searchString, String replacement)

String	toSPARQLArgumentList (Collection<? extends RDFNode > values)

Private Attributes
final String	defaultGraphURI

final boolean	supportsSPARQL11

final Set< String >	resourceQueries

final Set< String >	propertyQueries

final Set< String >	inversePropertyQueries

final Set< String >	anonPropertyQueries

final Set< String >	anonInversePropertyQueries

final CachedPropertyCollection	highIndegreeProperties

final CachedPropertyCollection	highOutdegreeProperties

String	previousDescribeQuery

String	contentType = null

final Set< String[]>	queryParamsSelect = new HashSet<String[]>()

final Set< String[]>	queryParamsGraph = new HashSet<String[]>()

AutocompleteEngine< SearchRecord >	engine

Pattern	queryParamPattern = Pattern.compile("(.?)=(.)")

Detailed Description

A data source backed by a SPARQL endpoint accessed through the SPARQL protocol.

Constructor & Destructor Documentation

◆ RemoteSPARQLDataSource() [1/2]

RemoteSPARQLDataSource	(	String	endpointURL,
		String	defaultGraphURI
	)

                                                                                   {
                 this(endpointURL, defaultGraphURI, false, null, null, null, null, null, null, null);
         }

References RemoteSPARQLDataSource.defaultGraphURI, and RemoteSPARQLDataSource.endpointURL.

◆ RemoteSPARQLDataSource() [2/2]

RemoteSPARQLDataSource	(	String	endpointURL,
		String	defaultGraphURI,
		boolean	supportsSPARQL11,
		Set< String >	resourceQueries,
		Set< String >	propertyQueries,
		Set< String >	inversePropertyQueries,
		Set< String >	anonPropertyQueries,
		Set< String >	anonInversePropertyQueries,
		CachedPropertyCollection	highIndegreeProperties,
		CachedPropertyCollection	highOutdegreeProperties
	)

                                                                                                                            {
                 this.endpointURL = endpointURL;
                 this.defaultGraphURI = defaultGraphURI;
                 this.supportsSPARQL11 = supportsSPARQL11;
                 if (resourceQueries == null || resourceQueries.isEmpty()) {
                         resourceQueries = supportsSPARQL11 ?
                                 new HashSet<String>(Arrays.asList(new String[]{
                                         "CONSTRUCT {?__this__ ?p ?o} WHERE {?__this__ ?p ?o. FILTER (?p NOT IN ?__high_outdegree_properties__)}",
                                         "CONSTRUCT {?s ?p ?__this__} WHERE {?s ?p ?__this__. FILTER (?p NOT IN ?__high_indegree_properties__)}"
                                 })) :
                                 Collections.singleton("DESCRIBE ?__this__");
                 }
                 if (propertyQueries == null || propertyQueries.isEmpty()) {
                         propertyQueries = Collections.singleton(
                                         "CONSTRUCT {?__this__ ?__property__ ?x} WHERE {?__this__ ?__property__ ?x. FILTER (!isBlank(?x))}");
                 }
                 if (inversePropertyQueries == null || inversePropertyQueries.isEmpty()) {
                         inversePropertyQueries = Collections.singleton(
                                         "CONSTRUCT {?x ?__property__ ?__this__} WHERE {?x ?__property__ ?__this__. FILTER (!isBlank(?x))}");
                 }
                 if (anonPropertyQueries == null || anonPropertyQueries.isEmpty()) {
                         anonPropertyQueries = Collections.singleton(
                                         "DESCRIBE ?x WHERE {?__this__ ?__property__ ?x. FILTER (isBlank(?x))}");
                 }
                 if (anonInversePropertyQueries == null || anonInversePropertyQueries.isEmpty()) {
                         anonInversePropertyQueries = Collections.singleton(
                                         "DESCRIBE ?x WHERE {?x ?__property__ ?__this__. FILTER (isBlank(?x))}");
                 }
                 this.resourceQueries = resourceQueries;
                 this.propertyQueries = propertyQueries;
                 this.inversePropertyQueries = inversePropertyQueries;
                 this.anonPropertyQueries = anonPropertyQueries;
                 this.anonInversePropertyQueries = anonInversePropertyQueries;
                 
                 this.highIndegreeProperties = highIndegreeProperties;
                 this.highOutdegreeProperties = highOutdegreeProperties;
         }

References RemoteSPARQLDataSource.anonInversePropertyQueries, RemoteSPARQLDataSource.anonPropertyQueries, RemoteSPARQLDataSource.defaultGraphURI, RemoteSPARQLDataSource.endpointURL, RemoteSPARQLDataSource.highIndegreeProperties, RemoteSPARQLDataSource.highOutdegreeProperties, RemoteSPARQLDataSource.inversePropertyQueries, RemoteSPARQLDataSource.propertyQueries, RemoteSPARQLDataSource.resourceQueries, and RemoteSPARQLDataSource.supportsSPARQL11.

Member Function Documentation

◆ addGraphQueryParam()

void addGraphQueryParam ( String param )

                                                      {
                 queryParamsGraph.add(parseQueryParam(param));
         }

References RemoteSPARQLDataSource.parseQueryParam(), and RemoteSPARQLDataSource.queryParamsGraph.

Referenced by Dataset.buildDataSource().

◆ addSelectQueryParam()

void addSelectQueryParam ( String param )

                                                       {
                 queryParamsSelect.add(parseQueryParam(param));
         }

References RemoteSPARQLDataSource.parseQueryParam(), and RemoteSPARQLDataSource.queryParamsSelect.

Referenced by Dataset.buildDataSource().

◆ canDescribe()

boolean canDescribe ( String absoluteIRI )

Indicates whether this data source may have some information about a given IRI.

If this is false, a client should not bother to call describeResource(String). This method is to allow for optimizations and should respond very fast.

It is also used by the UI to decide whether a click on a resource should go to a Pubby page (if true) or out to the Web (if false).

Parameters

absoluteIRI The IRI of a resource to be described

Returns: true if this data source might have something about it

Implements DataSource.

                                                        {
                 return true;
         }

◆ describeResource() [1/2]

Model describeResource	(	String	absoluteIRI,
		String	language
	)

Returns a subgraph of the data source describing one resource.

This should include both incoming and outgoing triples. However, it should exclude outgoing arcs where the property is a high-outdegree property, and it should exclude incoming arcs where the property is a high-indegree property. If labels for other resources are included in the result, then they will be used.

Parameters

absoluteIRI The IRI of the resource to be described

Returns: A subgraph of the data source describing the resource.

Implements DataSource.

                                                                            {
                 // TODO Auto-generated method stub
                 return null;
         }

◆ describeResource() [2/2]

Model describeResource ( String absoluteIRI )

Returns a subgraph of the data source describing one resource.

This should include both incoming and outgoing triples. However, it should exclude outgoing arcs where the property is a high-outdegree property, and it should exclude incoming arcs where the property is a high-indegree property. If labels for other resources are included in the result, then they will be used.

Parameters

absoluteIRI The IRI of the resource to be described

Returns: A subgraph of the data source describing the resource.

Implements DataSource.

                                                           {
                 // Loop over resource description queries, join results in a single model.
                 // Process each query to replace place-holders of the given resource.
                 Model model = ModelFactory.createDefaultModel();
                 for (String query: resourceQueries) {
                         Model result = execQueryGraph(preProcessQuery(query, resourceURI));
                         model.add(result);
                         model.setNsPrefixes(result);
                 }
                 return model;
         }

References RemoteSPARQLDataSource.execQueryGraph(), RemoteSPARQLDataSource.preProcessQuery(), and RemoteSPARQLDataSource.resourceQueries.

◆ execQueryGraph()

Model execQueryGraph ( String query )

private

                                                    {
                 Model model = ModelFactory.createDefaultModel();
                 previousDescribeQuery = query;
  
                 // Since we don't know the exact query type (e.g. DESCRIBE or CONSTRUCT),
                 // and com.hp.hpl.jena.query.QueryFactory could throw exceptions on
                 // vendor-specific sections of the query, we use the lower-level
                 // com.hp.hpl.jena.sparql.engine.http.HttpQuery to execute the query and
                 // read the results into model.
                 
                 HttpQuery httpQuery = new HttpQuery(endpointURL);
                 httpQuery.addParam("query", query);
                 if (defaultGraphURI != null) {
                         httpQuery.addParam("default-graph-uri", defaultGraphURI);
                 }
                 for (String[] param: queryParamsGraph) {
                         httpQuery.addParam(param[0], param[1]);
                 }
                 
                 // The rest is more or less a copy of QueryEngineHTTP.execModel()
                 httpQuery.setAccept(contentType);
                 InputStream in = httpQuery.exec();
  
                 // Don't assume the endpoint actually gives back the content type we
                 // asked for
                 String actualContentType = httpQuery.getContentType();
  
                 // If the server fails to return a Content-Type then we will assume
                 // the server returned the type we asked for
                 if (actualContentType == null || actualContentType.equals("")) {
                         actualContentType = contentType;
                 }
  
                 // Try to select language appropriately here based on the model content
                 // type
                 Lang lang = WebContent.contentTypeToLangResultSet(actualContentType);
                 if (!RDFLanguages.isTriples(lang))
                         throw new QueryException("Endpoint <" + endpointURL + 
                                         "> returned Content Type: " + actualContentType
                                         + " which is not a supported RDF graph syntax");
                 RDFDataMgr.read(model, in, lang);
  
                 // Skip prefixes ns1, ns2, etc, which are usually
                 // auto-assigned by the endpoint and do more harm than good
                 for (String prefix: model.getNsPrefixMap().keySet()) {
             if (prefix.matches("^ns[0-9]+$")) {
                 model.removeNsPrefix(prefix);
             }
         }
                 
                 return model;
         }

References RemoteSPARQLDataSource.contentType, RemoteSPARQLDataSource.defaultGraphURI, RemoteSPARQLDataSource.endpointURL, RemoteSPARQLDataSource.previousDescribeQuery, and RemoteSPARQLDataSource.queryParamsGraph.

Referenced by RemoteSPARQLDataSource.describeResource(), and RemoteSPARQLDataSource.listPropertyValues().

◆ execQuerySelect()

ResultSet execQuerySelect ( String query )

private

                                                         {
                 if(this.endpoint!=null && !this.endpoint.isClosed()) {
                         this.endpoint.close();
                 }
                 this.endpoint = new QueryEngineHTTP(endpointURL, query);
                 if (defaultGraphURI != null) {
                         endpoint.setDefaultGraphURIs(Collections.singletonList(defaultGraphURI));
                 }
                 for (String[] param: queryParamsSelect) {
                         endpoint.addParam(param[0], param[1]);
                 }
                 return endpoint.execSelect();
         }

References RemoteSPARQLDataSource.defaultGraphURI, RemoteSPARQLDataSource.endpoint, RemoteSPARQLDataSource.endpointURL, and RemoteSPARQLDataSource.queryParamsSelect.

Referenced by RemoteSPARQLDataSource.getHighDegreeProperties(), RemoteSPARQLDataSource.getIndex(), and RemoteSPARQLDataSource.getLabelIndex().

◆ getHighDegreeProperties()

Map<Property, Integer> getHighDegreeProperties	(	String	query,
		String	resourceURI
	)

private

                                             {
                 if (!supportsSPARQL11) return null;
                 query = preProcessQuery(query, resourceURI);
                 ResultSet rs = execQuerySelect(query);
                 Map<Property, Integer> results = new HashMap<Property, Integer>();
                 while (rs.hasNext()) {
                         QuerySolution solution = rs.next();
                         if (!solution.contains("p") || !solution.contains("count")) continue;
                         Resource p = solution.get("p").asResource();
                         int count = solution.get("count").asLiteral().getInt();
                         results.put(ResourceFactory.createProperty(p.getURI()), count);
                 }
                 return results;
         }

References RemoteSPARQLDataSource.execQuerySelect(), RemoteSPARQLDataSource.preProcessQuery(), and RemoteSPARQLDataSource.supportsSPARQL11.

Referenced by RemoteSPARQLDataSource.getHighIndegreeProperties(), and RemoteSPARQLDataSource.getHighOutdegreeProperties().

◆ getHighIndegreeProperties()

Map<Property, Integer> getHighIndegreeProperties ( String resourceIRI )

If describeResource(String) omits properties of high indegree, then those properties must be returned here with the count of arcs.

If high-indegree properties are not omitted, or the resource doesn't have any, then an empty map or null may be returned. Entries with value 0 will be ignored.

Parameters

resourceIRI The IRI of the resource to be described

Returns: A map containing high-indegree properties with number of arcs for the resource

Implements DataSource.

                                                                                     {
                 return getHighDegreeProperties(
                                 "SELECT ?p (COUNT(?s) AS ?count) " +
                                 "WHERE { " +
                                 "  ?s ?p ?__this__. " +
                                 "  FILTER (?p IN ?__high_indegree_properties__)" +
                                 "}" +
                                 "GROUP BY ?p",
                                 resourceURI);
         }

References RemoteSPARQLDataSource.getHighDegreeProperties().

◆ getHighOutdegreeProperties()

Map<Property, Integer> getHighOutdegreeProperties ( String resourceIRI )

If describeResource(String) omits properties of high outdegree, then those properties must be returned here with the count of arcs.

If high-outdegree properties are not omitted, or the resource doesn't have any, then an empty map or null may be returned. Entries with value 0 will be ignored.

Parameters

resourceIRI The IRI of the resource to be described

Returns: A map containing high-outdegree properties with number of arcs for the resource

Implements DataSource.

                                                                                      {
                 return getHighDegreeProperties(
                                 "SELECT ?p (COUNT(?o) AS ?count) " +
                                 "WHERE { " +
                                 "  ?__this__ ?p ?o. " +
                                 "  FILTER (?p IN ?__high_outdegree_properties__)" +
                                 "}" +
                                 "GROUP BY ?p", 
                                 resourceURI);
         }

References RemoteSPARQLDataSource.getHighDegreeProperties().

◆ getIndex()

List<Resource> getIndex ( )

A list of IRI resources described in this data source.

Ordering is implementation-defined. Usually a reasonable limit should be applied to the number of resources returned.

Implements DataSource.

                                          {
                 System.out.println("Index SPARQL Query");
                 List<Resource> result = new ArrayList<Resource>();
                 ResultSet rs = execQuerySelect(
                                 "SELECT DISTINCT ?s { " +
                                 "?s ?p ?o " +
                                 "FILTER (isURI(?s)) " +
                                 "} LIMIT " + DataSource.MAX_INDEX_SIZE);
                 while (rs.hasNext()) {
                         Resource s=rs.next().getResource("s");
                         //System.out.println("1. "+s.toString());
                         result.add(s);
                 }
                 System.out.println("Attempting second query");
                 if (result.size() < DataSource.MAX_INDEX_SIZE) {
                         rs = execQuerySelect(
                                         "SELECT DISTINCT ?o { " +
                                         "?s ?p ?o " +
                                         "FILTER (isURI(?o)) " +
                                         "} LIMIT " + (DataSource.MAX_INDEX_SIZE - result.size()));
                         while (rs.hasNext()) {
                                 Resource s=rs.next().getResource("o");
                                 //System.out.println("2. "+s.toString());
                                 result.add(s);
                         }
                 }
                 System.out.println(result.size()+" - "+result.isEmpty());
                 return result;
         }

References RemoteSPARQLDataSource.execQuerySelect(), and DataSource.MAX_INDEX_SIZE.

◆ getLabelIndex()

AutocompleteEngine<SearchRecord> getLabelIndex ( )

Implements DataSource.

                                                                {
                 System.out.println("SPARQL Datasource: Get Label Index!!!!");
                 if(engine==null) {
                         engine=SearchIndexInstance.getInstance();
                         System.out.println("Building Label Index....");
                         ResultSet rs = execQuerySelect(
                                         "SELECT DISTINCT ?s ?label WHERE { " +
                                         "?s <http://www.w3.org/2000/01/rdf-schema#label> ?label . " +
                                         "FILTER (isURI(?s)) " +
                                         "} LIMIT " + DataSource.MAX_INDEX_SIZE*6);
                         Integer i=0;
                         while (rs.hasNext()) {
                                 i++;
                                 QuerySolution st=rs.next();
                                 System.out.println(st.getLiteral("label").getString());
                                 engine.add(new SearchRecord(st.getLiteral("label").getString(),st.getResource("s")));
                         }
                         System.out.println("Got "+i+" labels!");
                 }
                 return engine;
         }

References RemoteSPARQLDataSource.engine, RemoteSPARQLDataSource.execQuerySelect(), SearchIndexInstance.getInstance(), and DataSource.MAX_INDEX_SIZE.

◆ getPreviousDescribeQuery()

String getPreviousDescribeQuery ( )

                                                  {
                 return previousDescribeQuery;
         }

References RemoteSPARQLDataSource.previousDescribeQuery.

Referenced by MetadataConfiguration.parsePlaceholder().

◆ listPropertyValues()

Model listPropertyValues	(	String	resourceIRI,
		Property	property,
		boolean	isInverse
	)

Returns a subgraph of the data source.

It lists the values of a particular property of a particular resource. Where values are blank nodes, a complete description of these anonymous resources must be included.

Parameters

resourceIRI	The resource to be examined
property	The property we're interested in
isInverse	Are we interested in outgoing arcs (`false`) or incoming (`true`)?

Returns: A subgraph of the data source.

Implements DataSource.

                                            {
                 // Loop over the queries, join results in a single model.
                 // Process each query to replace place-holders of the given resource and property.
                 List<String> queries = new ArrayList<String>();
                 queries.addAll(isInverse ? inversePropertyQueries : propertyQueries);
                 queries.addAll(isInverse ? anonInversePropertyQueries : anonPropertyQueries);
                 Model model = ModelFactory.createDefaultModel();
                 for (String query: queries) {
                         String preprocessed = preProcessQuery(query, resourceURI, property);
                         Model result = execQueryGraph(preprocessed);
                         model.add(result);
                         model.setNsPrefixes(result);
                 }
                 return model;
         }

References RemoteSPARQLDataSource.anonInversePropertyQueries, RemoteSPARQLDataSource.anonPropertyQueries, RemoteSPARQLDataSource.execQueryGraph(), RemoteSPARQLDataSource.inversePropertyQueries, RemoteSPARQLDataSource.preProcessQuery(), and RemoteSPARQLDataSource.propertyQueries.

◆ parseQueryParam()

String [] parseQueryParam ( String param )

private

                                                        {
                 Matcher match = queryParamPattern.matcher(param);
                 if (!match.matches()) {
                         throw new ConfigurationException("Query parameter \"" + param + 
                                         "\" is not in \"param=value\" form");
                 }
                 return new String[]{match.group(1), match.group(2)};
         }

References RemoteSPARQLDataSource.queryParamPattern.

Referenced by RemoteSPARQLDataSource.addGraphQueryParam(), and RemoteSPARQLDataSource.addSelectQueryParam().

◆ preProcessQuery() [1/2]

String preProcessQuery	(	String	query,
		String	resourceURI
	)

private

                                                                          {
                 return preProcessQuery(query, resourceURI, null);
         }

Referenced by RemoteSPARQLDataSource.describeResource(), RemoteSPARQLDataSource.getHighDegreeProperties(), and RemoteSPARQLDataSource.listPropertyValues().

◆ preProcessQuery() [2/2]

String preProcessQuery	(	String	query,
		String	resourceURI,
		Property	property
	)

private

                                                                                             {
                 String result = replaceString(query, "?__this__", "<" + resourceURI + ">");
                 if (property != null) {
                         result = replaceString(result, "?__property__", "<" + property.getURI() + ">");
                 }
                 result = replaceString(result, "?__high_indegree_properties__", 
                                 toSPARQLArgumentList(highIndegreeProperties == null ? null : highIndegreeProperties.get()));
                 result = replaceString(result, "?__high_outdegree_properties__", 
                                 toSPARQLArgumentList(highOutdegreeProperties == null ? null : highOutdegreeProperties.get()));
                 return result;
         }

References VocabularyStore.CachedPropertyCollection.get(), RemoteSPARQLDataSource.highIndegreeProperties, RemoteSPARQLDataSource.highOutdegreeProperties, RemoteSPARQLDataSource.replaceString(), and RemoteSPARQLDataSource.toSPARQLArgumentList().

◆ replaceString()

String replaceString	(	String	text,
		String	searchString,
		String	replacement
	)

private

                                                                                            {
                 int start = 0;
                 int end = text.indexOf(searchString, start);
                 if (end == -1) {
                         return text;
                 }
  
                 int replacementLength = searchString.length();
                 StringBuffer buf = new StringBuffer();
                 while (end != -1) {
                         buf.append(text.substring(start, end)).append(replacement);
                         start = end + replacementLength;
                         end = text.indexOf(searchString, start);
                 }
                 buf.append(text.substring(start));
                 return buf.toString();
         }

Referenced by RemoteSPARQLDataSource.preProcessQuery().

◆ setGraphContentType()

void setGraphContentType ( String mediaType )

Sets the content type to ask for in graph requests (CONSTRUCT and DESCRIBE) to the remote SPARQL endpoint.

                                                           {
                 this.contentType = mediaType;
         }

Referenced by Dataset.buildDataSource().

◆ toSPARQLArgumentList()

String toSPARQLArgumentList ( Collection<? extends RDFNode > values )

private

                                                                                   {
                 if (values == null) return "()";
                 StringBuilder result = new StringBuilder();
                 result.append('(');
                 boolean isFirst = true;
                 for (RDFNode term: values) {
                         if (!isFirst) {
                                 result.append(", ");
                         }
                         if (term.isURIResource()) {
                                 result.append('<');
                                 result.append(term.asResource().getURI());
                                 result.append('>');
                         } else {
                                 throw new IllegalArgumentException(
                                                 "toSPARQLArgumentList is only implemented for URIs; " + 
                                                 "called with term " + term);
                         }
                         isFirst = false;
                 }
                 result.append(')');
                 return result.toString();
         }