Working with Entities

About

An entity or a named entity is a thing with a name, a type and optionally a unique identifier (URI). Any distinct person, organization, place or product could be considered an entity. The list doesn't end here; these are just 4 highly common types of entities.

To make the definition of entities clearer, let's consider the role of the word "apple" in the following sentences:

  • Apple is a great source of vitamins.
  • Apple was founded by Steve Jobs and Steve Wozniak.

In the first sentence, "apple" refers to the following entity:

  • Text (also called surface form): "Apple"
  • Name: Apple
  • Type: Fruit, Eukaryote, Plant, etc.
  • Unique identifier: http://dbpedia.org/resource/Apple

Whereas in the second sentence, "apple" represents a different entity:

  • Text: "Apple"
  • Name: Apple Inc.
  • Type: Company, Organization, etc.
  • Unique identifier: http://dbpedia.org/resource/Apple_Inc.

In the News API, we extract entities found in the title and the body of each individual story, and you can filter your search results in most endpoints based on these values.

Searching by Entities

Most endpoints support the following attributes for filtering results by entities:

  • : List of surface forms for entities found in the story title
  • : List of types for entities found in the story title
  • : List of identifiers (DBpedia URI) for entities found in the story title
  • : List of surface forms for entities found in the story body
  • : List of types for entities found in the story body
  • : List of identifiers (DBpedia URI) for entities found in the story body

These parameters allow you to first decide where in the article (title or body) the desired entity(ies) should appear. Secondly, you can search based on entity names (e.g. "Apple" or "Kim Kardashian"), entity types (e.g. Person or College) and/or entity links (e.g. http://dbpedia.org/resource/Apple_Inc.).

DBpedia Links and Types (and when to use them)

DBpedia is a project aiming to extract structured content from the information created as part of the Wikipedia project. It could be thought of as the structured counterpart of Wikipedia, where distinct entities, their attributes and their associations to other entities are presented in a graph structure.

Using DBpedia links or identifiers to refer to entities has several advantages. Most importantly, it enables the presenting party and the receiving party to ensure that they are referring to the same universal entity and eliminate ambiguity as they communicate with each other.

For instance if two well-known people share the same name (e.g. Michael Jackson), we could differentiate them by using their DBpedia identifiers: for the entertainer and for the basketball player.

While using the News API, if you wish to focus on a well-known entity, by using its DBpedia link you can clarify your intent unambiguously, to reduce the number of false positives.

Additionally, knowing the DBpedia link of an entity enables us to query DBpedia (or Freebase) to retrieve additional information about that entity. For instance, to retrieve a list of all Apple products, we can run the following query through DBpedia's SPARQL interface:

select ?products where {
  <http://dbpedia.org/resource/Apple_Inc.> <http://dbpedia.org/ontology/product> ?products 
}
(try it here)

Knowing the DBpedia links of Apple's products, we can query the News API to retrieve stories that mention any of Apple's products.

Finding DBpedia URIs for Entities

Using the Autocompletes endpoint, you can retrieve the DBpedia link for an entity by providing its name (or a part of it). All you need to do is to set and provide all or a part of the entity's name using the parameter.

Similarly, you can search over all entity types by setting while using the Autocompletes endpoint.

For more information, please refer to the examples provided in the Autocompletes documentation.

Note: Not all entities have a DBpedia identifier. Typically, only well-known entities have one. If your Autocompletes search doesn't yield any results, consider providing the name of the entity as text ( or ) instead.

DBpedia Types

The DBpedia project has its own extensive ontology for entity types, which goes beyond the common types such as person, organization or place. For a complete list of these, refer to the Ontology Classes document on the DBpedia project website.

To filter based on DBpedia types, you can simply use the and/or parameters.

Note: For simplicity, we omit the "http://dbpedia.org/ontology/" prefix from DBpedia type strings. For instance, to retrieve all stories where any Member of Parliament (DBpedia type http://dbpedia.org/ontology/MemberOfParliament) is mentioned in the title, you can set .

Examples

The following example shows articles that are in English, mention Harvey Norman or Apple Inc. in the body, and were published between 1 day ago and now:

var AylienNewsApi = require('aylien-news-api');

var apiInstance = new AylienNewsApi.DefaultApi();

// Configure API key authorization: app_id
var app_id = apiInstance.apiClient.authentications['app_id'];
app_id.apiKey = "YOUR_APP_ID";

// Configure API key authorization: app_key
var app_key = apiInstance.apiClient.authentications['app_key'];
app_key.apiKey = "YOUR_APP_KEY";

var opts = {
  'language': ['en'],
  'entitiesBodyLinksDbpedia': [
    'http://dbpedia.org/resource/Harvey_Norman',
    'http://dbpedia.org/resource/Apple_Inc.'
  ],
  'publishedAtStart': 'NOW-1DAY',
  'publishedAtEnd': 'NOW'
};

var callback = function(error, data, response) {
  if (error) {
    console.error(error);
  } else {
    console.log('API called successfully. Returned data: ' + JSON.stringify(data));
  }
};
apiInstance.listStories(opts, callback);
import aylien_news_api
from aylien_news_api.rest import ApiException

# Configure API key authorization: app_id
aylien_news_api.configuration.api_key['X-AYLIEN-NewsAPI-Application-ID'] = 'YOUR_APP_ID'
# Configure API key authorization: app_key
aylien_news_api.configuration.api_key['X-AYLIEN-NewsAPI-Application-Key'] = 'YOUR_APP_KEY'

# create an instance of the API class
api_instance = aylien_news_api.DefaultApi()

language = ['en']
since = 'NOW-1DAY'
until = 'NOW'
entities_body_links_dbpedia = [
  'http://dbpedia.org/resource/Harvey_Norman',
  'http://dbpedia.org/resource/Apple_Inc.'
]

try:
    # List stories
    api_response = api_instance.list_stories(
      language=language,
      published_at_start=since,
      published_at_end=until,
      entities_body_links_dbpedia=entities_body_links_dbpedia
    )
    print(api_response)
except ApiException as e:
    print("Exception when calling DefaultApi->list_stories: %s\n" % e)
<?php
require_once(__DIR__ . '/vendor/autoload.php');

// Configure API key authorization: app_id
Aylien\NewsApi\Configuration::getDefaultConfiguration()->setApiKey('X-AYLIEN-NewsAPI-Application-ID', 'YOUR_APP_ID');

// Configure API key authorization: app_key
Aylien\NewsApi\Configuration::getDefaultConfiguration()->setApiKey('X-AYLIEN-NewsAPI-Application-Key', 'YOUR_APP_KEY');

$api_instance = new Aylien\NewsApi\Api\DefaultApi();

$opts = array(
  'published_at_start' => 'NOW-1DAY',
  'published_at_end' => 'NOW',
  'language' => ['en'],
  'entities_body_links_dbpedia' => [
    'http://dbpedia.org/resource/Harvey_Norman',
    'http://dbpedia.org/resource/Apple_Inc.'
  ]
);

try {
    $result = $api_instance->listStories($opts);
    print_r($result);
} catch (Exception $e) {
    echo 'Exception when calling DefaultApi->listStories: ', $e->getMessage(), "\n";
}
?>
package com.aylien.newsapisample;

import com.aylien.newsapi.*;
import com.aylien.newsapi.auth.*;
import com.aylien.newsapi.models.*;
import com.aylien.newsapi.parameters.*;
import com.aylien.newsapi.api.DefaultApi;

import java.util.*;

public class Main {

  public static void main(String[] args) {
    ApiClient defaultClient = Configuration.getDefaultApiClient();

    // Configure API key authorization: app_id
    ApiKeyAuth app_id = (ApiKeyAuth) defaultClient.getAuthentication("app_id");
    app_id.setApiKey("YOUR_APP_ID");

    // Configure API key authorization: app_key
    ApiKeyAuth app_key = (ApiKeyAuth) defaultClient.getAuthentication("app_key");
    app_key.setApiKey("YOUR_APP_KEY");

    DefaultApi apiInstance = new DefaultApi();

    StoriesParams.Builder storiesBuilder = StoriesParams.newBuilder();

    List<String> language = Arrays.asList("en");
    String since = "NOW-1DAY";
    String until = "NOW";
    List<String> entitiesBodyLinksDbpedia = Arrays.asList(
      "http://dbpedia.org/resource/Harvey_Norman",
      "http://dbpedia.org/resource/Apple_Inc."
    );

    storiesBuilder.setLanguage(language);
    storiesBuilder.setPublishedAtStart(since);
    storiesBuilder.setPublishedAtEnd(until);
    storiesBuilder.setEntitiesBodyLinksDbpedia(entitiesBodyLinksDbpedia);

    try {
      Stories result = apiInstance.listStories(storiesBuilder.build());
      System.out.println(result);
    } catch (ApiException e) {
      System.err.println("Exception when calling DefaultApi#listStories");
      e.printStackTrace();
    }
  }
}
# Load the gem
require 'aylien_news_api'

# Setup authorization
AylienNewsApi.configure do |config|
  # Configure API key authorization: app_id
  config.api_key['X-AYLIEN-NewsAPI-Application-ID'] = 'YOUR_APP_ID'
  
  # Configure API key authorization: app_key
  config.api_key['X-AYLIEN-NewsAPI-Application-Key'] = 'YOUR_APP_KEY'
end

api_instance = AylienNewsApi::DefaultApi.new

opts = {
  :published_at_start => "NOW-1DAY",
  :published_at_end => "NOW",
  :language => ["en"],
  :entities_body_links_dbpedia => [
    'http://dbpedia.org/resource/Harvey_Norman',
    'http://dbpedia.org/resource/Apple_Inc.'
  ]
}


begin
  #List stories
  result = api_instance.list_stories(opts)
  puts result
rescue AylienNewsApi::ApiError => e
  puts "Exception when calling DefaultApi->list_stories: #{e}"
end
package main

// Import the library
import (
    "fmt"
    newsapi "github.com/AYLIEN/aylien_newsapi_go"
)

func main() {
    api := newsapi.NewDefaultApi()

    // Configure API key authorization: app_id
    api.Configuration.APIKeyPrefix["X-AYLIEN-NewsAPI-Application-ID"] = "YOUR_APP_ID"

    // Configure API key authorization: app_key
    api.Configuration.APIKeyPrefix["X-AYLIEN-NewsAPI-Application-Key"] = "YOUR_APP_KEY"

    until := "NOW"
    since := "NOW-1DAY"
    language := []string{"en"}
    entitiesBodyLinksDbpedia := []string{
        "http://dbpedia.org/resource/Harvey_Norman",
        "http://dbpedia.org/resource/Apple_Inc."}

    storiesParams := &newsapi.StoriesParams{
        Language:                 language,
        PublishedAtStart:         since,
        PublishedAtEnd:           until,
        EntitiesBodyLinksDbpedia: entitiesBodyLinksDbpedia}

    storiesResponse, res, err := api.ListStories(storiesParams)
    if err != nil {
        panic(err)
    }
    _ = res

    fmt.Println(storiesResponse)
}
using System;
using Aylien.NewsApi.Api;
using Aylien.NewsApi.Client;
using Aylien.NewsApi.Model;
using System.Collections.Generic;

namespace Example
{
    public class WorkingWithEntitiesExample
    {
        static void Main(string[] args)
        {

            // Configure API key authorization: app_id
            Configuration.Default.ApiKey.Add("X-AYLIEN-NewsAPI-Application-ID", "YOUR_APP_ID");

            // Configure API key authorization: app_key
            Configuration.Default.ApiKey.Add("X-AYLIEN-NewsAPI-Application-Key", "YOUR_APP_KEY");

            var apiInstance = new DefaultApi();
            
            var since = "NOW-1DAY";
            var until = "NOW";
            var language = new List<String> { "en" };
            var entitiesBodyLinksDbpedia = new List<string> {
                "http://dbpedia.org/resource/Harvey_Norman",
                "http://dbpedia.org/resource/Apple_Inc."
            };

            try
            {
                // List stories
                Stories result = apiInstance.ListStories(
                    publishedAtStart: since,
                    publishedAtEnd: until,
                    language: language,
                    entitiesBodyLinksDbpedia: entitiesBodyLinksDbpedia
                );
                Console.WriteLine(result);
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception when calling DefaultApi.ListStories: " + e.Message );
            }
        }
    }
}