Newscatcher supports authentication with the API key. Once you received a key, you are allowed to make API requests.
There are 2 available endpoints /search and /all:
- Text search
/search- the main method for searching through news articles based on parameters like text, date and/or source. This is a perfect tool when you want to look for articles on a specific subject on a single source or on worldwide news sources. - Lastest articles
/all- an auxiliary method that allows you to choose a time-lapse and get the latest news on specific or multiple sources.
The request consists of:
- URL of the API
- endpoint followed by a question mark
?to pass parameters - parameters separated by
& - API Key to be put in Headers
You can specify your search using several parameters. All results are sorted by the score of accuracy to the requested word or phrase.
phrase=[string]- word or phrase that will be searched inside articles' content. By default, it is searched through bothtitleandsummaryfields
-
source=[string, string]- one or more news resources can be specified for searching. Better enter URL of official websites of news sources. It should be the normal form of the URL, for example, nytimes.com, theguardian.com (https://www.nytimes.com/,https://www.theguardian.com/international) -
fromandto- to search for a specific time period. In order to avoid errors, use on of the presented forms:YYYY-MM-DD,DD/MM/YYYY,YYYY-MM-DD HH:MI:SS,DD/MM/YYYY HH:MI:SS -
title=False- setTrueto search only inside articles' title -
summary=False- setTrueto search only inside articles' bodies -
maxsize=20- number of received results in one call. Restricted to 100 -
bydate=False- setTrueto sort articles by date -
isranked=True- by default, only sources which are in Majestic top million are returned. Set toFalseif you want to see all the results
You can use this method for exploring the latest articles of one or more news sources within a specific time-lapse. Articles are sorted by the rank of a news source.
All parameters are optional, however, you can use either hours or from and to, not both at the same time.
-
source=[string, string]- one or more news resources can be specified for searching. Better enter URL of official websites of news sources -
fromandto- search for a specific time period. In order to avoid errors, use the next formYYYY-MM-SS HH:MI:SSorDD/MM/YYYY -
hours=[integer]- articles published for the lasthourswill be displayed -
maxsize=20- number of received results in one call. Restricted to 100 -
bydate=False- setTrueto sort articles by date -
isranked=True- by default, only sources which are in Majestic top million are returned. Set toFalseif you want to see all the results
- _status - the status of the response
- _total - how many articles are found. 10,000 is a limit
- articles - list of found articles
articles:
-
_id - The unique id of an article
-
_score - Relevance score for
-
_source - The list of main relevant information of the article
- author - The main author of the article
- authors - The list of authors
- name - The name of author
- clean_url - The official URL of the news source
- comments - The URL to the comments part
- domain - The official domain of the news source
- feed_update - The last time when the news feed was updated
- language - The language of the article
- link - The direct URL of the article
- maj_ref_ips - Referring IPs to the main domain according to Majestic
- maj_ref_subnets - Referring subnets to the main domain according to Majestic
- parse_date - The date when the article was added to the database
- published - The date when the article was published, in UTC
- rank - The rank of the news source according to Majestic
- rights - The copyright
- summary - The description of the article
- tags - The list of tags or keywords concerning the article
- label - Not relevant
- scheme - Not relevant
- term - The keyword of the article
- title - The title of the article
- 50 latest articles on COVID-19
<API_URL>/search?phrase=COVID-19&maxsize=50 - 50 latest articles on COVID-19 from NYTimes, WSJ and The Guardian
<API_URL>/search?phrase=COVID-19&source=nytimes.com,wsj.com,theguardian.com&maxsize=50 - 50 latest articles on COVID-19 from NYTimes, WSJ and The Guardian for the specific date
<API_URL>/search?phrase=COVID-19&source=nytimes.com,wsj.com,theguardian.com&from=2020-03-25&to=2020-03-26&maxsize=50
- 25 latest articles from NYTimes and Foxnews for the last 24 hours sorted by date
<API_URL>/all?hours=24&source=nytimes.com,foxnews.com&bydate=True&maxsize=25 - 25 latest articles from NYTimes and Foxnews for the specific date
<API_URL>/all?source=nytimes.com,foxnews.com,theguardian.com&bydate=True&maxsize=25&from=2020-03-25&to=2020-03-26
import requests
def print_results(results, variable):
for i in results['articles']:
try:
print(i['_source'][str(variable)])
except:
print('Variable is not available for id - ' + str(i['_id']))
def save_variable(results, variable):
output = []
for i in results['articles']:
try:
output.append(i['_source'][str(variable)])
except:
if str(variable) in ['tags', 'authors']:
output.append(['Nothing found'])
else:
output.append('Nothing found')
return output
url = <API_URL> + 'search?' \
'phrase= Paris Coronavirus&' \
'title=True'
API_KEY = <YOUR_KEY>
response = requests.get(url, headers={'x-api-key': API_KEY})
result = response.json()
print('Number of found matches - ' + str(result['_total']))
print_results(result, 'title')
content = save_variable(result, 'summary')
tags = save_variable(result, 'tags')
curl -H "X-API-KEY: <PUT_YOUR_API_KEY> <API_URL>/search?phrase=Trump&title=True
There is an option to display results using a simple UI interface. Parameters are identical, only names of endpoints are different. Use an extension to your browser to be able to put an API KEY inside the headers. https://addons.mozilla.org/en-US/firefox/addon/modify-header-value/