Text and Data Mining (TDM) from HKUST Licensed Material: Home

Elsevier allows a certain amount of TDM to subscribed content.

• Supplies over 40 APIs for Elsevier’s products including Scopus, ScienceDirect, SciVal, PlumX, and others.
• Users need to obtain an API key via Elsevier’s Developer Portal.
• HKUST researchers only have access to the content HKUST subscribed to + Open Access.
• An Object Retrieval API is available for mining images.

• There are limitations on how much and what speed you can harvest for TDM:

  "...there are no hard limits on the number of items that may be downloaded via our API. Nevertheless, a reasonable and customary rate limit remains in place to ensure equal access to the API for all users, and we continue to ask users to use our service responsibly.

  We understand the need to be flexible and continue to monitor usage and consult with researchers. However, we do reserve the right to deactivate any API key if we believe usage is abusive or impacting the stability of our systems." -  Text and data mining FAQs

More info available here: https://www.elsevier.com/open-science/research-data/text-and-data-mining

Text & Data Mining with Factiva requires a separate license.

The Library can provide contact person for researchers to ask for quote.

Gale Digital Scholar Lab

• Is a platform for text analysis, data mining, and data visualization.
• Users can create and analyze content sets from Using our licensed content in “Gale Primary Sources” collections,
• Content includes:
  • Archives Unbound collections
    • China : records of the U.S. Department of State, 1930-1949
    • Chinese Maritime Customs Service : the Customs' gazette, 1869-1913.
    • Chinese Maritime Customs Service publications, 1859-1949
    • Papers of old Shanghai: business, banking, and insurance. [189-1942]
    • Policing the Shanghai International Settlement, 1894-1945
    • The Chinese Civil War and U.S.-China relations : records of the U.S. State Department's Office of Chinese Affairs, 1945-1955.
  • Asia and the west: diplomacy and cultural exchange
  • China and the modern world in 7 parts
    •  Diplomacy and political secrets (1869–1950)
    • Hong Kong, Britain and China : the Rise and Fall of Empires (1841-1951)
    •  Hong Kong, Britain and China, part II, 1965-1993.
    •   Imperial China and the West part I (1815-1881)
    • The Economist historical archive, 1843-2020 -Gale primary sources.
    • Financial times historical archive, 1888-2021 - Gale primary sources
    • The Independent historical archive, 1986-2016 - Gale primary sources
    • The Sunday Times historical archive, 1822-2006 - Gale primary sources
    • The Times Digital Archive, 1785-2019 - Gale primary source

  Gale (Cengage): Data Mining FAQs

  • HKUST has signed an Addendum to the licensing contract.
  • For researchers' personal use only.
  • Delivers data in a hard drive and the library is responsible to arrange users to access the content in the drive.
  • Charges a fee that based on the cost of production and delivery.

Constellate

Provides Text and data mining tools and teaching with JSTOR, Portico, and other IThaka collections.

JSTOR Dataset Services
Anyone can request a dataset through either of the two services below.

• Self-service: limit to 25,000 documents; does not cover full text.
• Large/full-text request: by special request and requires an agreement about the use of the data.

The Nexis Uni subscription from HKUST is good for students to use it for research, but not crawling or downloading large volume of data. 

Lexis-Nexis has a section on their website where you can ask about using or purchasing their "Data as a Service" for larger datasets.

They also have a  LexisNexis Bulk Content API mining personal consultation service.

Researchers can pay extra to text and data mine ProQuest content that HKUST Library already owns or subscribes to via the ProQuest TDM Studio

"Downloading articles from SAGE Journals for the purposes of text and data mining is expressly permitted in our standard licence agreements and our terms of use for no extra fee. You do not need to ask permission to systematically download articles provided that:

• You only use the articles for non-commercial text and data mining.
• You only download articles to which you have legitimate access, for example if they are open access or part of your institution's subscription. If you cannot view an article on SAGE Journals, you will not be able to download it.
• You respect the following limits when downloading SJ content:
  • 1 request every 6 seconds – Monday to Friday between Midnight and Noon in the "America/Los_Angeles" timezone;
  • 1 request every 2 seconds - Monday to Friday between Noon and Midnight in the "America/Los_Angeles" timezone, and all day Saturday and Sunday."

 - https://journals.sagepub.com/page/policies/text-and-data-mining

Text and Data Mining at Springer Nature

• Offers various APIs to facilitate TDM, e.g. Citations API, SN SciGraph APIs, and more.
• Provides a selection of metadata format such as JATS, Dublin Core, ONIX, or MARC records.
• Supports argumentation mining.

HKUST Library's license with Clarivate (owner of Web of Science) allows creating & using custom data sets:

• For internal, non-commercial purposes only
  1. Use the custom dataset for numerical or statistical analyses of data elements derived from the service
  2. Download the custom dataset for use in your own data analytics and proprietary tools
  3. Index the custom dataset for searching by authorized users and display results of such searches performed
  4. Create derivative databases consisting of the results of (1) to (3)

Limitations: You may not distribute, sublicense or publicize any portion of the custom dataset or derivative databases.

See Clarivate's Product/Service terms (p. 24)

Wiley: Text and Data Mining

• Prefers using Crossref Text and Data Mining Service for TDM.
• Users have to accept the click-through license in order to get an API token, but should check with their library to see if a separate agreement is already in place beforehand.

Text and Data Mining Resources -  by  Reese Manceaux of the Atkins Library at UNCC

Text Mining Resources - Princeton University Library