{{ (fullscreenTable) ? 'show networks' : 'fullscreen table' }} / download input articles: CSV, RIS /

( link to share )

{{ authorStringShort(currentGraph.source.authors) }}. {{ currentGraph.source.title }}. {{ currentGraph.source.journal }}. {{ currentGraph.source.year }}. (select)

Data retrieved through {{ currentGraph.API + ' (' + abbreviateAPI(currentGraph.API) + ')' }} API on {{ new Date(currentGraph.timestamp).toLocaleDateString() }} (estimated completeness: {{ completenessPercent }}%)

{{ inputArticlesFiltered.length + " / " + currentGraph.input.length }} {{ props.row.isSource ? 'Source: ' : '' }} {{ formatTags(props.row.title) }} {{ props.row.authors[0]?.LN }}

{{ (fullscreenTable) ? 'show networks' : 'fullscreen table' }} / download references: CSV, RIS /

References are cited by the input articles. They are sorted by in-degree, with the top references listed at the top. They tend to be older than most input articles. They are derived through backward citation searching (backward in time). This table is de-duplicated and does not contain any input articles.

{{ incomingSuggestionsFiltered.length + " / " + currentGraph.incomingSuggestions?.length }} {{ props.row.isInputArticle ? 'Input article: ' : '' }} {{ props.row.authors[0]?.LN }}

{{ (fullscreenTable) ? 'show networks' : 'fullscreen table' }} / download citations: CSV, RIS /

Citations are citing the input articles. Tey are sorted by out-degree, with the top citations listed at the top. They are derived through forward citation searching (forward in time). They tend to be newer than most input articles. This table is de-duplicated and does not contain any input articles / references.

{{ outgoingSuggestionsFiltered.length + " / " + currentGraph.outgoingSuggestions?.length }} {{ props.row.isInputArticle ? 'Input article: ' : '' }} {{ props.row.authors[0]?.LN }}
{{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }} / network settings / {{ (currentGraph.citationNetworkTurned) ? '↺' : '↻'}} how to read this network {{ (currentGraph.citationNetworkTurned) ? '←' : '↓' }} older articles newer articles {{ (currentGraph.citationNetworkTurned) ? '→' : '↑' }}
{{ (fullscreenNetwork) ? 'show tables' : 'fullscreen network' }} / network settings how to read this network (only authors with a minimum of input articles are shown)

Click + to enter source DOI
or scan a plain text file for DOIs instead File!

(Load examples or see Frequently asked questions)

References can be retrieved through all APIs. Citations can be retrieved through OpenAlex (OA), Semantic Scholar (S2) and OpenCitations (OC). Top references are the most cited references by the input articles. Top citations are citing the most input articles. Retrieving all references / citations works best with OpenAlex (OA, truly all references / citations thanks to cursor paging) & Semantic Scholar (S2, currently max. 1000 referenes / citations per paper).

Instructions

open all / collapse all

What is this web app about?

This web app aims to help scientists with their literature review using metadata from OpenAlex (OA), Semantic Scholar (S2), OpenCitations (OC) and Crossref (CR). Academic papers cite one another, thus creating a citation network (= graph). Each node (= vertex) represents an article and each edge (= link / arrow) represents a reference (outgoing) / citation (incoming). Citation networks are a topic of bibliometrics, for which other great software exists as well.

This web app visualizes subsets of the global citation network that I call 'local citation networks', defined by the references of a given set of input articles. In addition, the most cited references missing in the set of input articles are suggested for further review as 'Top references'. Other missing articles citing the highest number of input articles are 'Top citations'.

How to use this web app?

There are basically two ways to create new networks:

  1. Based on the references of a given paper ('source article'). This could be a review or just an interesting paper you'd like to dig into deeper. Either click the + button at the top right and enter the digital object identifier (DOI) of the source article or use the bookmarklet described below.
  2. The second way to create a new network is with a custom list of IDs. Either click the + button at the top right and then 'Enter custom list of IDs instead', or scan a plain text file for DOIs, which allows you to create your own unique lists of input articles. For example, this could be a .ris / .bib file exported by your reference manager (e.g. Zotero or EndNote). In theory, you can scan any plain text file (PDFs and Microsoft Office files don't work). However, due to the multitude of characters allowed in DOIs, the system might not catch all of them correctly (see this blog post by Crossref for more explanation).

Up to 5 tabs can be open at the same time, representing a different local citation network each. Only one query can be processed at a time, which is why some buttons are disabled when a query is still going on. The APIs can return errors when they're overloaded, the reference list is too long or your search frequency too high.

How to use the bookmarklet to open references of a paper?

A bookmarklet is a small JavaScript program that can be saved as a bookmark in your browser. This particular bookmarklet allows you to open a new network in Local Citation Network directly when surfing many journals' websites (e.g. Nature, Science, Lancet, …) with the advantage that the original order of references is preserved. Save the following link as a bookmark (e.g. by dragging it to your bookmarks bar) and then click on it when reading an article: Open in Local Citation Network

The bookmarklet is particularly useful when you want to use Local Citation Network as a 'second screen' when you are reading a paper, showing you its references in the correct order and allowing you to view their abstracts and to identify seminal articles as well as additional suggested articles. When using the bookmarklet, the reference list is scanned directly from the website, thus preserving the original order of references.

PubMed is also supported by the bookmarket, giving access to a vast number of biomedical papers. Before clicking the bookmarklet, scroll down on the paper page (e.g. this one) and click "Show all X references". Make sure to use the OpenAlex (OA) or Semantic Scholar (S2) APIs , as they are supporting PubMed IDs (PMIDs).

Unfortunately, not all journals' websites are supported. If you know how to code, check out the code of the bookmarklet on GitHub and extend it to support more journals! In this light it might be worth updating the bookmark in your browser from time to time to benefit from possible improvements.

Where do the 'Top references' come from?

'Top references' are the most cited references by the input articles that are still missing, thus they have high in-degrees. In order to create the local citation network, this web apps attempts to retrieve reference-lists from all input articles (usually some are missed, see completeness) and then orders these references by the number of local incoming citations (in-degree, see network). Some of these references might be among the input articles, depending on how connected they are. However, some of them usually aren't among the input articles and then become 'Top references'.

Keep in mind that because of the nature of this approach, 'Top references' are generally older than the input articles. This can help identifying seminal papers of the field but not newer state-of-the-art research. In order to identify newer articles, see 'Top citations'.

Where do the 'Top citations' come from?

'Top citations' are citing the most input articles, thus they have high out-degrees. They are generally newer than the input articles. Local Citation Network currently supports this feature with the APIs of OpenAlex (OA), Semantic Scholar (S2) and OpenCitations (OC).

How to read the citation network?

Zoom by scrolling and pan by dragging. Each node (= vertex) represents an article. They are ordered by year from top (newer) to bottom (older), with each year colored differently. Alternatively, nodes can also be colored by the article's journal in 'network settings'. The size of the nodes depends on the sum of their 'in-degree', the number of citations the paper received from the input articles, and their 'out-degree', the number of input articles they cite. The local 'in-degree' is not always proportional to the global citation count. Input articles are circle-shaped (●), except for the source article (if applicable), which is diamond-shaped (◆). Up-pointing triangles (▲) represent 'Top references', which are cited by the input articles. Down-pointing triangles (▼) are 'Top citations', which are citing the input articles.

Each edge (= link / arrow) represents a reference and citation at the same time: They are references (=outgoing citations) for their source nodes and (incoming) citations for their target nodes. Usually papers can only cite older papers, hence edges tend to point downwards (or sidewards for citations of the same year). Technically a citation network is a directed acyclic graph.

How to read the co-authorship network?

This 'local' co-authorship network shows the most common authors among the input articles. Depending on the discipline, the number of (co)authors among a set of multiple articles can quickly rise to the hundreds, which is why this author network only shows authors with a minimum number of input articles (not to be mistaken with their global number of publications, which is usually higher). The default minimum number of input articles is determined so that this network shows no more than 50 authors, but it can be changed with the dropdown at the bottom.

Each node (= vertex) represents an author. The size depends on their number of input & suggested articles and the color depends on the first or last article they first appear in. Authors of input articles are circle-shaped (●), except for authors of the source article (if applicable), who are diamond-shaped (◆). Click authors to filter articles-tables, ctrl+click to filter for more than one author.

Each edge (= link) represents a collaboration between two authors among the source, input and suggested articles, the width indicating the number of collaborations. Click edges to filter input & suggested articles.

Which APIs are supported and how to switch?

Switch the API with the button indicating "OA / S2 / OC / CR" in the top-right menu. All APIs provide references.

OpenAlex (OA) is an open source successor of Microsoft Academic, which was discontinued in 01/2022. It is fast, it has high coverage, serves most abstracts and provides citations. It also serves a flag for retracted papers, which can be seen here for example (it can be missing though).

Semantic Scholar (S2) has high coverage, serves most abstracts and provides citations. In addition, it allows accurate estimation of the reference-list completeness. However, it can be slower and has a rate limit.

The metadata API from OpenCitations (OC) builds upon Crossref but also features citations. However, it does not provide abstracts, author information (like affiliation or ORCID) and it is quite slow.

Crossref (CR) usually returns reference lists in their original order (you can use the bookmarklet for other APIs). Also, it allows for an accurate estimation of the reference-list completeness, it's usually very fast and can handle fairly large input lists. Crossref is a not-for-profit organization that collects metadata from scholarly journals and publications. However, it does not provide citations.

Thanks to the formidable Initiative for Open Citations (I4OC), the coverage of all of these services has increased vastly. However, some journals still do not support these ideals - feel free to write them, in case you find one!

What does 'Autosave results locally' do?

When activated, this option caches your recent networks (up to 5 tabs) and your preferred API locally in your browser in the so-called localStorage. When you revisit this web app, you can pick up where you left. When deactivated, this data is deleted. See privacy.

Troubleshooting

This web app is not working!? Which browsers are supported?

I've tested with the current version of Firefox (as of November 2022), but this web app should also run in current versions of Safari, Edge and Chrome. Microsoft Internet Explorer is not supported. This web app is meant for desktop use and not optimized for smartphones, which have only been tested sparsely. I'm a single developer and created this web app in my spare time for free and for fun, so I don't have the time and resources to test all possible devices, web browsers and scenarios. This web app is open source, feel free to support it by extending compatibility or optimizing mobile / tablet use! If you find a bug or problem with the newest versions of Firefox / Safari / Edge / Chrome, please report it on GitHub or contact me.

The queries have turned very slow!? / How to cancel loading?

APIs are public with varying response speeds depending on their overall usage and workload. In addition, they might be throttling individual users (as identified by IP address) when they perform too many queries in a row. Checking your internet connection, only querying what's really important, switching to another API and trying again later might help!

Sometimes (particularly with unstable internet connections) queries seem to run forever. In case you want to cancel loading, activate 'Autosave results locally' (see autosave) and refresh the page. You can deactivate autosave afterwards again.

Is the data complete? / Some references are missing!?

The data is usually incomplete. Some DOIs cannot be found at all in some APIs and for those that are found, often some metadata like authors, abstracts or reference-lists are missing or sometimes incorrect. For the local citation network and 'Top references', the completeness of the reference-lists is most important.

The estimated completeness of the reference-lists can be seen above the search bar in the "Input articles" tab and is calculated in the following way: For OpenAlex and OpenCitations it is the fraction of input articles that have reference-lists themselves (multiplied by the fraction of specified DOIs found, in case of a custom input list).

Semantic Scholar and Crossref allow for more subtle calculation, as they often also provide the total reference count, which is often larger than the reference-list of DOIs (older references often don't have DOIs and neither do some specific books, papers or conference abstracts). The estimated completeness is thus calculated as the product of three fractions:

  1. Source reference completeness: (Number of input articles found in API) / (Total reference count of source or total number of lines in custom input list)
  2. (Number of input articles (excluding source) that have reference-lists themselves) / (Number of input articles excluding source)
  3. Average input articles' reference completeness among those input articles that do have reference-lists: (Length of reference-list) / (Total reference count)

How can I preserve the original order of references in the input article list?

You can use the bookmarklet to open reference lists in their original order. Alternatively, you can use Crossref, which does seem to save the original order of references. See here how to switch the API used.

Background

What is bibliometrics? What other software can you recommend?

Bibliometrics is a part of information science that uses statistical methods to analyze publications. It emcompasses scientometrics, which is specifically looking at scientific publications, and citation analysis.

Nowadays, there are many tools similar in spirit to this one: Citation Gecko, Connected Papers, Litmaps, Inciteful and ResearchRabbit, among others. Aaron Tay has written a blog post in 06/2021 for comparison.

Other great software to help with bibliometric / scientometric analyses include Anne-Wil Harzing's Publish or Perish, which allows sophisticated queries of a plethora of APIs including Crossref but also Google Scholar, Scopus and Web of Science. If you're looking to create more complex networks than this web app allows, check out the Leiden University's great VOSviewer and CitNetExplorer, which allow you to create and customize everything from citation networks to author networks, university networks and journal networks. VOSviewer also allows text mining and co-occurrence networks.

Why aren't you using Google Scholar?

Unfortunately Google Scholar doesn't share their data and discourages webscraping. It doesn't have an API.

I have an idea! Can you implement XYZ?

It's great to see you're getting involved! While I see numerous ways to extend this project myself, I don't have much time to work on it anymore unfortunately. However, this web app is open source and I'd be very happy to see others extend this project! In the meantime, check out these great applications, which might already provide what you're looking for.

I'd like to contribute! Is this open source and are you looking for help? / Is this on GitHub?

Yes, great! This project is open source (GPL-3) and the source can be found on GitHub. I don't have much more time to work on this project, so I'm actively looking for help and contributors. Bugfixes are always welcome but please contact me before any large pull-requests so we can coordinate.

Does this web app use cookies? What's the privacy policy?

No, this web app doesn't use any cookies. No user-tracking or fingerprinting is performed and no usage-data is collected by the author of this web app. Optionally, so-called localStorage can be used to cache results and settings locally when "Autosave results locally" is manually turned on (see autosave).

This web app is running locally on your computer and only contacts the API servers during runtime. Crossref has a general privacy statement. This web app is hosted by GitHub pages, see their privacy statement here.

Technicalities: Which JavaScript libraries are used and why no CDN?

This web app uses Vue.js, Buefy and vis.js Network. I opted against using content delivery networks (CDN) to speed up delivery of these JavaScript libraries and thus initial loading-time, because of the potential user-tracking by CDNs. If you're interested, this rising problem is partly countered by the great browser-plugin Decentraleyes (see their FAQ for more details).

Who are you, why did you create this and how can I contact you?

I'm Tim Wölfle from Germany and started creating this project 2019 during the literature review for my Master's thesis. I've often manually scanned reference lists of papers to identify seminal articles and thought this must be automized. You can contact me on Twitter or by e-mail.