Skip to Main Content

EPIB 629 Knowledge Synthesis

Deduplicating in EndNote

Removing duplicates

Systematic review software may offer this functionality, or you can use one of the following methods to remove duplicates from a merged EndNote library.

Before deduplicating, you will need a merged EndNote library containing the records from all your separate EndNote libraries for the individual database searches, if you had previously exported records from each database into separate libraries:

  • Create a new EndNote library that will contain the records from all the databases you searched (I like to put DEDUPING in the EndNote library name)
  • Import the records from each EndNote library you created for the individual database searches:
    • Go to EndNote menu > File > Import > File
    • Next to "Import File", browse to choose the .enl file (NOT the .enlx file) for each library of downloaded records from your searches and select "EndNote Library" as the "Import Option"
    • Once all the records have been added to this new library, check to make sure the final number of records, before removing duplicates, matches the sum of the records found for all the database searches

Using this merged library of records from your individual database searches, you are now ready to remove duplicates. Here are two methods you can use:

1) Earlier version of "Bramer method" for deduplicating, with steps provided in Word document format:

2) Paper describing more advanced configuration options for removing duplicates in EndNote: Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240-243. doi:10.3163/1536-5050.104.3.014

After deduplication

Create a compressed library for backup after having removed as many duplicates as possible, with a filename like SearchTerms-yyyymmdd-Deduplicated—xRecords.enlx. This will be the library for screening. You can export the deduplicated library to Rayyan, for example, or other synthesis software, to facilitate the screening process, which usually involves more than one reviewer.

Deduplicating with SRA DeDuplicator

Removing duplicate records with the IEBH SR-Accelerator Deduplicator 

Use the online Deduplicator tool. This is a new version as of August 2021.

Large sets of records

The offline DeDuplicator (old version) may be useful for large sets of records (which used to be considered ≥ 2000). Download the SRA-dedupe-UI application from GitHub. As of October 2020, there were only Linux and Windows versions available.

  • Help using DeDuplicator Offline
  • When exporting your records to an XML file, don't forget to select them all (Help Importing/Exporting EndNote records)
  • If you choose to use the stand alone executable version for Windows, you may get a message that "Windows protected your PC": Click on More info and then a Run anyway button will appear, which you should click if you feel comfortable trusting the software developers

Keep a copy of the RIS or XML file for your records.

See also: Rathbone, J., Carter, M., Hoffmann, T., & Glasziou, P. (2015). Better Duplicate Detection for Systematic Reviewers: Evaluation of Systematic Review Assistant-Deduplication Module. Systematic Reviews, 4(1), 6. doi:10.1186/2046-4053-4-6


Liaison Librarian

Profile Photo
Genevieve Gore
Liaison Librarian, Schulich Library of Physical Sciences, Life Sciences, and Engineering
Contact: Website

Need help? Ask us!

McGill LibraryQuestions? Ask us!
Privacy notice