I imported my reference database from Zotero to Jabref and cleaned it up. In the last few days I found that some references are missing from Jabref (several hundred records of 3000+). This is more than the number of duplicates I removed in Jabref.
The best idea I have is to import the references from the old Zotero set to a new JabRef database, generate a fresh set of BibTex keys based on my existing template, and then merge the two databases. The difference between the original and merged JabRef databases should only be the new imported records plus the tens of records that I edited manually (and likely have different BibTex keys).
For the actual difference between import file and open file, one of the more experienced maintainers would need to chip in here.
Why i think it is relevant to be precise here is because in Jabref versoin 5.3 importing entries by using XMP metadata that was attached to pdf files was not working well, so what you texted here reminded me of that problem. Back then, it was something very specific to importing via XMP only. The other methods of importing bibliographic data were not affected.
What i meant to say: Can you confirm that when you export your Zotero library and then open that library with Jabref that there are 3816 entries within Jabref? (Without having cleaned duplicates and deleting stuff by you). If yes, that would mean it has nothing to do with import, but there is something going on with what you did AFTER you imported it. Just trying to narrow down the possible causes by excluding that it has somethign to do with opening the file or importing the entries
I may not actually need to do a full merge. Just a quick comparison of entries (would love to export author, year, title in flat files), import to a conventional database and then run a join.
I had found that entries from one author present in previous libraries (Zotero and Mendeley) were not found when I searched in JabRef (“No entries present”). This seems to be an intermittent issue that occurs when I have hundreds of megabytes of swap memory under Ubuntu. I’ve also seen it under Windows.
My browser keeps chewing up more and more memory over time. If I don’t keep it open I don’t have this problem in JabRef
I have 8GB physical memory and a similar amount of swap under Ubuntu, so it isn’t a hardware issue
I would suggest to use this thread for the original question: “What is the best way to merge two databases with large overlaps?” It is a good question and maybe somebody else in future may have an even better answer to what you found, so let’s leave it open and discuss the other issues in different threads.