[Solved] EOF in mid-string

Help
1

Hello,

First of all, thanks for this tools, containing more and more features.
I manage a “large” bibtext file (about 1080 entries (with non documents attached, research purpose).
Each time I update something on a “good file” (with no errors) and saving it, I get an error when I open it :slight_smile:1. "Une erreur est survenue pendant le traitement de l’entrée: ‘Error in line 22336: EOF in mid-string’. Entrée omise"
I get a partial file (in my example, 790 entries of 1086). The line indicate is positioned at the end of file.

For example, if I merge 2 entries and I save the file under e new name, the new file “is corrupted”.
I can send you files for example.
I’ve removed all the “special characters” (and latex formulation too), even if the file is UTF-8 coded.

Jabref release 3.5 and 3.6 dev snapshot.
Jar file.

I found a partiel cause of my problem. Some times, I don't know why, some entries are partially duplicated. An example :slight_smile: @Article{Martin2015a, Abstract = {Enhanced recovery pathways (ERPs) are thought to improve surgical outcomes by standardizing perioperative patient care established in evidence-based literature. The objective of this study was to determine the impact of a colorectal surgery ERP on hospital length of stay (LOS) and other patient outcomes. This is a comparative effectiveness study of patients undergoing elective colorectal surgery 2 years prior (pre-ERP group) and 2 years after (ERP group) implementation of an ERP program. The primary outcome was hospital LOS. Secondary outcomes included postoperative complications, 30-day readmissions, and 30-day reoperations. Multivariable regression analyses were utilized to control for patient factors, general health factors, diagnosis, surgeon, colon versus rectal operations, and open versus minimally invasive operations-laparoscopic and robotic. An ERP checklist was developed to track adherence to components of the pathway. RESULTS: The study population included 1036 patients: 523 in the pre-ERP group and 513 in the ERP group. Unadjusted LOS was significantly shorter in the ERP group than the control pre-ERP group [3 (IQR 3.5) vs 5 days (IQR 4.6); p<0.0001]. Multivariable regression analysis confirmed the reduction in LOS, controlling for age, colon/rectum procedure, open/laparoscopic/robotic approach, primary diagnosis, and alvimopan use. Postoperative outcomes were not significantly different between groups except for 30-day readmissions, which were unexpectedly higher in the ERP group (14.6 vs 8.7%, p=0.04). A newly implemented ERP on a dedicated colorectal surgery service in an academic non-university hospital setting resulted in shorter hospital LOS, but increased readmissions, for patients undergoing elective open and minimally invasive colon and rectal surgery. Future multi-institutional studies are needed to understand the impact of ERP on postoperative complications and readmissions.}, Doi = {10.1007/s00464-015-4714-8}, Institution = {{Division of Colorectal Surgery, Department of Surgery, Saint Joseph Mercy Health System, 5325 Elliott Dr MHVI #104, Ann Arbor, MI, 48106, USA. Robert.Cleary}, }

@Article{Martin2015b,
Abstract = {Enhanced recovery pathways (ERPs) are thought to improve surgical outcomes by standardizing perioperative patient care established in evidence-based literature. The objective of this study was to determine the impact of a colorectal surgery ERP on hospital length of stay (LOS) and other patient outcomes. This is a comparative effectiveness study of patients undergoing elective colorectal surgery 2 years prior (pre-ERP group) and 2 years after (ERP group) implementation of an ERP program. The primary outcome was hospital LOS. Secondary outcomes included postoperative complications, 30-day readmissions, and 30-day reoperations. Multivariable regression analyses were utilized to control for patient factors, general health factors, diagnosis, surgeon, colon versus rectal operations, and open versus minimally invasive operations-laparoscopic and robotic. An ERP checklist was developed to track adherence to components of the pathway. RESULTS: The study population included 1036 patients: 523 in the pre-ERP group and 513 in the ERP group. Unadjusted LOS was significantly shorter in the ERP group than the control pre-ERP group [3 (IQR 3.5) vs 5 days (IQR 4.6); p<0.0001]. Multivariable regression analysis confirmed the reduction in LOS, controlling for age, colon/rectum procedure, open/laparoscopic/robotic approach, primary diagnosis, and alvimopan use. Postoperative outcomes were not significantly different between groups except for 30-day readmissions, which were unexpectedly higher in the ERP group (14.6 vs 8.7%, p=0.04). A newly implemented ERP on a dedicated colorectal surgery service in an academic non-university hospital setting resulted in shorter hospital LOS, but increased readmissions, for patients undergoing elective open and minimally invasive colon and rectal surgery. Future multi-institutional studies are needed to understand the impact of ERP on postoperative complications and readmissions.},
Doi = {10.1007/s00464-015-4714-8},
Institution = {Division of Colorectal Surgery, Department of Surgery, Saint Joseph Mercy Health System, 5325 Elliott Dr MHVI #104, Ann Arbor, MI, 48106, USA. Robert.Cleary@stjoeshealth.org.},
Issn = {0930-2794},
Keywords = {Enhanced recovery after surgery, Enhanced recovery pathway, Fast-track surgery, Minimally invasive colorectal surgery, Quality improvement},
Language = {eng},
Medline-pst = {aheadofprint},
Month = {Dec},
Pii = {10.1007/s00464-015-4714-8},
Pmid = {26694181},
Type = {Journal Article},
author = {Martin, Thomas D. and Lorenz, Talya and Ferraro, Jane and Chagin, Kevin and Lampman, Richard M. and Emery, Karen L. and Zurkan, Joan E. and Boyd, Jami L. and Montgomery, Karin and Lang, Rachel E. and Vandewarker, James F. and Cleary, Robert K.},
title = {Newly implemented enhanced recovery pathway positively impacts hospital length of stay},
journal = {Surg Endosc},
year = {2015},
}

The first entry cause a problem : Error in line 17681 or above: Empty text token.This could be caused by a missing comma between two fields.’

I found too a comment in the middle of the file :
@Comment{jabref-meta: databaseType:bibtex;}

Regards

MVT

2

The problem with the first entry is that the institution field contains an opening bracket { to much. If you remove it then everything should be fine.

I think we should improve two things here:

  • Better error message which shows where the problem occurred.
  • Display a warning already on writing the bib-file and/or encrypt unbalanced brackets.
3

Hi Tobias,

By the mean time I found some errors like this one.
I saw too that when you merge 2 entries, you’ll find in the file saved the too entries :
The entry merged
and some “résidus” (garbage) of the old one.

It seems the garbage appears when the field “Institution” is filled.
Finally, I recover the whole file.

To do this, I’ve unchecked the ordered column (3 clics, one for sorting the entries from a-z, one for sorting from z to a, the third one to have the entries in the same order than the file. I found, in the original file, the errors.

I don’t know is this is a bug or a misusage by me.
You will find the same problem when, from an entry, you do an Internet search (medline for me with the PMID ou the DOI). In the window presenting the search result, some times, it indicates you a possible duplicate entry. When you click on the small icon, it shows you the 2 entries (left and rignt panel with the difference shown). If you choose “ne conserver que les entrées fusionnées” (save only the 2 merge entries ? - the second button from the bottom right), the merge is not done… and the “file is corrupted”.

I’ll try to find a python3 script to check the brackets (or delimiters eg. 5° and []. I found one on stackoverflow but it is not complete.
Sorry for my bad English, I’m French. Nobody is perfect :wink:

4

Hi MVT!

Thanks for your report. I just filed in a bug report regarding this issue. You can find it here at GitHub: https://github.com/JabRef/jabref/issues/1716

The problem occurs if a bibtex field contains the character “@” - as your “institution” field in the example. The remainder of the entry is skipped after the “@” resulting in a corrupt bibtex file.

I’ll notify you here as soon as we have fixed this bug.

Regards
Matthias

5

Hi Matthias,

May be I found another source of the problem. I’ll try to explain the case clearly. If needed, I can send you the files saved at each step.
At the first time, I comprised the problem (not all the entries displayed) as I’ve lost all the entries not shown. I tried to load my bib file in an other tool, Referencer with Linux Mint. All the entries where shown. I did a new export in a new bib file to open it into jabref. I did some updates or merge in Jabref. After saving the data and open the file again in jabref, I got new errors. After having understood how to find the “bad entry” I saw other possible sources of y problem.
At the beginning my bib file results of the merging of different sources eg. Medline (export from pubmed, import into Endnote, export all the selection with a custom format (bibtex amended) to have the complete data, including abstract, DOI and PMID) in a .bib file; I did some other requests with other tools e.g. publishandperish or Mendeley.
After all these manipulations, I updated all my entries to complete the information I need.
So, into the .bib file some entries doesn’t have the “format used by Jabref”; I mean : @article{ref… begins at col 1, the indentation varies depending the source of the data (2 spaces (Jabref), 4 spaces or more (other tools), = sign aligned or not, indentation or alignment of the equal sign is made with tabs (Referencer), for the whole line, partially or not), fields order is not the same between the bib file exported from EndNote, Mendeley or Referencer (I suppose Referencer uses the bibtex library with Python ?) and the fields order done by Jabref; So, if you update an entry in Jabref “generated by Referencer for example”, into the file, you’ll find the new entry with the Jabref format and some garbage before the entry, especialy when you have data with special characters like @ in the institution field).
If you merge 2 entries, you’ll have the same result.
This is done may be because :

  1. Others tools doesn’t use the same rules than Jabref for indentation or = sign;.
  2. When the fields order are arranged in a different manner than Jabref do it, Jabref, even if the data is Ok in the bibtex view, the corresponding fields are not shown. For example, if the URL is not the last field of your entry, it is not shown in the detailed view.
    Sorry for this long long message; I hope it is understandable…
    Thanks a lot.
    Another possible issue : when you use the “Web interface” to search information to complete the data of an entry, when the “possible duplicate entry is shown”, I you want to merge the request result with the entry, it is not working.
    You can do the test with a medline publication : Get the DOI. Create the entry (get bibtex data from the DOI). Save the entry. Search the same information with the PMID into the Internet (websearch, source = medline). Try to merge the 2 entries (click on "possible duplicate entry into the list of the result - you must be have only one line after searching with the PMID) and try to save the merged data.

At last, I’ve 1100 entries in my bib file.

Regards

MVT aka Michel

6

Thanks for your long reply with detailed descriptions!

I’m currently out of office replying from my mobile, so I can not check whether the merge entry Dialog is also broken or whether this is related to the previously mentioned Problem with the ‘@’ symbol.
The ‘@’ problem should be solved with a build from http://builds.jabref.org/fix-1716/ - can you please check whether the other problems persist?

Thanks!

7

Thanks Matthias.
I’ll try it

8

@mvt91 Any feedback for us?

9

Hi Matthias,
I sent you “a long email” but it seems it didn’t work.
I’ll post my answer again.
Yes, it works.

Selon Matthias Geiger jabref@discoursemail.com:

@mvt91 Any feedback for us?

Visit Topic or reply
to this email to respond.

To unsubscribe from these emails, [click

here](Forum - JabRef).

10

Great! Glad to hear this!

11

Thanks Matthias for your quick update.
I tried to reproduce the error. Everything is OK.
I tried again transferring the “bad file” into Referencer, export it and open it in Jabref. It works.
The merge (duplicate found after a Web search) seems to work correctly too.
When you have an error, the file is completely loaded, bue the entry is not shown. You have to parse the file with an editor to find the lines “@comment” and remove the “bad lines”.

Thanks for everything.

This problem is solved.