I just tried the method you mentioned, when I drag and drop the file, JabRef did generate a bib doc for the pdf based on XMP metadata, but it didn't rename the original pdf file. Is there any option to rename the pdf file based on bibtex key? I didn't find it on my own.
Regarding importing arXiv papers, I was just told that there is an "arXiv" type of entry so that the updated bibtex info can be fetched from arxiv.org if I give the arXiv number. Just tested it and worked out.