One possibility is that the paper exists, but someone got the bibliographic information wrong, and everyone else simply copied this wrong bibliographic info. (A variation would be that the info supplied by the "export citation" functionality could be wrong, and nobody corrected it when they imported it into their literature database.) So your first order of business should be to investigate whether the paper in question exists "elsewhere". Authors and titles are usually not affected by this kind of error, so search for those first.
In parallel, write to the authors of the papers that cited this mystery paper. Ask them very politely that the citation may be off and that you cannot locate the paper. Perhaps they could send you this paper in electronic form? If the paper is cited widely, then someone at least should be able to help you.
In parallel, search for the authors of this paper, and contact them if possible. They will most likely have the biggest incentive of getting back to you.
Make notes of all your efforts, including the initial search by your librarian. If nobody is responsive, and you can't find the paper itself, and you can't even locate the authors of your mystery paper, then you may have a paper in the making... just not the kind of paper you originally set out to write.
