Skip to main content



A day in the life of the arXiv admin team

The arXiv administrator team handles the 500-600 new article submissions that come into arXiv every day (double that on Mondays). When a user sends a paper to arXiv it goes through a series of checks to detect technical issues with the paper and also to make sure it meets our moderation standards. The administrators shepherd this process by responding to automated technical flags and communicating with our volunteer moderators who consider the classification and quality aspects of articles. We are also sending a constant stream of email to users in response to their questions or if we find issues with their papers.

In addition to those new papers, we have 300-400 daily submissions that update existing papers, either for replacement versions, journal references, or withdrawal requests. Each of those types of submissions are also checked to make sure they are well formed and appropriate.

Working on a system that has evolved over 25 years involves a workaround or two and different components working in parallel to complete some tasks. We jump from our user support email, to our submission discussion system used by moderators, to perl scripts for metadata and postscript fixes, to debugging LaTeX.

100% of the submissions are sent through our automated checks. We also eyeball the metadata for every submission. Typically around 15% of submissions get ‘fixed’ with some human curation, either cleaning up the metadata, classification changes by our moderators, or asking the user to fix technical issues. A small portion of submissions end up getting rejected from arXiv. It is one of those jobs where the vast majority of our effort is spent on a fairly small number of problem submissions. For most arXiv users our work, and that of our 160 volunteer moderators, is invisible. We get emails everyday from authors who are surprised to find out that not only can papers be delayed but that their own paper has been held up.

While our goal is rapid dissemination and to address all issues in a single day there are a variety of reasons why papers may be delayed. Some submissions just need an extra day or two for our volunteer moderators to look them over. Some raise challenging questions that we discuss at our weekly team meetings and may involve extensive discussion with our moderators.

For especially complex technical, policy or legal questions we can tap other members of the arXiv team. We work closely with Gail Steinhart, Cornell Scholarly Communication Librarian and arXiv Program Associate, on author disputes, developing best practices for user support, user engagement/testing, and researching copyright questions. We chat daily with arXiv’s developers for user reported bugs or to help answer user questions about bulk data access. Challenging policy issues may escalate to Steinn Sigurdsson, our Scientific Director.

On a daily basis the work has a mix of the fascinating and the mundane. We repeatedly see the same issues over and over, such as the author not noticing that their references did not appear to compile correctly in the final PDF (likely because they tried to upload references in bib format rather than bbl). While much of the technical help we provide, such as fixing TeX errors is routine, we also get some zingers that are fun to dive into and figure out. We also get drawn into challenging situations. We continually facilitate discussions with moderators and authors about what is ‘acceptable for arXiv’, professional ethics, scientific discourse, and arXiv moderation standards and transparency.

What is the arXiv admin team up to today? Amanda Bartley, arXiv Administrator, is working on user support. She had a hum dinger this morning of disentangling user accounts for what turned out to be an unauthorized proxy submitter which is against arXiv policy. Rebecca Goldweber, arXiv Associate Administrator, has been responding to system flags and following the moderator discussions, including a rare case where moderators from different subject areas both thought the paper best fit into their field. Jake Weiskoff, Senior arXiv Administrator, has over time become our resident TeX-spert . He has been debugging papers and working on a project to improve our process for fixing TeX accents in the metadata.

One of the big motivators for our team is the exciting developments in the communities we serve. We get caught up in the buzz of discovery as it happens. There is great sense of connection and commitment that comes from working alongside arXiv’s 170 moderators who volunteer their time and expertise every day for the benefit of arXiv users. We also have professional interest in the evolving communication needs of the scientists. In our desire to continually improve arXiv as a tool for the community we are thrilled by the major upgrades underway to arXiv’s infrastructure. We have an extensive wish list for improving the system and many feature requests from users that were not feasible in our legacy system but are making their way into planned updates.

Jim Entwood
arXiv Operations Manager

Annual Update

We are pleased to provide an update with a brief summary of our 2017 activities and 2018 plans:

https://confluence.cornell.edu/display/arxivpub/arXiv+Update+-+January+2018

We remain grateful for strong support from our member organizations, Simons Foundation, and essential contributions from arXiv’s advisory groups as they consistently provide us with input as representatives of scientific and library communities. We salute the contributions of 170 volunteer moderators who are crucial to our operation. Also we’d like to thank the Sloan Foundation and the Heising-Simons Foundation for their generous support of the next generation initiative.

arXiv Team
Oya Y. Rieger (Program Director), Steinn Sigurdsson (Scientific Director), Jim Entwood (Operations Manager), Martin Lessmeister (IT Lead), Sandy Payette (Technology Strategy Advisor), Erick Peirson (Lead Architect), Gail Steinhart (Program Associate), Chloe McLaren (Membership Program Coordinator)

arXiv Technical Evaluation Rubric

While we eventually decided to adopt an incremental, microservices-based approach to redeveloping arXiv (see the post arXiv NG: Classic Renewal for context), we did spend considerable time evaluating existing repository technologies. To that end, we developed a technical evaluation rubric that we applied to candidate technologies, and are pleased to share that here in case readers of this blog find value in using or adapting the rubric for their own purposes.

1991-2017 arXiv submission statistics available

We’re pleased to share the arXiv submission rate statistics for 1991-2017. The overall current submission rate (averaged over calendar year 2017) is 10293 submissions/month (123523 total for 2017). More detail, including breakdowns by subject area, is available on the arXiv.org website.

Visit to Astrophysics Data System

ADS (Astrophysics Data System) held its second ADS Users Group meeting on 2–3 November. In advance of the meeting, on 1 November I spent the day meeting with Alberto Accomazzi (ADS PI), Michael Kurtz (Project Scientist), Edwin Henneken (IT Specialist), and members of the ADS development team.

The ADS Digital Library was founded in the early 1990s, at about the same time as arXiv, and has become an indispensable resource for the astronomy and astrophysics research community (see Kurtz et al, 2000).

We’ve worked closely with ADS over the years. Given the recent ramp-up of development effort at ADS to support the ADS Bumblebee project, and the parallel ramp-up for arXiv NG, this is an opportune time for ADS and arXiv to coordinate and possibly collaborate on new problems of mutual interest. This post is a recap of some of the things that we discussed. Read more

arXiv.org surpasses 1 billion downloads

Just a quick note that arXiv logged its one-billionth download at the end of October. Read the full story.

Open Access – It takes a community

Open Access Logo

Now in its 26th year arXiv.org owes it’s success as an Open Access e-print service to our community of users and supporters. During Open Access Week we would like to take the opportunity to say thanks to the authors that provide free access to their cutting edge research by posting articles to arXiv. We also wish to thank our team of volunteer moderators for ensuring arXiv submissions meet our moderation standards. We are grateful to Paul Ginsparg’s continuing inspiration and insight into the technical and social dynamics of arXiv. We also wish to thank the Scientific and Member Advisory Board members for guidance to help arXiv grow and evolve with the changing needs of the community.

Of course arXiv could not run without funding and we thank Cornell University, supporting foundations, member libraries, and donors for their ongoing support. We have also benefited recently from users who volunteer to take our surveys and provide feedback as we improve and upgrade services.

Part of arXiv’s success also comes from the ecosystem of open access scholarly communication. This includes partners like Inspire, and ADS, as well as individuals and groups building tools and services on top of arXiv.

Thanks all for your part in arXiv’s continued success!

Jim Entwood
Operations Manager
On behalf of the arXiv Team

GW170817

The arXiv received 67 preprints as part of today’s announcement of the discovery by the LIGO/Virgo Collaboration of the coalescence of a binary neutron star in NGC4993, accompanied by a short Gamma Ray Burst.

The preprints were submitted over a period of several days and were held to be released together as a contiguous block on astro-ph and gr-qc.  Two of the preprints were held back because of technical issues leaving a batch of 65 to be released.  The plan to make the release a contiguous block of arXiv IDs failed for technical reasons,  our admins worked late to diagnose the source of the problem, which boiled down to a flaw in the script setting up the block, it assumed implicitly that any such block of preprints would be submitted on the same day…

The list of LVC and EM collaboration preprints we were informed about is below,  there are other manuscripts discussing the event, those are from independent researchers generally unaffiliated with the collaboration:

  1. GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral: https://arxiv.org/abs/1710.05832
  2. Multi-messenger Observations of a Binary Neutron Star Merger: https://arxiv.org/abs/1710.05833
  3. Gravitational Waves and Gamma-rays from a Binary Neutron Star Merger: GW170817 and GRB 170817A: https://arxiv.org/abs/1710.05834
  4. A gravitational-wave standard siren measurement of the Hubble constant: https://arxiv.org/abs/1710.05835
  5. Estimating the Contribution of Dynamical Ejecta in the Kilonova Associated with GW170817: https://arxiv.org/abs/1710.05836
  6. GW170817: Implications for the Stochastic Gravitational-Wave Background from Compact Binary Coalescences: https://arxiv.org/abs/1710.05837
  7. On the Progenitor of Binary Neutron Star Merger GW170817: https://arxiv.org/abs/1710.05838
  8. Search for High-energy Neutrinos from Binary Neutron Star Merger GW170817 with ANTARES, IceCube, and the Pierre Auger Observatory: https://arxiv.org/abs/1710.05839
  9. Fermi-LAT observations of the LIGO/Virgo event GW170817: https://arxiv.org/abs/1710.05450
  10. An Ordinary Short Gamma-Ray Burst with Extraordinary Implications: Fermi-GBM Detection of GRB 170817A: https://arxiv.org/abs/1710.05446
  11. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/Virgo GW170817. I. Dark Energy Camera Discovery of the Optical Counterpart: https://arxiv.org/abs/1710.05459
  12. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. II. UV, Optical, and Near-IR Light Curves and Comparison to Kilonova Models:https://arxiv.org/abs/1710.05840
  13. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. III. Optical and UV Spectra of a Blue Kilonova From Fast Polar Ejecta: https://arxiv.org/abs/1710.05456
  14. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. IV. Detection of Near-infrared Signatures of r-process Nucleosynthesis with Gemini-South:https://arxiv.org/abs/1710.05454
  15. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. V. Rising X-ray Emission from an Off-Axis Jet: https://arxiv.org/abs/1710.05431
  16. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VI. Radio Constraints on a Relativistic Jet and Predictions for Late-Time Emission from the Kilonova Ejecta: https://arxiv.org/abs/1710.05457
  17. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VII. Properties of the Host Galaxy and Constraints on the Merger Timescale:https://arxiv.org/abs/1710.05458
  18. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VIII. A Comparison to Cosmological Short-duration Gamma-ray Bursts: https://arxiv.org/abs/1710.05438
  19. Swope Supernova Survey 2017a (SSS17a), the Optical Counterpart to a Gravitational Wave Source: https://arxiv.org/abs/1710.05452
  20. Light Curves of the Neutron Star Merger GW170817/SSS17a: Implications for R-Process Nucleosynthesis: https://arxiv.org/abs/1710.05443
  21. Early Spectra of the Gravitational Wave Source GW170817: Evolution of a Neutron Star Merger: https://arxiv.org/abs/1710.05432
  22. The Unprecedented Properties of the First Electromagnetic Counterpart to a Gravitational Wave Source: https://arxiv.org/abs/1710.05440
  23. Origin of the heavy elements in binary neutron-star mergers from a gravitational wave event: https://arxiv.org/abs/1710.05463
  24. The Old Host-Galaxy Environment of SSS17a, the First Electromagnetic Counterpart to a Gravitational Wave Source: https://arxiv.org/abs/1710.05439
  25. Electromagnetic Evidence that SSS17a is the Result of a Binary Neutron Star Merger: https://arxiv.org/abs/1710.05434
  26. A Neutron Star Binary Merger Model for GW170817/GRB170817a/SSS17a: https://arxiv.org/abs/1710.05453
  27. Illuminating Gravitational Waves: A Concordant Picture of Photons from a Neutron Star Merger: https://arxiv.org/abs/1710.05436
  28. A Radio Counterpart to a Neutron Star Merger: https://arxiv.org/abs/1710.05435
  29. Swift and NuSTAR observations of GW170817: detection of a blue kilonova: https://arxiv.org/abs/1710.05437
  30. The X-ray counterpart to the gravitational wave event GW 170817: https://arxiv.org/abs/1710.05433
  31. A kilonova as the electromagnetic counterpart to a gravitational-wave source: https://arxiv.org/abs/1710.05841
  32. Optical Follow-up of Gravitational-wave Events with Las Cumbres Observatory: https://arxiv.org/abs/1710.05842
  33. Optical emission from a kilonova following a gravitational-wave-detected neutron-star merger: https://arxiv.org/abs/1710.05843
  34. Observations of the first electromagnetic counterpart to a gravitational wave source by the TOROS collaboration: https://arxiv.org/abs/1710.05844
  35. The Emergence of a Lanthanide-Rich Kilonova Following the Merger of Two Neutron Stars: https://arxiv.org/abs/1710.05455
  36. How Many Kilonovae Can Be Found in Past, Present, and Future Survey Datasets?: https://arxiv.org/abs/1710.05845
  37. Optical Observations of LIGO Source GW 170817 by the Antarctic Survey Telescopes at Dome A, Antarctica: https://arxiv.org/abs/1710.05462
  38. Follow up of GW170817 and its electromagnetic counterpart by Australian-led observing programs: https://arxiv.org/abs/1710.05846
  39. ALMA and GMRT constraints on the off-axis gamma-ray burst 170817A from the binary neutron star merger GW170817: https://arxiv.org/abs/1710.05847
  40. J-GEM observations of an electromagnetic counterpart to the neutron star merger GW170817: https://arxiv.org/abs/1710.05848
  41. The unpolarized macronova associated with the gravitational wave event GW170817: https://arxiv.org/abs/1710.05849
  42. Kilonova from post-merger ejecta as an optical and near-infrared counterpart of GW170817: https://arxiv.org/abs/1710.05850
  43. MASTER optical detection of the first LIGO/Virgo neutron stars merging GW170817: https://arxiv.org/abs/1710.05461
  44. A peculiar low-luminosity short gamma-ray burst from a double neutron star merger progenitor: https://arxiv.org/abs/1710.05851
  45. AGILE Observations of the Gravitational Wave Source GW 170817: Constraining Gamma-Ray Emission from a NS-NS Coalescence: https://arxiv.org/abs/1710.05460
  46. The Diversity of Kilonova Emission in Short Gamma-Ray Bursts: https://arxiv.org/abs/1710.05442
  47. The environment of the binary neutron star merger GW170817: https://arxiv.org/abs/1710.05444
  48. The first direct double neutron star merger detection: implications for cosmic nucleosynthesis: https://arxiv.org/abs/1710.05445
  49. A Deep Chandra X-ray Study of Neutron Star Coalescence GW170817: https://arxiv.org/abs/1710.05852
  50. Afterglows and Macronovae Associated with Nearby Low-Luminosity Short-Duration Gamma-Ray Bursts: Application to GW170817/GRB170817A: https://arxiv.org/abs/1710.05910
  51. GRB170817A associated with GW170817: multifrequency observations and modeling of prompt gamma-ray emission: https://arxiv.org/abs/1710.05448
  52. INTEGRAL Detection of the First Prompt Gamma-Ray Signal Coincident with the Gravitational Wave Event GW170817: https://arxiv.org/abs/1710.05449
  53. The Rapid Reddening and Featureless Optical Spectra of the optical counterpart of GW170817, AT 2017gfo, During the First Four Days: https://arxiv.org/abs/1710.05853
  54. The discovery of the electromagnetic counterpart of GW170817: kilonova AT 2017gfo/DLT17ck: https://arxiv.org/abs/1710.05854
  55. A comparison between SALT/SAAO observations and kilonova models for AT 2017gfo: the first electromagnetic counterpart of a gravitational wave transient – GW170817:https://arxiv.org/abs/1710.05855
  56. The Distance to NGC 4993: The Host Galaxy of the Gravitational-wave Event GW170817: https://arxiv.org/abs/1710.05856
  57. GRB 170817A as a jet counterpart to gravitational wave trigger GW 170817: https://arxiv.org/abs/1710.05857
  58. Spectroscopic identification of r-process nucleosynthesis in a double neutron star merger: https://arxiv.org/abs/1710.05858
  59. Jet-driven and jet-less fireballs from compact binary mergers: https://arxiv.org/abs/1710.05859
  60. Multimessenger tests of the weak equivalence principle from GW170817 and its electromagnetic counterparts: https://arxiv.org/abs/1710.05860
  61. Distance and properties of NGC 4993 as the host galaxy of a gravitational wave source, GW170817: https://arxiv.org/abs/1710.05861
  62. TeV gamma-ray observations of the binary neutron star merger GW170817 with H.E.S.S: https://arxiv.org/abs/1710.05862
  63. Lanthanides or dust in kilonovae: lessons learned from GW170817: https://arxiv.org/abs/1710.05863
  64. An empirical limit on the kilonova rate from the DLT40 one day cadence Supernova Survey: https://arxiv.org/abs/1710.05864
  65. Subaru Hyper Suprime-Cam Survey for An Optical Counterpart of GW170817: https://arxiv.org/abs/1710.05865

arXiv NG: Classic Renewal

To paraphrase an observation by our new Scientific Director: from the perspective of most of our users, arXiv runs on magic. With the exception of a small handful of hiccups, arXiv has just worked for over 25 years. Since I joined the arXiv IT team as lead software architect in June, I’ve been working hard to pull back the proverbial curtain and take stock of how the sausage is made, and to synthesize the team’s aspirations and expertise for the arXiv Next Generation (arXiv-NG) project. We’ve done a considerable amount of research and soul-searching, and an architecture for NG has emerged.

Over the coming weeks and months I’ll discuss the highlights of the NG architecture on this blog, and keep you up to date on development progress. This post is a brief 30,000-foot view of where we’re going over the next two years.

Read more

Donate to arXiv

arXiv will run an online fundraising campaign for four days, from October 16-19, 2017, to help raise additional funds (see Donations to arXiv). arXiv’s baseline maintenance costs are sponsored by more than 210 member organizations, the Simons Foundation, and Cornell University Library. This online campaign aims to garner additional resources from the program’s active and supportive user base.  Stewardship of resources such as arXiv involves not only covering the operational costs but also continuing to enhance their value based on the needs of the user community and the evolving patterns and modes of scholarly communication. It is essential to raise additonal funds in order to fund new initiatives that are beyond the routine operational work, and to robustly support arXiv’s Open Access mission. Donations to arXiv are tax deductible, eligible for employer matches via benevity, and easy to schedule.

Donations can be made here. We thank you for your support.

keep looking »

Skip to toolbar