Skip to main content

A day in the life of the arXiv admin team

The arXiv administrator team handles the 500-600 new article submissions that come into arXiv every day (double that on Mondays). When a user sends a paper to arXiv it goes through a series of checks to detect technical issues with the paper and also to make sure it meets our moderation standards. The administrators shepherd this process by responding to automated technical flags and communicating with our volunteer moderators who consider the classification and quality aspects of articles. We are also sending a constant stream of email to users in response to their questions or if we find issues with their papers.

In addition to those new papers, we have 300-400 daily submissions that update existing papers, either for replacement versions, journal references, or withdrawal requests. Each of those types of submissions are also checked to make sure they are well formed and appropriate.

Working on a system that has evolved over 25 years involves a workaround or two and different components working in parallel to complete some tasks. We jump from our user support email, to our submission discussion system used by moderators, to perl scripts for metadata and postscript fixes, to debugging LaTeX.

100% of the submissions are sent through our automated checks. We also eyeball the metadata for every submission. Typically around 15% of submissions get ‘fixed’ with some human curation, either cleaning up the metadata, classification changes by our moderators, or asking the user to fix technical issues. A small portion of submissions end up getting rejected from arXiv. It is one of those jobs where the vast majority of our effort is spent on a fairly small number of problem submissions. For most arXiv users our work, and that of our 160 volunteer moderators, is invisible. We get emails everyday from authors who are surprised to find out that not only can papers be delayed but that their own paper has been held up.

While our goal is rapid dissemination and to address all issues in a single day there are a variety of reasons why papers may be delayed. Some submissions just need an extra day or two for our volunteer moderators to look them over. Some raise challenging questions that we discuss at our weekly team meetings and may involve extensive discussion with our moderators.

For especially complex technical, policy or legal questions we can tap other members of the arXiv team. We work closely with Gail Steinhart, Cornell Scholarly Communication Librarian and arXiv Program Associate, on author disputes, developing best practices for user support, user engagement/testing, and researching copyright questions. We chat daily with arXiv’s developers for user reported bugs or to help answer user questions about bulk data access. Challenging policy issues may escalate to Steinn Sigurdsson, our Scientific Director.

On a daily basis the work has a mix of the fascinating and the mundane. We repeatedly see the same issues over and over, such as the author not noticing that their references did not appear to compile correctly in the final PDF (likely because they tried to upload references in bib format rather than bbl). While much of the technical help we provide, such as fixing TeX errors is routine, we also get some zingers that are fun to dive into and figure out. We also get drawn into challenging situations. We continually facilitate discussions with moderators and authors about what is ‘acceptable for arXiv’, professional ethics, scientific discourse, and arXiv moderation standards and transparency.

What is the arXiv admin team up to today? Amanda Bartley, arXiv Administrator, is working on user support. She had a hum dinger this morning of disentangling user accounts for what turned out to be an unauthorized proxy submitter which is against arXiv policy. Rebecca Goldweber, arXiv Associate Administrator, has been responding to system flags and following the moderator discussions, including a rare case where moderators from different subject areas both thought the paper best fit into their field. Jake Weiskoff, Senior arXiv Administrator, has over time become our resident TeX-spert . He has been debugging papers and working on a project to improve our process for fixing TeX accents in the metadata.

One of the big motivators for our team is the exciting developments in the communities we serve. We get caught up in the buzz of discovery as it happens. There is great sense of connection and commitment that comes from working alongside arXiv’s 170 moderators who volunteer their time and expertise every day for the benefit of arXiv users. We also have professional interest in the evolving communication needs of the scientists. In our desire to continually improve arXiv as a tool for the community we are thrilled by the major upgrades underway to arXiv’s infrastructure. We have an extensive wish list for improving the system and many feature requests from users that were not feasible in our legacy system but are making their way into planned updates.

Jim Entwood
arXiv Operations Manager

Annual Update

We are pleased to provide an update with a brief summary of our 2017 activities and 2018 plans:

We remain grateful for strong support from our member organizations, Simons Foundation, and essential contributions from arXiv’s advisory groups as they consistently provide us with input as representatives of scientific and library communities. We salute the contributions of 170 volunteer moderators who are crucial to our operation. Also we’d like to thank the Sloan Foundation and the Heising-Simons Foundation for their generous support of the next generation initiative.

arXiv Team
Oya Y. Rieger (Program Director), Steinn Sigurdsson (Scientific Director), Jim Entwood (Operations Manager), Martin Lessmeister (IT Lead), Sandy Payette (Technology Strategy Advisor), Erick Peirson (Lead Architect), Gail Steinhart (Program Associate), Chloe McLaren (Membership Program Coordinator)

arXiv Technical Evaluation Rubric

While we eventually decided to adopt an incremental, microservices-based approach to redeveloping arXiv (see the post arXiv NG: Classic Renewal for context), we did spend considerable time evaluating existing repository technologies. To that end, we developed a technical evaluation rubric that we applied to candidate technologies, and are pleased to share that here in case readers of this blog find value in using or adapting the rubric for their own purposes.

1991-2017 arXiv submission statistics available

We’re pleased to share the arXiv submission rate statistics for 1991-2017. The overall current submission rate (averaged over calendar year 2017) is 10293 submissions/month (123523 total for 2017). More detail, including breakdowns by subject area, is available on the website.

Visit to Astrophysics Data System

ADS (Astrophysics Data System) held its second ADS Users Group meeting on 2–3 November. In advance of the meeting, on 1 November I spent the day meeting with Alberto Accomazzi (ADS PI), Michael Kurtz (Project Scientist), Edwin Henneken (IT Specialist), and members of the ADS development team.

The ADS Digital Library was founded in the early 1990s, at about the same time as arXiv, and has become an indispensable resource for the astronomy and astrophysics research community (see Kurtz et al, 2000).

We’ve worked closely with ADS over the years. Given the recent ramp-up of development effort at ADS to support the ADS Bumblebee project, and the parallel ramp-up for arXiv NG, this is an opportune time for ADS and arXiv to coordinate and possibly collaborate on new problems of mutual interest. This post is a recap of some of the things that we discussed. Read more surpasses 1 billion downloads

Just a quick note that arXiv logged its one-billionth download at the end of October. Read the full story.

Open Access – It takes a community

Open Access Logo

Now in its 26th year owes it’s success as an Open Access e-print service to our community of users and supporters. During Open Access Week we would like to take the opportunity to say thanks to the authors that provide free access to their cutting edge research by posting articles to arXiv. We also wish to thank our team of volunteer moderators for ensuring arXiv submissions meet our moderation standards. We are grateful to Paul Ginsparg’s continuing inspiration and insight into the technical and social dynamics of arXiv. We also wish to thank the Scientific and Member Advisory Board members for guidance to help arXiv grow and evolve with the changing needs of the community.

Of course arXiv could not run without funding and we thank Cornell University, supporting foundations, member libraries, and donors for their ongoing support. We have also benefited recently from users who volunteer to take our surveys and provide feedback as we improve and upgrade services.

Part of arXiv’s success also comes from the ecosystem of open access scholarly communication. This includes partners like Inspire, and ADS, as well as individuals and groups building tools and services on top of arXiv.

Thanks all for your part in arXiv’s continued success!

Jim Entwood
Operations Manager
On behalf of the arXiv Team


The arXiv received 67 preprints as part of today’s announcement of the discovery by the LIGO/Virgo Collaboration of the coalescence of a binary neutron star in NGC4993, accompanied by a short Gamma Ray Burst.

The preprints were submitted over a period of several days and were held to be released together as a contiguous block on astro-ph and gr-qc.  Two of the preprints were held back because of technical issues leaving a batch of 65 to be released.  The plan to make the release a contiguous block of arXiv IDs failed for technical reasons,  our admins worked late to diagnose the source of the problem, which boiled down to a flaw in the script setting up the block, it assumed implicitly that any such block of preprints would be submitted on the same day…

The list of LVC and EM collaboration preprints we were informed about is below,  there are other manuscripts discussing the event, those are from independent researchers generally unaffiliated with the collaboration:

  1. GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral:
  2. Multi-messenger Observations of a Binary Neutron Star Merger:
  3. Gravitational Waves and Gamma-rays from a Binary Neutron Star Merger: GW170817 and GRB 170817A:
  4. A gravitational-wave standard siren measurement of the Hubble constant:
  5. Estimating the Contribution of Dynamical Ejecta in the Kilonova Associated with GW170817:
  6. GW170817: Implications for the Stochastic Gravitational-Wave Background from Compact Binary Coalescences:
  7. On the Progenitor of Binary Neutron Star Merger GW170817:
  8. Search for High-energy Neutrinos from Binary Neutron Star Merger GW170817 with ANTARES, IceCube, and the Pierre Auger Observatory:
  9. Fermi-LAT observations of the LIGO/Virgo event GW170817:
  10. An Ordinary Short Gamma-Ray Burst with Extraordinary Implications: Fermi-GBM Detection of GRB 170817A:
  11. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/Virgo GW170817. I. Dark Energy Camera Discovery of the Optical Counterpart:
  12. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. II. UV, Optical, and Near-IR Light Curves and Comparison to Kilonova Models:
  13. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. III. Optical and UV Spectra of a Blue Kilonova From Fast Polar Ejecta:
  14. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. IV. Detection of Near-infrared Signatures of r-process Nucleosynthesis with Gemini-South:
  15. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. V. Rising X-ray Emission from an Off-Axis Jet:
  16. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VI. Radio Constraints on a Relativistic Jet and Predictions for Late-Time Emission from the Kilonova Ejecta:
  17. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VII. Properties of the Host Galaxy and Constraints on the Merger Timescale:
  18. The Electromagnetic Counterpart of the Binary Neutron Star Merger LIGO/VIRGO GW170817. VIII. A Comparison to Cosmological Short-duration Gamma-ray Bursts:
  19. Swope Supernova Survey 2017a (SSS17a), the Optical Counterpart to a Gravitational Wave Source:
  20. Light Curves of the Neutron Star Merger GW170817/SSS17a: Implications for R-Process Nucleosynthesis:
  21. Early Spectra of the Gravitational Wave Source GW170817: Evolution of a Neutron Star Merger:
  22. The Unprecedented Properties of the First Electromagnetic Counterpart to a Gravitational Wave Source:
  23. Origin of the heavy elements in binary neutron-star mergers from a gravitational wave event:
  24. The Old Host-Galaxy Environment of SSS17a, the First Electromagnetic Counterpart to a Gravitational Wave Source:
  25. Electromagnetic Evidence that SSS17a is the Result of a Binary Neutron Star Merger:
  26. A Neutron Star Binary Merger Model for GW170817/GRB170817a/SSS17a:
  27. Illuminating Gravitational Waves: A Concordant Picture of Photons from a Neutron Star Merger:
  28. A Radio Counterpart to a Neutron Star Merger:
  29. Swift and NuSTAR observations of GW170817: detection of a blue kilonova:
  30. The X-ray counterpart to the gravitational wave event GW 170817:
  31. A kilonova as the electromagnetic counterpart to a gravitational-wave source:
  32. Optical Follow-up of Gravitational-wave Events with Las Cumbres Observatory:
  33. Optical emission from a kilonova following a gravitational-wave-detected neutron-star merger:
  34. Observations of the first electromagnetic counterpart to a gravitational wave source by the TOROS collaboration:
  35. The Emergence of a Lanthanide-Rich Kilonova Following the Merger of Two Neutron Stars:
  36. How Many Kilonovae Can Be Found in Past, Present, and Future Survey Datasets?:
  37. Optical Observations of LIGO Source GW 170817 by the Antarctic Survey Telescopes at Dome A, Antarctica:
  38. Follow up of GW170817 and its electromagnetic counterpart by Australian-led observing programs:
  39. ALMA and GMRT constraints on the off-axis gamma-ray burst 170817A from the binary neutron star merger GW170817:
  40. J-GEM observations of an electromagnetic counterpart to the neutron star merger GW170817:
  41. The unpolarized macronova associated with the gravitational wave event GW170817:
  42. Kilonova from post-merger ejecta as an optical and near-infrared counterpart of GW170817:
  43. MASTER optical detection of the first LIGO/Virgo neutron stars merging GW170817:
  44. A peculiar low-luminosity short gamma-ray burst from a double neutron star merger progenitor:
  45. AGILE Observations of the Gravitational Wave Source GW 170817: Constraining Gamma-Ray Emission from a NS-NS Coalescence:
  46. The Diversity of Kilonova Emission in Short Gamma-Ray Bursts:
  47. The environment of the binary neutron star merger GW170817:
  48. The first direct double neutron star merger detection: implications for cosmic nucleosynthesis:
  49. A Deep Chandra X-ray Study of Neutron Star Coalescence GW170817:
  50. Afterglows and Macronovae Associated with Nearby Low-Luminosity Short-Duration Gamma-Ray Bursts: Application to GW170817/GRB170817A:
  51. GRB170817A associated with GW170817: multifrequency observations and modeling of prompt gamma-ray emission:
  52. INTEGRAL Detection of the First Prompt Gamma-Ray Signal Coincident with the Gravitational Wave Event GW170817:
  53. The Rapid Reddening and Featureless Optical Spectra of the optical counterpart of GW170817, AT 2017gfo, During the First Four Days:
  54. The discovery of the electromagnetic counterpart of GW170817: kilonova AT 2017gfo/DLT17ck:
  55. A comparison between SALT/SAAO observations and kilonova models for AT 2017gfo: the first electromagnetic counterpart of a gravitational wave transient – GW170817:
  56. The Distance to NGC 4993: The Host Galaxy of the Gravitational-wave Event GW170817:
  57. GRB 170817A as a jet counterpart to gravitational wave trigger GW 170817:
  58. Spectroscopic identification of r-process nucleosynthesis in a double neutron star merger:
  59. Jet-driven and jet-less fireballs from compact binary mergers:
  60. Multimessenger tests of the weak equivalence principle from GW170817 and its electromagnetic counterparts:
  61. Distance and properties of NGC 4993 as the host galaxy of a gravitational wave source, GW170817:
  62. TeV gamma-ray observations of the binary neutron star merger GW170817 with H.E.S.S:
  63. Lanthanides or dust in kilonovae: lessons learned from GW170817:
  64. An empirical limit on the kilonova rate from the DLT40 one day cadence Supernova Survey:
  65. Subaru Hyper Suprime-Cam Survey for An Optical Counterpart of GW170817:

arXiv NG: Classic Renewal

To paraphrase an observation by our new Scientific Director: from the perspective of most of our users, arXiv runs on magic. With the exception of a small handful of hiccups, arXiv has just worked for over 25 years. Since I joined the arXiv IT team as lead software architect in June, I’ve been working hard to pull back the proverbial curtain and take stock of how the sausage is made, and to synthesize the team’s aspirations and expertise for the arXiv Next Generation (arXiv-NG) project. We’ve done a considerable amount of research and soul-searching, and an architecture for NG has emerged.

Over the coming weeks and months I’ll discuss the highlights of the NG architecture on this blog, and keep you up to date on development progress. This post is a brief 30,000-foot view of where we’re going over the next two years.

Read more

Donate to arXiv

arXiv will run an online fundraising campaign for four days, from October 16-19, 2017, to help raise additional funds (see Donations to arXiv). arXiv’s baseline maintenance costs are sponsored by more than 210 member organizations, the Simons Foundation, and Cornell University Library. This online campaign aims to garner additional resources from the program’s active and supportive user base.  Stewardship of resources such as arXiv involves not only covering the operational costs but also continuing to enhance their value based on the needs of the user community and the evolving patterns and modes of scholarly communication. It is essential to raise additonal funds in order to fund new initiatives that are beyond the routine operational work, and to robustly support arXiv’s Open Access mission. Donations to arXiv are tax deductible, eligible for employer matches via benevity, and easy to schedule.

Donations can be made here. We thank you for your support.

keep looking »

Skip to toolbar