Standing Committee Meeting Report Data Services

September 2017

Fall 2017 Action Items

Action Items from the Spring 2017 DSSC Meeting

Action Item S2017:5

Development of the Jupyter notebook template for user interaction with MUSTANG system is recommended, with a caveat that actual development be delayed until the Fall when funding environment is clearer.
Responsible: DMC QA Group
Status:  QA Staff have started looking into creating Jupyter notebook tutorials about using MUSTANG for quality assurance

Action Items from the Fall 2017 DSSC Meeting

Action Item F2017:1

Previous Action Item S2017:4 addressed the issue related to the HPCWG effort lead by Carl Tape initially started by the DSSC.  It is felt that the CIG is the more appropriate organization to act as the coordinator of IRIS.  IRIS should continue to participate in this effort as a member of this effort, not as the lead organization.
This decision should be communicated to the CIG and their reaction to this recommendation communicated to the IRIS DSSC.
Responsible: Ahern

Action Item F2017:2

Previous Action Item F016:1 Follow up with Greg on archiving historical data, often from other agencies. Are there potential sources of funding within other parts of NSF?
Responsible: Stump will assume full responsibility but the point of contact should move from Anderson to Benoit.
Responsible: Stump

Action Item F2017:3

The modified Data Provider Agreement (DPA) was approved by the DSSC with the addition that wording will be added to assist contributors of non-seismic data sets to understand the formatting requirements of the timeseries data (miniSeed) and the metadata (dataless Seed or StationXML) as well as the identification of tools that can assist in creation of the timeseries and the metadata and if necessary consult the Data Services Director for assistance.
Responsible: Ahern
Status: The modified DPA was posted on the DSSC page as version 1.5.  Considered complete.

Action Item F2017:4

When the outcome of the NGEO proposal is known, organize a teleconference of the DSSC to discuss the outcomes of the award with specific DSSC impacts.  Identify this as something that all Standing Committees should consider as well.  Consider having the Spring 2018 DSSC in Boulder with overlap with the UNAVCO GDSAC.
Responsible: Stump

Action Item F2017:5

Ahern to send Data Services component of the NGEO proposal to new members of the DSSC (Thomas, Holtkamp, Pyle, Rubenstein)
Responsible: Ahern

Action Item F2017:6

DSSC members should review the governance document distributed at the meeting and provide specific feedback to Stump to take to the COCOM and BOD meetings.
Responsible: Stump and DSSC

Action Item F2017:7

DMC staff should demonstrate the EMC capabilities and how to interact with it at the IRIS AGU Booth.
Reposnible: Ahern

Action Item F2017:8

Discuss the development of EMC as an educational tool and consider involving IRIS EPO in this discussion.
Responsible: Trabant, Levin, and Taber

Action Item F2017:9

Take the idea of development of EMC tutorials, workshops, etc. to the COCOM and bring EPO into this activity.
Responsible: Stump and Ahern

Action Item F2017:10

Resume the DMC-DCC meetings on an annual basis targeting a single day meeting.
Responsible: Benson

Action Item F2017:11

Communicate to Woodward the need for the ISSC to recommend a new QAAC chair. The DSSC recommends letting Levin chair the next QAAC meeting with participation of the new ISSC appointee. DSSC recommendations for various QAAC positions are identified in the minutes.

  • Holtkamp as vice-chair,
  • Alexandros Savvaidis (TexNet), and
  • Fan-Chi Lin for Large-N

Responsible: Ahern, Levin, Stump

Action Item F2017:12

DSSC decided to not move forward with the HVSR product. Instead they encourage DMC staff to draft an SRL article describing the application, making the code available on GitHub.  Victor Tsai will help find relevant co-authors to participate.
Responsible: Ahern, Trabant and Tsai                                                                                                                                                                                                                      

Action Item F2017:13

The DSSC endorsed to plan related to developing the miniSeed3 prototype format. This includes the development of a miniSeed2 to miniSeed3 convertor, modifications to libmseed to make many clients support either miniSeed2 or miniSeed3 formats.
Responsible: Ahern

Action Item F2017:14

DSSC endorsed the DMC managing data from the pre-digital CIW data assuming that they meet the requirements in the Data Provider Agreement. Communicate this to CIW and at that point no further action required.
Responsible: Ahern

Action Item F2017:15

The parties responsible below will develop a short (2-3 pages) “Strategic Priorities of Data Services Looking Forward”.  It will be circulated to the DSSC for comment and improvements.
Responsible: Levin, Ahern, Stump, Thomas

Action Item F2017:16

Explore the idea of recommending the BoD consider seeking an NRC study related to seismology, seismological infrastructure, etc. that could be presented to the NSF. Perhaps a one-day NRC panel to discuss the long-term health of seismological infrastructure in the US.  Take to COCOM.
Responsible: Stump and Ahern

Action Item F2017:17

Respond to the ISSC that the DSSC considered ways to move forward regarding the concept of “data bricks, real or virtual”. The committee was enthusiastic in its support for robust datasets that can be rapidly accessed after they are built locally.  The response will be sent to the ISSC and presented at Cocom.
Responsible: Stump and Ahern

Action Item F2017:18

The DSSC endorsed the plan presented by Ahern related to unfinished tasks and carryover. Regarding carryover it specifically endorsed the following use of carryover for three tasks and all or part of a fourth task depending on

  • Task 1: Replace DMC and ADC DBMS Servers
  • Task 2: Expand Hitachi Storage System
  • Task 3: miniSeed2 to miniSeed3 bidirectional format translator
  • Task 4: Expand SeisComp3 web services
    • Subtask 4.1 matchtimeseries option in the station web service
    • Subtask 4.2  availability web service
    • Subtask 4.3 timeseries web service
    • Subtask 4.4 timeseriesplot web service

Action Item F2017:19

The DSSC discussed potential new members of the committee.  Four possible members were identified and will be taken to the CoCom.  Also take the name of the DSSC liaison to IDSC to Cocom.
Responsible: Stump and Ahern

Action Item F2017:20

The DSSC hopes to have the next DSSC meeting in the Boulder area in conjunction with UNAVCO.  Ahern to check with Meertens as to the viability of meeting at UNAVCO.
Responsible: Ahern

Action Item F2017:21

Consolidate responses regarding the IRIS DS Data Workshops including their direct benefit to IRIS and their importance to long-term IRIS goals and needs.  When ready take to the BoD for their guidance.
Responsible: Ahern

Brief Meeting Summary

The IRIS DSSC met in New Mexico on September 28-29, 2017 and had meetings at both the Albuquerque Seismological Laboratory and the  PASSCAL Instrument Center, two of the most important data centers for NSF funded data entering the DMC.

Previous Action items were reviewed along with their current status.  The Action Items are posted on the DSSC web site at http://dev.iris.edu/hq/meeting_report/ds/2017/09/.

The DSSC reviewed a modified Data Provider Agreement that now includes a more complete list of requirements for potential contributors of data to the DMC as well as clearly indicating services the DMC will provide to them in return.  Additionally better guidelines were included in the DPA for time series data from new types of sensors.  The DSSC added links to specific tools in the DPA to help users provide their data in acceptable formats.

Ahern updated the committee on current plans to develop an updated version of the miniSeed format to better accommodate deployments of large numbers of recording channels such as will be generated in Large-N deployments, QuakeCatcher Network, and deployments of sensors connected to mobile phones.  The DSSC supported the development of an IRIS prototype format named miniSeed3 as a possible option.  IRIS continues working with the FDSN to attempt to coordinate the development internationally.

The DSSC was updated on the status of on-going projects and product development at the DMC. All projects are on schedule and will be completed before the end of NSF SAGE Year 5. The DSSC encouraged more engagement from IRIS EPO in how the Earth Model Collaboration (EMC) can be used in an educational and general public settings resulting in demonstrations of the EPO involvement of EMC at the IRIS booth at the AGU.

Working closely with the PIC, the DMC now operates a ph5 data repository at the DMC and has roughly 50 data sets managed in this manner.  IRIS DMC staff have developed identical web services for the ph5 repository as for the passive source repository that IRIS has operated for decades.  This will enable full integration of the active source and passive source data at the DMC and will be allow seamless access through all DMC access methods in the future.

The DSSC endorsed the concept of the data from a former analog, but broadband, global network deployed and operated by Carnegie Institute be managed at the DMC since it meets the requirements of the DPA and will not have a major impact on the DMC.

DSSC did not support the development of the Horizontal Vertical Spectral Ratio (HVSR) product as a routinely generated product. Instead they tasked DMC staff to write a paper with community authors and submit it to SRL identifying that the system has been developed and the software is freely available and can be used by outside parties to generate such products. The DMC will not generate these products on a routine basis.

The DSSC discussed the idea of distributing data sets (such as GSN or TA)  on physical data bricks. While the technology exists to do this, the impact on human resources is a concern.  The DSSC supported a multistep approach to this issue beginning with the development of an error-free data transmission system and then reevaluate what is needed as a next step.  This will address the occasional errors where web services fail quietly and a small percentage of requests do not result in data reaching the client.

The DSSC discussed, modified, and approved the use of anticipated DS carryover that resulted from a variety of sources, much related to salary savings resulting from funded EarthCube projects.  This list of priorities was forwarded to the IRIS Cocom and then to the Board as input to the larger IRIS process.