ESDS Progress Over the Past Few Months#

The first ESDS “meeting” took place in mid March 2021, with the kickoff event being the CGD town hall (here is the blog post detailing the topics at that meeting). Since that meeting, we have moved forward, holding a variety of Work in Progress talks, partnering with Xdev to offer Python Tutorials, and publish a series of blog posts aimed at documenting scientific workflows here at NCAR.

Most of our blog posts thus far have been focused on Python tutorials, workflow examples, or conference summaries. One new type of post we will be posting now are monthly summaries on ESDS activities. Since we have not posted one yet, we grouped up all of our activities since early March. Going forward, we will post on a monthly basis!

We outline some of the key points of progress over the past few months below! We also have update our vision, values, and near-term goals, accessible on our about us page. If you are interested in more of the details, be sure to check out the ESDS Vision Whitepaper.

Work in Progress Talks#

If you are interested in the links to these talks, or the discussion points, check out the agenda using this link!

Workflow Examples#

  • HiRes CESM MARBL repo (5 April 2021) - Mike Levy (CGD)

  • SMYLE Analysis (19 April 2021) - Steve Yeager (CGD)

  • Data Preprocessing and Workflows for Machine Learning (12 July 2021) - Maria Molina (CGD)

  • Machine Learning at NCAR (9 August 2021) - David John Gagne (CISL)

  • Oxygen Minimum Zone Analysis with Pangeo (23 August 2021) - Julius Busecke (Columbia LDEO)

  • Fast parallel access to existing netCDF file collections with Zarr (20 September 2021) - Lucas Sterzinger (CISL)

Python Package Overviews#

We include links to the respective packages below!

Office Hours#

In addition to the educational content mentioned above, we have been offering weekly office hours, staffed by Xdev members.

We have had a total of 23 attendees so far, with the most common topics being Xarray, Matplotlib, and Dask.

office hours questions

Recently, we have a had a slight decline in attendance, so if you are interested in finding help with your Python-oriented workflow, be sure to stop by on Mondays at 3 PM MT!

xdev office hours attendance

Blog Posts#

Over the course of the last few months, we posted a total of 23 blog posts, not including posts related to Python tutorials.

Documentation and Citations#

Data Access#

Data Computation#

Data Visualization#

End to End Workflow#

Conference Summaries, Discussions, and Resources#

Python Tutorials#

As mentioned previously, the Python Tutorial series has been on ongoing training resource, led by Julia Kent and fellow Xdevers. Below is a list of the tutorial blog posts and associated videos.

Numpy Tutorial (10 March 2021)#

Matplotlib Tutorial (24 March 2021)#

Object Oriented Programming Tutorial (09 April 2021)#

Cartopy Tutorial (26 April 2021)#

Git and Github Tutorial (12 May 2021)#

Pandas Tutorial (26 May 2021)#

Xarray Tutorial (09 and 23 June 2021)#

Part 1#

Part 2#

Dask Tutorial (14 July and 11 August 2021)#

Part 1#

Part 2#

GeoCAT Viz Tutorial (25 August 2021)#

GeoCAT Comp Tutorial (08 September 2021)#