esmtools: a toolbox for Earth system model analysis

Most Recent Release

v1.1.1 of esmtools mainly introduces dask-friendly, vectorized, lightweight functions for standard statistical functions. They also intelligently handle datetimes on the independent (x) axis:

• linear_slope()
• linregress()
• polyfit()
• rm_poly()
• rm_trend()

Installation

You can install the latest release of esmtools using pip or conda:

pip install esmtools

conda install -c conda-forge esmtools

You can also install the bleeding edge (pre-release versions) by running

pip install git+https://github.com/bradyrx/esmtools@master --upgrade

Getting Started

• Examples
• Calling Functions via Accessors

Examples

import xarray as xr
import PMMPIESM as PM
import glob
import numpy as np
import matplotlib.pyplot as plt

Takahashi Decomposition

path = '/work/mh0727/m300524/experiments/results/'

tos = xr.open_dataarray(path+'control_tos_mm.nc')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/times.py:122: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  result = decode_cf_datetime(example_value, units, calendar)
/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/variables.py:69: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  return self.func(self.array)

spco2 = xr.open_dataarray(path+'control_spco2_mm.nc')*10

ds = xr.merge([tos,spco2])

def temp_decomp_takahashi(ds, time_dim='time'):
    """
    Decompose spco2 into thermal and non-thermal component.

    Reference
    ---------
    Takahashi, Taro, Stewart C. Sutherland, Colm Sweeney, Alain Poisson, Nicolas Metzl, Bronte Tilbrook,
    Nicolas Bates, et al. “Global Sea–Air CO2 Flux Based on Climatological Surface Ocean PCO2, and Seasonal
    Biological and Temperature Effects.” Deep Sea Research Part II: Topical Studies in Oceanography, The
    Southern Ocean I: Climatic Changes in the Cycle of Carbon in the Southern Ocean, 49, no. 9 (January 1,2002):
    1601–22. https://doi.org/10/dmk4f2.

    Input
    -----
    ds : xr.Dataset containing spco2[ppm] and tos[C or K]

    Output
    ------
    thermal, non_thermal : xr.DataArray
        thermal and non-thermal components in ppm units

    """
    fac = 0.0432
    tos_mean = ds['tos'].mean(time_dim)
    tos_diff = ds['tos'] - tos_mean
    thermal = ds['spco2'].mean(time_dim) * (np.exp(tos_diff * fac))
    non_thermal = ds['spco2'] * (np.exp(tos_diff * -fac))
    return thermal, non_thermal

thermal, non_thermal = temp_decomp_takahashi(ds)

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/core/nanops.py:161: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)
/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/core/nanops.py:161: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

thermal_seasonality = thermal.groupby('time.month').mean('time')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/core/nanops.py:161: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

thermal_seasonality.plot(col='month',col_wrap=3, yincrease=False, robust=True)

<xarray.plot.facetgrid.FacetGrid at 0x2afa998ef160>

non_thermal_seasonality = non_thermal.groupby('time.month').mean('time')

non_thermal_seasonality.plot(col='month',col_wrap=3, yincrease=False, robust=True)

<xarray.plot.facetgrid.FacetGrid at 0x2afa99b684a8>

Potential pco2

h3d = xr.open_dataset('/work/mh0727/m300524/experiments/vga0214/outdata/hamocc/vga0214_hamocc_data_3d_mm_32920101_32921231.nc')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/times.py:122: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  result = decode_cf_datetime(example_value, units, calendar)
/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/variables.py:69: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  return self.func(self.array)

talk = h3d['talk']
dissic = h3d['dissic']

# dirty fix to get pco2_insitu
k=1
pco2_insitu = k*(2* dissic - talk)**2/(talk-dissic)

m3d = xr.open_dataset('/work/mh0727/m300524/experiments/vga0214/outdata/mpiom/vga0214_mpiom_data_3d_mm_32920101_32921231.nc')

t_insitu = m3d['thetao']

def potential_pco2(t_insitu, pco2_insitu):
    """
    Calculate potential pco2 in the inner ocean.

    Reference:
    - Sarmiento, Jorge Louis, and Nicolas Gruber. Ocean Biogeochemical Dynamics.
        Princeton, NJ: Princeton Univ. Press, 2006., p.421, eq. (10:3:1)

    """
    t_sfc = t_insitu.sel(depth=6)
    pco2_potential = pco2_insitu * (1 + 0.0423 * (t_sfc - t_insitu))
    return pco2_potential

pot_pco2 = potential_pco2(t_insitu, pco2_insitu)

(pot_pco2/pco2_insitu).mean('time').isel(x=0).plot(yincrease=False)
plt.title('N-S section Pacific: Factor increase of potential_pCO2 compared to pCO2')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/core/nanops.py:161: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

Text(0.5,1,'N-S section Pacific: Factor increase of potential_pCO2 compared to pCO2')

import xarray as xr
import PMMPIESM as PM
import glob
import numpy as np

path = '/work/mh0727/m300524/experiments/results/'

tos = xr.open_dataarray(path+'control_tsw_mm.nc')

light = xr.open_dataarray(path+'control_soflwac_mm.nc')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/times.py:122: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  result = decode_cf_datetime(example_value, units, calendar)
/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/coding/variables.py:69: SerializationWarning: Unable to decode time axis into full numpy.datetime64 objects, continuing using dummy cftime.datetime objects instead, reason: dates out of range
  return self.func(self.array)

fe = xr.open_dataarray(path+'control_dfeos_mm.nc')

po4 = xr.open_dataarray(path+'control_po4os_mm.nc')

no3 = xr.open_dataarray(path+'control_no3os_mm.nc')

ds_nut = xr.merge([fe,po4,no3])
ds_tp = xr.merge([tos,light])

Nutrient availability

Context: value of 1 means limited by nutrient. The less the more primary productivity.

nutlim, iron_lim, nitrate_lim, phos_lim = PM.hamocc.get_nutlimf(ds_nut)

nutlim_seasonality = nutlim.groupby('time.month').mean('time')

/work/mh0727/m300524/anaconda3/envs/my_jupyter/lib/python3.6/site-packages/xarray/core/nanops.py:161: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis=axis, dtype=dtype)

nutlim_seasonality.plot(col='month',col_wrap=3, yincrease=False)

<xarray.plot.facetgrid.FacetGrid at 0x2b188e025a58>

iron_lim_seasonality = iron_lim.groupby('time.month').mean('time')

iron_lim_seasonality.plot(col='month',col_wrap=3, yincrease=False, vmin=.95)

<xarray.plot.facetgrid.FacetGrid at 0x2b19b3886898>

nitrate_lim_seasonality = nitrate_lim.groupby('time.month').mean('time')

nitrate_lim_seasonality.plot(col='month',col_wrap=3, yincrease=False, vmin=.95)

<xarray.plot.facetgrid.FacetGrid at 0x2b19b4c31b70>

Temp light dependence

Temperature-light dependent primary productivity growth factor. The larger the more PP.

ds.data_vars

Data variables:
    tos      (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan nan
    soflwac  (time, lat, lon) float32 nan nan nan nan nan ... 0.0 0.0 0.0 0.0
    dfeos    (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan nan
    po4os    (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan nan
    no3os    (time, y, x) float32 nan nan nan nan nan ... nan nan nan nan nan

def temfa_phofa(ds):
    temfa = .6 * 1.066 ** (ds['tsw'] - 273.15)
    phofa = ds['soflwac'] * 0.02
    return temfa * phofa / (np.sqrt( phofa ** 2 + temfa ** 2 ))

tp = temfa_phofa(ds_tp)

tp_seasonality = tp.groupby('time.month').mean('time')

tp_seasonality.plot(col='month',col_wrap=3, cmap='viridis', robust=True, levels=10, vmin=0)

<xarray.plot.facetgrid.FacetGrid at 0x2b19b2cc6f60>

cyanos

Calling Functions via Accessors

A subset of esmtools functions are registered as xarray accessors. What this means is that you can call some of these functions as you would .isel(), .coarsen(), .interp(), and so on with xarray.

There is just one extra step to do so. After importing esmtools, you have to do add the module call after ds and then the function. For example, you can call ds.grid.convert_lon() to transform between -180 to 180 longitudes and 0 to 360 longitudes.

List of currently supported modules/functions. See the API for usage.

1. grid
   • convert_lon()

import esmtools
import numpy as np
import xarray as xr

lat = np.linspace(-89.5, 89.5, 180)
lon = np.linspace(0.5, 359.5, 360)
empty = xr.DataArray(np.empty((180, 360)), dims=['lat', 'lon'])
data = xr.DataArray(np.linspace(0, 360, 360), dims=['lon'],)
data, _ = xr.broadcast(data, empty)
data = data.T
data['lon'] = lon
data['lat'] = lat

print(data)

<xarray.DataArray (lat: 180, lon: 360)>
array([[  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ],
       [  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ],
       [  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ],
       ...,
       [  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ],
       [  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ],
       [  0.        ,   1.00278552,   2.00557103, ..., 357.99442897,
        358.99721448, 360.        ]])
Coordinates:
  * lon      (lon) float64 0.5 1.5 2.5 3.5 4.5 ... 355.5 356.5 357.5 358.5 359.5
  * lat      (lat) float64 -89.5 -88.5 -87.5 -86.5 -85.5 ... 86.5 87.5 88.5 89.5

Our sample data is just a plot of longitude.

data.plot(x='lon', y='lat')

<matplotlib.collections.QuadMesh at 0x7f71b70ebf98>

However, it ranges from 0 to 360, which is sometimes problematic. We can use the accessor convert_lon() to convert this to -180 to 180.

help(data.grid.convert_lon)

Help on method convert_lon in module esmtools.accessor:

convert_lon(coord='lon') method of esmtools.accessor.GridAccessor instance
    Converts longitude grid from -180to180 to 0to360 and vice versa.

    .. note::
        Longitudes are not sorted after conversion (i.e., spanning -180 to 180 or
        0 to 360 from index 0, ..., N) if it is 2D.

    Args:
        ds (xarray object): Dataset to be converted.
        coord (optional str): Name of longitude coordinate, defaults to 'lon'.

    Returns:
        xarray object: Dataset with converted longitude grid.

    Raises:
        ValueError: If ``coord`` does not exist in the dataset.

converted = data.grid.convert_lon(coord='lon')

Now we’ve switched over to the -180 to 180 coordinate system.

converted

<xarray.DataArray (lat: 180, lon: 360)>
array([[180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724],
       [180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724],
       [180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724],
       ...,
       [180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724],
       [180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724],
       [180.50139276, 181.50417827, 182.50696379, ..., 177.49303621,
        178.49582173, 179.49860724]])
Coordinates:
  * lon      (lon) float64 -179.5 -178.5 -177.5 -176.5 ... 177.5 178.5 179.5
  * lat      (lat) float64 -89.5 -88.5 -87.5 -86.5 -85.5 ... 86.5 87.5 88.5 89.5

converted.plot(x='lon', y='lat')

<matplotlib.collections.QuadMesh at 0x7f71b71404a8>

This is equivalent to running the functional convert_lon() argument:

converted = esmtools.grid.convert_lon(data, coord='lon')

converted.plot(x='lon', y='lat')

<matplotlib.collections.QuadMesh at 0x7f71b47d2470>

Help & Reference

• API Reference
• Contribution Guide
• Changelog History
• Release Procedure
• Contributors
• Additional Packages

API Reference

This page provides an auto-generated summary of esmtools’s API. For more details and examples, refer to the relevant chapters in the main part of the documentation.

Carbon

from esmtools.carbon import ...

Functions related to analyzing ocean (and perhaps terrestrial) biogeochemistry.

calculate_compatible_emissions(…)	Calculate compatible emissions.
co2_sol(t, s)	Compute CO2 solubility per the equation used in CESM.
get_iam_emissions()	Download IAM emissions from PIK website.
plot_compatible_emissions(…[, …])	Plot combatible emissions.
potential_pco2(t_insitu, pco2_insitu)	Calculate potential pCO2 in the interior ocean.
schmidt(t)	Computes the dimensionless Schmidt number.
spco2_sensitivity(ds)	Compute sensitivity of surface pCO2 to changes in driver variables.
spco2_decomposition_index(ds_terms, index[, …])	Decompose oceanic surface pco2 in a first order Taylor-expansion.
spco2_decomposition(ds_terms[, detrend, …])	Decompose oceanic surface pco2 in a first order Taylor-expansion.
temp_decomp_takahashi(ds[, time_dim, …])	Decompose surface pCO2 into thermal and non-thermal components.

Composite Analysis

from esmtools.composite import ...

Functions pertaining to composite analysis. Composite analysis takes the mean view of some field (e.g., sea surface temperature) when some climate index (e.g., El Nino Southern Oscillation) is in its negative or positive mode.

composite_analysis(field, index[, …])

Create composite maps based on some variable’s response to a climate index.

Conversions

from esmtools.conversions import ...

Functions related to unit conversions.

convert_mpas_fgco2(mpas_fgco2)

Convert native MPAS CO2 flux (mmol m-3 m s-1) to (molC m-2 yr-1)

Grid Tools

from esmtools.grid import ...

Functions related to climate model grids.

convert_lon(ds[, coord])

Converts longitude grid from -180to180 to 0to360 and vice versa.

Physics

from esmtools.physics import ...

Functions related to physics/dynamics.

stress_to_speed(x, y)

Convert ocean wind stress to wind speed at 10 m over the ocean.

Spatial

from esmtools.spatial import ...

Functions related to spatial analysis.

extract_region(ds, xgrid, ygrid, coords[, …])	Extract a subset of some larger spatial data.
find_indices(xgrid, ygrid, xpoint, ypoint)	Returns the i, j index for a latitude/longitude point on a grid.

Statistics

from esmtools.stats import ...

Functions dealing with statistics.

ACF
autocorr(ds[, dim, nlags])	Compute the autocorrelation function of a time series to a specific lag.
corr(x, y[, dim, lead, return_p])	Computes the Pearson product-moment coefficient of linear correlation.
linear_slope(x[, y, dim, nan_policy])	Returns the linear slope with y regressed onto x.
linregress(x[, y, dim, nan_policy])	Vectorized applciation of scipy.stats.linregress.
polyfit(x[, y, order, dim, nan_policy])	Returns the fitted polynomial line of y regressed onto x.
nanmean(ds[, dim])	Compute mean of data with NaNs and suppress warning from numpy.
rm_poly(x[, y, order, dim, nan_policy])	Removes a polynomial fit from y regressed onto x.
rm_trend(x[, y, dim, nan_policy])	Removes a linear fit from y regressed onto x.
standardize(ds[, dim])	Standardize Dataset/DataArray

Temporal

from esmtools.temporal import ...

Functions related to time.

to_annual(ds[, calendar, how, dim])

Resample sub-annual temporal resolution to annual resolution with weighting.

Testing

from esmtools.testing import ...

Functions specifically focused on statistical testing.

multipletests(p[, alpha, method])	Apply statsmodels.stats.multitest.multipletests for multi-dimensional xr.objects.
ttest_ind_from_stats(mean1, std1, nobs1, …)	Parallelize scipy.stats.ttest_ind_from_stats and make dask-compatible.

Contribution Guide

Contributions are highly welcomed and appreciated. Every little help counts, so do not hesitate! You can make a high impact on esmtools just by using it and reporting issues.

The following sections cover some general guidelines regarding development in esmtools for maintainers and contributors. Nothing here is set in stone and can’t be changed. Feel free to suggest improvements or changes in the workflow.

Contribution links

• Contribution Guide
  • Feature requests and feedback
  • Report bugs
  • Fix bugs
  • Write documentation
  • Preparing Pull Requests

Feature requests and feedback

We are eager to hear about your requests for new features and any suggestions about the API, infrastructure, and so on. Feel free to submit these as issues with the label “feature request.”

Please make sure to explain in detail how the feature should work and keep the scope as narrow as possible. This will make it easier to implement in small PRs.

Report bugs

Report bugs for esmtools in the issue tracker with the label “bug”.

If you are reporting a bug, please include:

• Your operating system name and version.
• Any details about your local setup that might be helpful in troubleshooting, specifically the Python interpreter version, installed libraries, and esmtools version.
• Detailed steps to reproduce the bug.

If you can write a demonstration test that currently fails but should passm that is a very useful commit to make as well, even if you cannot fix the bug itself.

Fix bugs

Look through the GitHub issues for bugs.

Talk to developers to find out how you can fix specific bugs.

Write documentation

esmtools could always use more documentation. What exactly is needed?

• More complementary documentation. Have you perhaps found something unclear?
• Docstrings. There can never be too many of them.

You can also edit documentation files directly in the GitHub web interface, without using a local copy. This can be convenient for small fixes.

Our documentation is written in reStructuredText. You can follow our conventions in already written documents. Some helpful guides are located here and here.

Note

Build the documentation locally with the following command:

$ conda env update -f ci/environment-dev-3.6.yml
$ cd docs
$ make html

The built documentation should be available in the docs/build/.

If you need to add new functions to the API, add the functions to api.rst then run sphinx-autogen -o api api.rst from the docs/source directory. You might need to run make clean from the docs/ directory and then make html again to get the links to build properly.

Preparing Pull Requests

1. Fork the esmtools GitHub repository. It’s fine to use esmtools as your fork repository name because it will live under your user.

2. Clone your fork locally using git, connect your repository to the upstream (main project), and create a branch:

   $ git clone git@github.com:YOUR_GITHUB_USERNAME/esmtools.git
   $ cd esmtools
   $ git remote add upstream git@github.com:bradyrx/esmtools.git
   
   # now, to fix a bug or add feature create your own branch off "master":
   
   $ git checkout -b your-bugfix-feature-branch-name master
   

   If you need some help with Git, follow this quick start guide: https://git.wiki.kernel.org/index.php/QuickStart

3. Install dependencies into a new conda environment:

   $ conda env update -f ci/environment-dev-3.7.yml
   $ conda activate esmtools-dev
   

4. Make an editable install of esmtools by running:

   $ pip install -e .
   

5. Install pre-commit and its hook on the esmtools repo:

   $ pip install --user pre-commit
   $ pre-commit install
   

   Afterwards pre-commit will run whenever you commit.

   https://pre-commit.com/ is a framework for managing and maintaining multi-language pre-commit hooks to ensure code-style and code formatting is consistent.

   Now you have an environment called esmtools-dev that you can work in. You’ll need to make sure to activate that environment next time you want to use it after closing the terminal or your system.

   You can now edit your local working copy and run/add tests as necessary. Please follow PEP-8 for naming. When committing, pre-commit will modify the files as needed, or will generally be quite clear about what you need to do to pass the commit test.

6. Break your edits up into reasonably sized commits.

   $ git commit -a -m “<commit message>” $ git push -u

7. Run all the tests

   Now running tests is as simple as issuing this command:

   $ coverage run --source esmtools -m py.test
   

   This command will run tests via the “pytest” tool against Python 3.6.

8. Create a new changelog entry in CHANGELOG.rst:

   • The entry should be entered as:

   <description> (:pr:`#<pull request number>`) `<author's names>`_

   where <description> is the description of the PR related to the change and <pull request number> is the pull request number and <author's names> are your first and last names.

   • Add yourself to list of authors at the end of CHANGELOG.rst file if not there yet, in alphabetical order.

1. Add yourself to the contributors <https://esmtools.readthedocs.io/en/latest/contributors.html>_ list via docs/source/contributors.rst.

2. Finally, submit a pull request through the GitHub website using this data

   ..code::

   head-fork: YOUR_GITHUB_USERNAME/esmtools compare: your-branch-name

   base-fork: bradyrx/esmtools base: master

Note that you can create the Pull Request while you’re working on this. The PR will update as you add more commits. esmtools developers and contributors can then review your code and offer suggestions.

Changelog History

esmtools v1.1.4 (2020-##-##)

Bug Fixes

• to_annual() no longer returns 0.0 where the original dataset had NaNs. (GH#75) (GH#95) Riley X. Brady.

esmtools v1.1.3 (2020-07-17)

Bug Fixes

• Revert to old esmtools behavior for stats functions. This allows one to pass single Datasets and DataArrays to linear_slope, linregress, polyfit, rm_poly, and rm_trend. In this case, the fit is performed over dim from the given xarray object. This still retains the e.g. rm_trend(x, y) behavior as well. (GH#93) Riley X. Brady.

Internals/Minor Fixes

• Update required xarray version to v0.16.0 to allow for use of xr.infer_freq. (GH#92) Riley X. Brady.

esmtools v1.1.2 (2020-07-09)

Internals/Minor Fixes

• Fix flake8 F401 error by using TimeUtilAccessor directly in first instance in code. (GH#86) Riley X. Brady.
• Add conda badge and conda installation instructions. (GH#87) Riley X. Brady.
• Migrate corr and autocorr from climpred to esmtools with some light edits to the code. (GH#88) Riley X. Brady.

Deprecated

• climpred removed as a dependency for esmtools. (GH#88) Riley X. Brady.
• autocorr deprecated, since it can be run via corr(x, x). ACF renamed to autocorr, which reflects pandas-style naming. (GH#88) Riley X. Brady.

esmtools v1.1.1 (2020-07-08)

Features

• xarray implementation of statsmodels.stats.multitest.multipletests. (GH#71) Aaron Spring
• Implements nan_policy=... keyword for linear_slope(), linregress(), polyfit(), rm_poly(), rm_trend(). (GH#70) Riley X. Brady.
  • 'none', 'propagate': Propagate nans through function. I.e., return a nan for a given grid cell if a nan exists in it.
  • 'raise': Raise an error if there are any nans in the datasets.
  • 'drop', 'omit': Like skipna, compute statistical function after removing nans.
• Adds support for datetime axes in linear_slope(), linregress(), polyfit(), rm_poly(), rm_trend(). Converts datetimes to numeric time, computes function, and then converts back to datetime. (GH#70)`Riley X. Brady`_.
• linear_slope(), linregress(), polyfit(), rm_poly(), rm_trend() are now dask-compatible and vectorized better. (GH#70) Riley X. Brady.

Bug Fixes

• Does not eagerly evaluate dask arrays anymore. (GH#70) Riley X. Brady.

Internals/Minor Fixes

• Adds isort and nbstripout to CI for development. Blacken and isort code. (GH#73) Riley X. Brady

Documentation

• Add more robust API docs page, information on how to contribute, CHANGELOG, etc. to sphinx. (GH#67) Riley X. Brady.

Deprecations

• Removes mpas and vis modules. The former is better for a project-dependent package. The latter essentially poorly replicates some of proplot functionality. (GH#69) Riley X. Brady.
• Removes stats.smooth_series, since there is an easy xarray function for it. (GH#70) Riley X. Brady.
• Changes stats.linear_regression to stats.linregress. (GH#70) Riley X. Brady.
• Changes stats.compute_slope to stats.linear_slope. (GH#70) Riley X. Brady.
• Removes stats.area_weight and stats.cos_weight since they are available through xarray. (GH#83) Riley X. Brady.

esmtools v1.1 (2019-09-04)

Features

• co2_sol and schmidt now can be computed on grids and do not do time-averaging (GH#45) Riley X. Brady.
• temp_decomp_takahashi now returns a dataset with thermal/non-thermal components (GH#45) Riley X. Brady.
• temporal module that includes a to_annual() function for weighted temporal resampling (GH#50) Riley X. Brady.
• filtering module renamed to spatial and find_indices made public. (GH#52) Riley X. Brady.
• standardize function moved to stats. (GH#52) Riley X. Brady.
• loadutils removed (GH#52) Riley X. Brady.
• calculate_compatible_emissions following Jones et al. 2013 (GH#54) Aaron Spring
• Update corr to broadcast x and y such that a single time series can be correlated across a grid. (GH#58) Riley X. Brady.
• convert_lon_to_180to180 and convert_lon_to_0to360 now wrapped with convert_lon and now supports 2D lat/lon grids. convert_lon() is also available as an accessor. (GH#60) Riley X. Brady.

Internals/Minor Fixes

• Changed name back to esmtools now that the readthedocs domain was cleared up. Thanks Andrew Walter! (GH#61) Riley X. Brady.
• esmtools documentation created with docstring updates for all functions.

esm_analysis v1.0.2 (2019-07-27)

Internals/Minor Fixes

• Changed name from esmtools to esm_analysis since the former was registered on readthedocs.

esmtools v1.0.1 (2019-07-25)

Internals/Minor Fixes

• Add versioning and clean up setup file.
• Add travis continuous integration and coveralls for testing.

esmtools v1.0.0 (2019-07-25)

Formally releases esmtools on pip for easy installing by other packages.

Release Procedure

We follow semantic versioning, e.g., v1.0.0. A major version causes incompatible API changes, a minor version adds functionality, and a patch covers bug fixes.

1. Create a new branch release-vX.x.x with the version for the release.

• Update CHANGELOG.rst
• Make sure all new changes, features are reflected in the documentation.

1. Open a new pull request for this branch targeting master

2. After all tests pass and the PR has been approved, merge the PR into master

3. Tag a release and push to github:

   $ git tag -a v1.0.0 -m "Version 1.0.0"
   $ git push origin master --tags
   

4. Build and publish release on PyPI:

   $ git clean -xfd  # remove any files not checked into git
   $ python setup.py sdist bdist_wheel --universal  # build package
   $ twine upload dist/*  # register and push to pypi
   

5. Update the stable branch (used by ReadTheDocs):

   $ git checkout stable
   $ git rebase master
   $ git push -f origin stable
   $ git checkout master
   

6. Update esmtools conda-forge feedstock

• Fork esmtools-feedstock repository

• Clone this fork and edit recipe:

  $ git clone git@github.com:username/esmtools-feedstock.git
  $ cd esmtools-feedstock
  $ cd recipe
  $ # edit meta.yaml
  

• Update version
• Get sha256 from pypi.org for esmtools
• Fill in the rest of information as described here

• Commit and submit a PR

Contributors

Core Developers

• Riley X. Brady (github)

Contributors

• Aaron Spring (github)

For a list of all the contributions, see the github contribution graph.

Additional Packages

esmtools is a kitchen sink for various xarray wrappers related to Earth System Model analysis. It serves to fill in the gaps between specialized packages, but does not intend to reinvent the wheel. Here is a list of helpful xarray-based packages that I have found useful in my analyses:

• climpred : Analysis of initialized Earth System Model forecasts.
• eofs : Compute empirical orthogonal functions (EOFs) for xarray objects.
• regionmask : Helps with creating regional masks in xarray objects.
• xESMF : Regrid xarray output using the ESMF engine.
• xrft : Fourier transforms for xarray.
• xskillscore : Various skill score and bias metrics for xarray.