cmaqsatproc package

Subpackages

Submodules

cmaqsatproc.cmaq module

cmaqsatproc.cmaq.open_griddesc(GDNAM, gdpath=None)[source]

Create an xarray Dataset that defines a CMAQ grid using GRIDDESC

Parameters:

  • GDNAM (str) – Name of grid as defined in GRIDDESC file

  • gdpath (str) – Path to GRIDDESC file. If None (default), use contents of default_griddesc_txt instead. default_griddesc_txt can be modified directly, although that is discouraged.

Returns:

gf – File with coordinates based on GDNAM in gdpath

Return type:

xarray.Dataset

cmaqsatproc.cmaq.open_ioapi(path, **kwargs)[source]

Open an IOAPI file in NetCDF format using xarray and construct coordinate variables. (time based on TFLAG or properties, ROW/COL in projected space, and LAY based on VGTYP, VGLVLS, and VGTOP)

Parameters:

  • path (str) – Path to the IOAPI file in NetCDF format

  • kwargs (mappable) – Passed to xr.open_dataset(path, **kwargs)

Returns:

qf – File with data and coordinates based on path

Return type:

xarray.Dataset

cmaqsatproc.utils module

cmaqsatproc.utils.EasyDataFramePoint(df, xkey='cn_x', ykey='cn_y')[source]

Thin wrapper on geopandas.points_from_xy. Create points from a rows with centers named xkey and ykey.

Parameters:

df (pandas.DataFrame) – Must contain x, y

Returns:

points – List of shapely.geometry.Polygons

Return type:

list

cmaqsatproc.utils.EasyDataFramePolygon(df, wrap=True, progress=False, lowmem=False)[source]

Create polygons from a row with corners of a pixel specificied using columns ll_x, ll_y … uu_x, uu_y.

The wrap functionality prevents polygons from straddling the dateline.

Parameters:

  • df (pandas.DataFrame) – Must contain ll, lu, uu, ul for x and y (e.g., ll_x, ll_y)

  • wrap (bool) – If True (default), each polygon that crosses the dateline will be truncated to the Western portion.

Returns:

polys – List of shapely.geometry.Polygons

Return type:

list

cmaqsatproc.utils.cdconvert(inval, inunit, outunit)[source]

Converts between du, mole m**-2, and molecules cm**-2

cmaqsatproc.utils.csp_version()[source]
cmaqsatproc.utils.getcmrgranules(temporal, bbox=None, poly=None, verbose=0, **kwds)[source]

Return all links from the Common Metadata Repository for the product granules with short_name, date_range, and optionally a bounding box.

Parameters:

  • temporal (str) – NASA temporal that is recognized by NASA Common Metadata Repository i.e, YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SSZ/YYYY-MM-DDTHH:MM:SSZ or YYYY-MM-DDTHH:MM:SSZ/P01D (or P01M or P01Y) etc.

  • bbox (list) – Longitude/latitude bounding edges as floats (wlon,slat,elon,nlat)

  • poly (shapely.geometry.Polygon) – The exterior will be converted to floats and ordered x1,y1,…,xN,yN

  • filterfunc (function) – Takes a link dictionary from CMR and returns True if it should be retained

  • concept_id (str) – Optional, NASA concept_id that is the unique identifier for a collection in NASA’s Common Metadata Repository

  • short_name (str) – Optional, short_name identifier for a collection that is often, but not always unique in the NASA Common Metadata Repository. If you have the short_name and want the concept_id put the url below in your browser where you replace OMNO2 (the example) with your short_name https://cmr.earthdata.nasa.gov/search/collections?short_name=OMNO2

Returns:

jr – json result as a dictionary

Return type:

dictionary

Return all links from the Common Metadata Repository for the product granules with *args and **kwds are passed thru to getcmrgranules

Parameters:

  • *args – See getcmrgranuels.

  • **kwds – See getcmrgranuels.

  • filterfunc (function) – Takes a link dictionary from CMR and returns True if it should be retained

Returns:

links – List of all links where filterfunc is None or just links for which filterfunc returns True.

Return type:

list

cmaqsatproc.utils.grouped_weighted_avg(values, weights, by)[source]
cmaqsatproc.utils.rootremover(strlist, insert=False)[source]

Find the longest common root and replace it with {root}

Parameters:

  • strlist (list) – List of strings from which to find a common root and replace with ‘{root}’

  • insert (bool) – If true, insert f’root: {root}’ at the beginning of the short list.

Returns:

List with each element of strlist where the longest common root has been removed. If insert, then the root is inserted

Return type:

stem, short_list

cmaqsatproc.utils.walk_groups(f, gkey, outputs=None)[source]

Module contents

Overview

cmaqsatproc provides satellite data processing for CMAQ. This has four basic steps:

  1. Create a custom level 3 gridded satellite product on CMAQ grid,

  2. Subset CMAQ consistent with satellite overpass and valid samples,

  3. integrate CMAQ mixing ratios to number density (or other metric), and

  4. apply CMAQ air mass factor to the satellite.

Core Objects

  • open_griddesc : define a CMAQ grid by name or GRIDDESC.

  • reader_dict : dictionary of satellite readers.

  • open_ioapi : Used to open CMAQ and MCIP data.

  • print_reader_list : useful to find out what can be done.

By TropOMI NO2 Example

# Import Libraries import cmaqsatproc as csp

# Define grid, satellite data (by reader), and date to process GDNAM = ‘12US1’ date=’2019-07-24’ readername = ‘TropOMINO2’ # or TropOMIHCHO, VIIRS_AERDB, …

cg = csp.open_griddesc(GDNAM) satreader = csp.reader_dict[readername]

# Custom L3 output path outl3path = f’{readername}_{date}_{GDNAM}.nc’ # CMAQ satellite output outcsppath = f’{readername}_{date}_{GDNAM}_CMAQ.nc’

# Use NASA Common Metadata Repository to find satellite data # and remote access methods. Grid results on CMAQ grid. l3 = satreader.cmr_to_level3(

temporal=f’{date}T00:00:00Z/{date}T23:59:59Z’, bbox=cg.csp.bbox(), grid=cg.csp.geodf, verbose=9

) # Optionally, store custom l3 result l3.to_netcdf(outl3path)

# Collect CMAQ 3D CONC and MCIP data qf = csp.open_ioapi(f’CCTM_CONC_{date}_{GDNAM}.nc’)[[‘NO2’]] mf = csp.open_ioapi(f’METCRO3D_{date}_{GDNAM}.nc’) qf[‘DENS’] = mf[‘DENS’] qf[‘ZF’] = mf[‘ZF’] qf[‘PRES’] = mf[‘PRES’]

# cmaq_process applies temporal subsetting consistent with the satellite # overpass, integrates NO2 to mole density, and applies CMAQ vertical # to create an alternative AMF and “adjusted” satellite column overf = satreader.cmaq_process(qf, l3) overf.to_netcdf(outcsppath)

cmaqsatproc.open_griddesc(GDNAM, gdpath=None)[source]

Create an xarray Dataset that defines a CMAQ grid using GRIDDESC

Parameters:

  • GDNAM (str) – Name of grid as defined in GRIDDESC file

  • gdpath (str) – Path to GRIDDESC file. If None (default), use contents of default_griddesc_txt instead. default_griddesc_txt can be modified directly, although that is discouraged.

Returns:

gf – File with coordinates based on GDNAM in gdpath

Return type:

xarray.Dataset

cmaqsatproc.open_ioapi(path, **kwargs)[source]

Open an IOAPI file in NetCDF format using xarray and construct coordinate variables. (time based on TFLAG or properties, ROW/COL in projected space, and LAY based on VGTYP, VGLVLS, and VGTOP)

Parameters:

  • path (str) – Path to the IOAPI file in NetCDF format

  • kwargs (mappable) – Passed to xr.open_dataset(path, **kwargs)

Returns:

qf – File with data and coordinates based on path

Return type:

xarray.Dataset

cmaqsatproc.print_reader_list()[source]

Print to the screen a short description reader_dict keys and a short description of the reader they are associated with.