-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Onboard NOAA dataset #353
Conversation
int_date_list: typing.List[str], | ||
gen_location_list: dict, | ||
) -> None: | ||
if pipeline_name == "GHCND by year": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps "GHCND by year" can be defined as a constant at the top of this file (or better yet, read from a yaml file). Same goes for other similar cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What added value would that bring to the code? What would the variable be called?
datasets/noaa/pipelines/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
datasets/noaa/pipelines/_images/run_csv_transform_kub/csv_transform.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just fix the indentation on the JSON files and we're good to go.
@adlersantos Fixes are complete as per your request. Testing showed one other issue which is now resolved. Please Review. Thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
depends_on_past: False | ||
start_date: '2021-03-01' | ||
max_active_runs: 1 | ||
schedule_interval: "0 1 0 0 6" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add a comment for clarity.
schedule_interval: "0 1 0 0 6" | |
schedule_interval: "0 1 0 0 6" # Every Saturday at 1am server time |
Description
ghcnd_by_year
ghcnd_countries
ghcdn_inventory
ghcnd_states
ghcnd_stations
gsod_stations
ghcnd_hurricanes
lightning_strikes_by_year
Checklist
datasets/noaa
and nothing outside of that directory