enh(add_existing_baseyear): Pull out valid_grouping_years algorithm from heat

[coroa]

Changes proposed in this Pull Request

Minimal refactoring to re-use the grouping_years validation also for power. And fix a wrong indent.

Checklist

• I tested my contribution locally and it works as intended.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to envs/environment.yaml.
• Changes in configuration options are added in config/config.default.yaml.
• Changes in configuration options are documented in doc/configtables/*.csv.
• Sources of newly added data are documented in doc/data_sources.rst.
• A release note doc/release_notes.rst is added.

[FabianHofmann]

I am a bit confused about this expression here, this can be a negative number or? If grouping_year[0] == 1990 and baseyear == 2020 and lifetime == 20 for example.

[coroa]

I must admit, i did not think about this. It is a literal port of Amos':

# Fill NA from .diff() with value for the first interval
_years[0] = valid_grouping_years[0] - baseyear + default_lifetime

[coroa]

Ok, sorry, Amos, i was wrongly blaming you. The origin story of that expression is complicated.

@lisazeyen came up with:
https://github.com/PyPSA/pypsa-eur/blame/a72388b989d1d667ec7e44d66f6c3b494b46d000/scripts/add_existing_baseyear.py#L466-L467

         _years = (valid_grouping_years.diff().shift(-1)
                  .fillna(baseyear-valid_grouping_years.iloc[-1]))

ie. (baseyear - last of grouping_years) which sounds quite sensible (for an interval, unless the last grouping year is the same as the baseyear).

and then @lindnemi changed it to:

# Fill NA from .diff() with value for the first interval
_years[0] = valid_grouping_years[0] - baseyear + default_lifetime

in #1102 with associated discussion in #1091 .

[coroa]

It sounds like there was some back and forth, i'll not try to understand this fully now, maybe its best to have a short discussion.

[amos-schledorn]

Sorry for the late reply @coroa! I think I only made some small refactoring changes here. I'll be OOO for a week but happy to discuss this further then, if helpful.

[amos-schledorn]

I haven't really worked with this, so I'm a little out of my comfort zone. I'd just suggest a function docstring for valid_grouping_years. Otherwise happy if @FabianHofmann is happy.

[fneum]

Yes, as you say I remember this was quite a delicate issue #1019 #1091 #1102.

I would let @lindnemi review this.

[coroa]

@lindnemi if it makes sense, we can also have a short call next week to try to see what to do with it, and what would be the ideal aspiration.

[lindnemi]

Sorry for the late reply, I totally missed the mentions. If you need some urgent feedback better ping me on discord.

Great to see you want to improve the grouping years, here is a bit of context on how the grouping_years work

1. Basic idea is: A grouping year, groups the powerplants built in the time period before, i.e., 2006-2010 -> 2010
2. Special case: grouping_year == baseyear. In this case the resulting component, e.g. solar-2020 represents both, the historical capacities built in 2016-2020 AND the additional capacities the model chooses to build in the optimization for the planning_horizon=2020. This implies that for such components p_nom_extendable will be true.
   1. For the power sector, the historical capacities are written into p_nom and then enforced by setting p_nom_min
   2. However, for the heating sector, the historical capacities are only written into p_nom and not into p_nom_min. This means the historical capacities would be ignored during optimization (because of p_nom_extendable they can be overwritten). To avoid that, for the heating sector, grouping_years always have to be smaller than the baseyear.
   3. This means, for the power sector grouping_years <= baseyear, while for the heating sector grouping_years < baseyear
3. Regarding the concern of @FabianHofmann that _yearsmight be a negative number: In my original code I had the requirement grouping_year > baseyear - default_lifetime, s.t. the _years variable is always positive. This PR changes it to grouping_year >= baseyear - default_lifetime which means that an interval of 0 is possible and that some generators with p_nom=0 could be added to the model. However, still no negative _years should occur.

[lindnemi]

For a proper refactor I would suggest to unify grouping_years_heat and grouping_years_power from the config, and to unify how existing capacities are treated (currently by overwriting p_nom_min/excluding the baseyear). I think ideally we would distinguish historical components from components that are active during optimization, possibly by adding historical to the name of the components.

[coroa]

Hi @lindnemi , funnily i am now also coming back with only quite a delay to this PR.

Thanks for your special case description, ie point 2, i was not aware of this tricky difference.

What do you think are the benefits of splitting the extendable capacity from the fixed capacity (with a historical suffix) in comparison to the p_nom/p_nom_min combination?

[coroa]

And thanks for point 3. That is indeed a mistake.

[lindnemi]

Hi @lindnemi , funnily i am now also coming back with only quite a delay to this PR.

Thanks for your special case description, ie point 2, i was not aware of this tricky difference.

What do you think are the benefits of splitting the extendable capacity from the fixed capacity (with a historical suffix) in comparison to the p_nom/p_nom_min combination?

Only benefit is that the distinction between historical assets and expanded assets would be clearer, but on second thought it's clear enough already, so let's just leave it as is. The harmonization between electricity and heating sector would have more benefits.