Allow discontinuity in operational variables between timeslices

[DillonJ]

This relates to #1237

@aporrasc is trying to set up an investment planning model. He is trying to create a model that consists of multiple snapshot years in a single optimisation. He is trying to do this with temporal blocks representing each snapshot year. One for investments, and one for operations, for each snapshot year. E.g:
TB_Ops_2030: block_start = 01/01/2030, block_end = 01/01/2031, resolution: 12h
TB_Ops_2040: block_start = 01/01/2040, block_end = 01/01/2041, resolution: 12h
TB_Investments_2030: block_start = 01/01/2030, block_end = 01/01/2031, resolution: 1Y
TB_Investments_2040: block_start = 01/01/2040, block_end = 01/01/2041, resolution: 1Y

[@aporrasc I'm leaving out the 10Y thing as I think it's a bit of a complication, doing it as above reduces the problem to the single issue of operational variable continuity]

The problem is that we need to free up the node state variables between the two operational temporal blocks. In this case, we don't want continuity between the last timeslice of 2030 and the first of 2040. We wants a free history timeslice just before 2040 to free up the node state variables (and other variables) so the 2040 operational variables have a "fresh start". Because we are solving this as a single optimisation, we only have free variables preceding 2030 but not before 2040. We don't want the model to roll because we want the investment decisions to see all the snapshot years at once.

For example, the cyclic condition constraint for the 2030 TB will constrain the node_state in the last timeslice of 2030 relative to the last history timeslice (timslice immediately preceding 2030). But for the 2040 TB, cyclic condition will constrain the node_state of last timeslice of 2040 relative to the last timeslice of 2030 creating a constraint that will overly constraint investments and operations. We effectively want a free variable in between the end of 2030 and the beginning of 2040 to free up the operational variables so there is no continuity between the 2030 operational variables and the 2040 operational variables (but we do want continuity for the investment variables).

An idea I had was to have a couple of TB parameter like "operational_variables_free" and "investment_variables_free" or something like that. Then you can insert such a TB in between 2030 and 2040 and it would create a sort of middle history window. You would just need to deactivate constraints and bounds, just like the history variables... but only for operational variables.

This is just an idea, maybe someone can think of a better way? If there was an easy way to create some timeslices in the intervening period and include them in the history timeslices collection, would that do it? Does anyone have other ideas how this could be done relatively easily?

@datejada @Tasqu @nnhjy @nhniina @gnawin @manuelma

[DillonJ]

Regarging a middle history window that is created based on a TB with some parameters set, we have the issue of wanting to be selective - I.e. freeing up the operational variables between TBs but preserving continuity for the investment variables. One way of doing this would be to define the TBs as Alvaro initially had them, i.e. operational TBs of 1 year with a 9 year gap and consecutive investment TBs of 10Y duration with no gap.

[Mastomaki]

If I understand the problem correctly, I already did this in the elexia branch. There the temporal blocks are disconnected and history time slices are created for each temporal block. But there was no parameter to control this.

[Tasqu]

For the immediate problem with the cyclic_condition for storage dynamics and the issues arising from t_before_t at the start of a disjointed temporal_block, we might have a relatively simple fix.

If cyclic_condition=true, I think it's pretty clear what the user wants and the current behaviour is practically a bug. In this case, I don't think we need to consider the time gap at all. We might be able to force t_before_t before the first time slice of the cyclic temporal_block to map to its last time slice. If we're lucky, this will also handle more complex cases like minimum startup and shutdown times or connection delays that look further past the disjoint.

As for the implementation of the above suggestion, my main worry is whether the temporal_block information is readily available when deducing when to apply the cyclic_condition. If I remember correctly, the time slices are a bit "jumbled up", making the above changes more difficult than it might seem at first glance. The challenge is essentially if we always know when to loop and when not to.

For defining different behaviours for the disjoints, I'm not sure if we want or need more temporal_blocks. We could introduce a parameter for the node__temporal_block etc. relationships to define how we want the time slices to interact outside themselves. E.g. set it to connect to earlier blocks if able by default (what it currently tries), declare the cyclic_condition, or declare to generate new history like @Mastomaki did in the elexia branch. While doable in principle, I fear this would require renovating the temporal structures in the code by quite a bit.

[aporrasc]

Actually, there are more issues here.

I would like to use the same initial_node_state for each operational temporal block. I understand that initial_X parameters should initialize variables, this should be a proxy for SpineOpt when there is no history preceding the temporal block.

[aporrasc]

Here is the current version of the model. This is feasible for me.

https://vttgroup.sharepoint.com/sites/EUESIMopo/_layouts/15/download.aspx?UniqueId=35e37c598a80430993020b347e768c63&e=J0BfUR

[manuelma]

I think the best is to introduce the concept of subwindow so that a single optimisation window (rolling or non-rolling) can have many disjoint subwindows, represented by different temporal_blocks. This could be implemented via a temporal_block parameter such as is_subwindow, false by default, that when set to true indicates that this block is a subwindow that needs its own history and doesn't need to fill gaps to the following blocks.

So in @aporrasc model, we would set is_subwindow to true for both operations temporal_blocks. The result will be that the model will treat them as subwindows, so it will create a history of the appropriate size for both blocks, and won't bridge gaps in between them. Note that since constraints are never created for history time slices, then we will effectively have a free start for the operational variables in the second block, as required.

[tarskul]

@manuelma sounds good. Just want to add that I'm wondering if we can make a more general structure. What if we would introduce a relationship between temporal blocks? That could open up some possibility of combining different temporal blocks. It may also be more intuitive for the user when they can see how the temporal blocks are related. They can specify in those relations whether the subsequent blocks are continuous or discontinuous or maybe even discontinuous in cyclic constraint but continuous in other constraints. But I'm unsure myself, it could also backfire and make it more confusing. What do you think?

[manuelma]

If you ask me, the requirement is better satisfied with an object parameter as I propose. Correct me if I'm wrong, but all we are trying to do is to be able to express that a block needs a fresh start (which we would implement by giving it its own history). So it is an absolute property of the block, not a relationship property between the block and something else.
Regarding the parameter name, I proposed is_subwindow but it could also be is_snapshot or has_free_start or whatever is clearer.
Finally, regarding expanding the requirement, that's fine with me but we need to finalize it fully before designing the solution—otherwise we would be implementing things "just in case", which is generally discouraged.

[datejada]

Thanks, @manuelma. I think it is a good idea to have a parameter. I was wondering if it is "easy" to detect is_subwindow = true automatically. It seems that it should be possible, but it might not be easy in the code. What do you think? If not, we need to clarify in the documentation that this parameter is needed for those situations and that the user must add it as part of their input data.

[manuelma]

It's a bit tricky, because at the moment the default behaviour in case of disjoint blocks is to "glue" them together, or more specifically, to bridge the gaps in between so the last time slice before the gap and the first one after the gap are still, somehow, considered consecutive (as if the gap was some sort of weird void). Not sure why we did it that way, anyone recalls?
I think if we wanna keep it that way, we probably need to document it better too, with examples and all that, if that's not the case already.
But then, if I understand correctly, your idea @datejada is to change that default and assume that in the presence of disjointed blocks, what the user wants is a snapshot or subwindow, that is a block with a free start.
Personally I'd prefer to keep the default and use the parameter is_subwindow, exactly as I propose above, but of course I'm open.

[tarskul]

That is why I felt that is was more of a relationship parameter where you could indicate whether it is supposed to be glued or not ;) The default behaviour would then be simply whatever the relationships that the user has chosen instead of assuming that everything is glued.

But I do understand that the is_subwindow parameter of the block is currently more practical (although I think I would prefer a different name that is more clear, disjoint_block since we use temporal_block and not temporal_window and indicate that this particular block is disjoint? dunno). I would also agree, for the sake of the current users, that it is better to keep the default behaviour the same as much as possible.

[datejada]

@manuelma Thanks for the explanation, and I agree with your proposal. Regarding the naming, it seems to me that @tarskul proposal disjoint_block is closer to the current SpineOpt jargon, but, maybe is_disjoint_block to keep the boolean meaning. In any case, the documentation should have a section where the default behavior is shown + and an explanation of the changes in the model when the feature of is_disjoint_block (or is_subwindow) is set to True.