v0.14: Reporting power

Version v0.14 of FlexMeasures begins a major upgrade in reporting capability. Also, this version begins work to support scheduling of heat storage. Finally, we added a bunch of developer support features.

See changelog for a complete list or read on below about the larger new features we added.

Reporting

FlexMeasures feeds upon raw measurement data (e.g. solar generation) and data from third parties (e.g. weather forecasts). They come together in the FlexMeasures scheduling algorithms.

However, there are more use cases for enriching these data by combining them:

1. Pre-calculations: E.g. from a tariff and some tax rules we compute the real financial impact of price data.
2. Post-calculations: To be able to show the customer value, we regularly want to compute things like money or CO2 saved.

These calculations can be done with code, but there'll be many repetitions. 

We now added an infrastructure that allows us to define computation pipelines (see an example below) and CLI commands for developers to list available reporters and trigger their computations regularly:

• flexmeasures show reporters
• flexmeasures add report

The reporter classes we are designing are using pandas under the hood and can be sub-classed, allowing us to build new reporters from stable simpler ones, and even pipelines. Remember: re-use is developer power!

We believe this infrastructure will become very powerful and enable FlexMeasures hosters and plugin developers to implement exciting new features.

So here is a glimpse into a reporter we made - it is based on the AggregatorReporter (which is for the combination of any two sensors).

This simplified example reporter basically calculates pv - consumption at grid connection point. This tells us how much solar power we fed back to the grid (positive values) and/or the amount of grid power within the overall consumption which did not come from local solar panels (negative values).

First, this is the configuration of how the computation works:

    {
        "method" : "sum",
        "weights" : {
            "pv" : 1.0,
            "consumption" : -1.0
        }
    }

Second, this parameterizes the computation (fro which sensors does data come from, which range & where does it go):

    {
        "input": [
            {
                "name" : "pv",
                "sensor": 1,
                "source" : 1,
            },
            {
                "name" : "consumption",
                "sensor": 1,
                "source" : 2,
            }
        ],
        "output": [
            {
                "sensor": 3,
            }
        ],
        "start" : "2023-01-01T00:00:00+00:00",
        "end" : "2023-01-03T00:00:00+00:00",
    }

Much more elaborate reporters are possible to be done this way, and we will showcase more later. 

This work was done in Pull Requests 641, 659, 712 and 686. 

P.S. We are not done. You are invited to follow along in the current discussion, where you'll find new ideas like the EnergyBillReporteror the CO2Reporter. We might even support YAML for even higher readability!

Working on heat scheduling: storage efficiency

Optimizing heat storage is our next big goal, after our work towards batteries. 

We have decided to extend the in-built storage scheduling algorithm towards this goal. There might be other approaches for solving some complex heat storage configurations with custom advanced algorithms, but we believe that a majority of cases can be solved in-built.

The first action was to support the buffer-like behavior of heat storage, namely that energy (heat) is being lost over time. This lead to the new parameter storage-efficiency, which is now part of the flex model. So now we can express that a hot water storage tank will lose around 1% of the stored heat per day. This will be reflected in what the scheduler finds to be optimal.

This work was done in Pull Request 679.

Developer support: docs, packaging

We extended the core team that works on FlexMeasures, and that is always an opportunity to spot problems in the support for developers. We added two main improvements:

• Better developer docs, making it easier to install FlexMeasures and understand what it's doing (Pull Requests 664, 665, 676, 687, 698)
• Packaging improvements, making it easier to use FlexMeasures in deployments (Pull Requests: 670, 671)

API sunsetting

Finally, as our long journey to a more flexible datamodel is coming close to completion, we are sunsetting (as we pre-announced in version 0.12) API versions below 3. Please help your users upgrade to API version 3 before you use FlexMeasures v0.14!

While version 0.13 still allowed hosts to get the sunset endpoints back if needed, we now have sunsetted them and they return HTTP status code 410 by default.

We compiled some helpful information for FlexMeasures hosts to manage this transition well.  

This work was done in Pull Request 667.